Blog Archives

vCloud Director – 1.5 RHEL 5 Bug Hot CPU ADD – Quick Work Around

So I was doing some testing in vCloud Director 1.5 and noticed my RHEL Linux 5 vApp wasn’t able to enable Virtual CPU Hot add.

I went in and check my vCenter settings to see what the deal was:

Changing the setting on my vCenter updated it in my vCloud Director..

The alternative to having to do this workaround would be to change the template version within vCloud Director to RHEL version 6

You will notice the Virtual CPU hot add becomes available to check. I used this method on existing templates and it did not seem to break the templates.
However, if you are trying to create new templates of RHEL 6 with RHEL5 5 OS you may want to make sure your SCSI controller is correct. Again, changing it on my vApps seemed to make no impact to my OS currently installed.

It’s apparent bug to vCloud Director and @Lamw was kind enough to help me out.

vSphere – vCloud – The Query 1.5 API “feature” pageSize bug?

Alright, this is going to be difficult for me to really explain so I will do my best to serve it justice. First, I am not a coder and I do not know the ins and outs of the API and code. What I will attempt to explain to you is how you can reproduce this issue on your VCD instance. I also want to note this is vanilla VCD 1.5 with no updates yet. I currently do have a case with VMware opened and I have yet to resolve it.

Let’s get to the nitty gritty.

First off, I want to say that I am not 100% sure that any other queries you use produce the same affect. This issue seems to happen with only the VMadmin query.

First I would recommend reading about connecting the Rest API with Will’s blog over at VMware:

http://blogs.vmware.com/vsphere/2012/03/exploring-the-vcloud-rest-api-part-1.html

Now that you have read that and understand how to connect to the REST API I will show you an example of a basic VMadmin query.
(Note: you need to have over 128 VCD Vapps to reproduce this type of issue)

GET http://vcd.url.com/api/query?type=adminVM

This showed me that I had 333 queries returned however on the 1st page I only found 128. Now the way the script talked to VCD API was rather plain and it was basically doing this query and dumping it to a XML file. The idea was that this was similar to 1.0 API where I could get all the data I wanted and dumped into an XML file. This wasn’t the case. It seems I couldn’t get around this 128 limit. So I decided to try the next query:

Get http://vcd.instance.com/api/query?type=adminVM&pageSize=999

After running it I still got 333 queries returned but only 128 on the single page EVEN after specifying a pageSize=999 so this isn’t the end of it… let’s dig deeper. After further researching I had actually found documented proof that this was a hard setting somewhere.

Page 212 of the VCD 1.5 API Guide taken from here: http://www.vmware.com/pdf/vcd_15_api_guide.pdf

So it became obvious to me at this point that no matter what your query is it would always default to 128 objects per page. So I tried to also do the following to change this hard setting (at the recommendation of someone) located in a global.properties file in the following directory on the vCloud Director cells:

/opt/vmware/vcloud-director/etc

add/change the following: restapi.queryservice.maxPageSize=1024

I added this to the global.properties file and the VCD cells service were also restarted. Can you guess what still happened? Nothing… this didn’t change anything at all. In fact, it still remained broken. Folks, this still wasn’t the worse part about it. Lets cover the part that I believe is a true bug in the API and had someone on Twitter also comment that there is a possible bug in adminVM query.

Lets say I do a query for a pageSize=135 and my query returns 153 results. We get the usual 128 queries per page. Here is an example of the commands I used:

GET https://vcd.url.com/api/query?type=adminVM&pageSize=135&page=1&fields=name&sortAsc=name

Sort ascending gives me an alphabetical sorting of all my vApp names and I can find a Breaking point for my virtual machines (I know my ABC’s and what should be next so to speak). So I copy and paste the results into Notepad++ and it shows me 128 entries of the page size of 135 (give or take a few for other lines returned not relevant to the query. The bug as discussed is evident. However, it doesn’t show the other 7 entries it should be showing. Remember, we did the page size for 135. So now let’s take a peek at page 2.

GET https://vcd.url.com/api/query?type=adminVM&pageSize=135&page=2&fields=name&sortAsc=name

So after you run this query you will the list of the remaining 153 results. However if you take notes you will notice that it is in fact completely missing the 7 other entries. So basically your query takes the 7 it could NOT list and dumps it out to somewhere in the Cloud…. So what does this mean aside from the fact that there is a bug?

You will need to use a looping construct and not specify a page size greater then 128. (see Will’s comments below)

This is a bug and I don’t think I could make it any clearer. I wish I could’ve provided some screenshots but I think if someone does there due diligence they will see what I am talking about. If you have 2000 VCD vApps and you do a page size of 500 you would lose 372 queries between each page. No matter how you specify the page size, modify the Global.properties its just broken plain and simple. If someone would like to provide some screen shots I would be happy to put them up here to show some better detail.

If you want to discuss in further detail feel free to comment and I will follow up.

UPDATE: After reviewing with VMware on some things I found out this is actually a true but with the vCloud 1.5 API bug.  The good news is that there is a fix slated to be published in August, perhaps they will allow for a private fix if you really need it. Stay tuned. If anyone has some information aside from this please provide and I will link it! Thanks again. Also, this is not related to any type of Query parameter this is more to do with how the Query service works.

VMware vSphere Labs – Foundations – Downloads and knowing what you need

This video just covers the basics of what you need to download to get your vSphere Lab up and going. For more this video isn’t useful but it does address understanding Trials, how to access to products, and etc. This is covering a VMware Workstation 8 Lab set up.

Basically here is an overview of what the video covers:

  1. 60 Day Trials
  2. The Value of Partnerships
  3. Understanding the foundation and products for building your lab
  4. What to consider from a product stand point
  5. #VMTNSubscription Movement

***Disclaimer: The thoughts and views expressed on VirtualNoob.wordpress.com and Chad King in no way reflect the views or thoughts of his employer or any other views of a company. These are his personal opinions which are formed on his own. Also, products improve over time and some things maybe out of date. Please feel free to contact us and request an update and we will be happy to assist. Thanks!~

Blah Blah Cloud… part 1

When you look at cloud today in context of VMware what is your biggest concern? For some of us it may be networking, others storage, and maybe even focused in a more broader perspective like; availability, scalability, and BU/DR. Since I have been working with vCloud director day in and day out I have been asking some deeper technical questions centered more around scalability of storage and other related components to the overall design. I have been challenged in various ways because of this technology. Prior to vCloud it was vSphere and a lot of how you implemented and managed vSphere was much less complex. Cloud brings another level of complexity – especially if your initial “design and management” is poor to begin with. Usually you end spending more time and money going back addressing issues related to simple best practices that most Architects and Engineers should already know. In some cases it’s a disconnect between that design and infrastructure team and the help desk. This may not always be the case but in my experience it seems to happen more often than naught.

I am sure we could all spend plenty of time talking about operations, procedures, protocols, standards, and blah blah…. but this isn’t the point of this blog…. Even though these things are of the highest importance and the more effort that is put into this the better the results you will get and the less cost you will end up spending. Anyways…

So, as I was saying vCloud has challenged me in several ways. Now not only do I have to consider the design of vSphere, but I also have to look at the design of vCloud director and how we manage all these different components. Even though you simply add vCloud director still doesn’t mean that is in the end of it all. More complexity comes with integration of other applications, Application availability, and Backup and DR. I have been amazed at how many things I see as an oversight due to the lack of expertise in this area. This is no offense to anyone but really VMware is still in its infancy when running against other markets. Though I strongly sense that VMware is going to be majority market share for a while.

Crossing the gaps:
Since I have been studying and learning day and day out covering VMware best practices and other companies best practices (not VMware) I continue to see a lot of disconnects in certain areas (vCloud Director). Storage guys have no idea or clue about running virtualized workloads on Arrays and often times they care not to even want to learn about VMware. Usually they already have plenty to do but this disconnect on some level will affect the implementation. I honestly say that in most cases the Architect should be the one researching and ensuring that all the components which make up the cloud computing stack should be standardize and implemented correctly, even so these gaps still cause setbacks. Which now leads me into the networking side of things. Networking engineers I see are beginning to come up to speed more quickly on virtualization. The main factor of this is because of Cisco UCS and how it appeals to those network administrators and engineers, and add to that FCoE/CNA’s. However, the disconnect once again lies in that knowledge transfer of the virtual platform of how it works and best practices designed around VMware. I first one to say that many don’t really get the choice especially if a company just threw you into the fire. It’s like right now we are looking at giving our network team the keys to the kingdom (CISCO UCS) but yet they have nearly ZERO understanding and training of how any of it works…. scary right? We have to cross these gaps people we need to make sure that we have people positioned in areas who can understand and impart that training or have someone available as a resource.

My Real Concerns:
vCloud director is something totally new and alien to me when I first stepped in the cloud. I had to learn and quickly. Having my background I quickly go to the manuals, read the blogs, get plugged into good sources, learn even more, read books, and I start auditing. I start looking at designs that may be questionable and start asking the questions of “Is it ignorance” or “What the … was he thinking?” and quickly find that usually it was the latter.. simply ignorance. No one really is to blame because we have to understand YES, it is a NEW technology – BUT how much more critical is it that we research and ensure that we are implementing a design that is “rock” solid before rolling it out… Yes, I know deadlines are deadlines but it is what it is either way. You either spend a lot more money in the long run or spend a little bit more to get it right the first time. We are now having to go back and perform a second phase and for the past couple of months we have been remediating a lot of different things that could’ve been done right had a simple template been designed correctly. We now spend countless additional hours updating and working more issues because of this one simple thing. This isn’t even getting into the storage and other concerns I have.

Cloud and What’s Scary?:
Yeah, I know right scary? I don’t know about yours but some of the ones I have seen are. Here is what scares the heck out of me. ABC customer decides deploy a truck load of Oracle, MSSQL, IIS, Weblogic, and etc Virtual machines all on the fly. Next thing we know we see some latency on the storage back-end and see some impact to performance. Come to find out a bunch of cloning operations are kicking off… I/O is spiking, the VM’s are writing many types of Iops and in a matter of about 12 hours we are having some major issues. This is called “Scalability” or sometimes “Elasticity” whatever you want to call it. Some catalogs host every kind of application and majority of the apps are all tier 1 virtualized workloads. This isn’t the little stuff most corporations virtualize. They usually put this stuff off for later because the need of having a high performance server and old traditional thinking still tells them to not do it (Playing it safe). Scaling a cloud to accommodate tier 1 workloads is going to be something I think we are going to be seeing a lot more. In fact, most vendors provide documentation of implementing solutions on VMware Cloud Director – but they almost NEVER cover the application workloads. I am speaking to Storage, Networking, and Server Hardware. This is probably because in most cases due to the mixed nature you can have in an environment you should do THOROUGH testing to ensure that you can scale out and run an optimal amount of workloads… some would call it vBlock..

Anyways I didn’t mean to write a blog this long but I have just had a lot on my mind lately and I will continue to write more as I continue my VMware Cloud journey.

Cheers,

***Disclaimer: The thoughts and views expressed on VirtualNoob.wordpress.com and Chad King in no way reflect the views or thoughts of his employer or any other views of a company. These are his personal opinions which are formed on his own. Also, products improve over time and some things maybe out of date. Please feel free to contact us and request an update and we will be happy to assist. Thanks!~

Brown Bag Cloud Architecture “Eco-System” Notes

So I checked out the Brown Bag over at Cody’s Site. I have to say I was very happy with the Cloud Architecture.  It definitely was a different spin to learn about the “Eco-System” and also listen to some of the questions.  Here are the notes I gathered:

  • vSphere which was once the management layer is now more of an application layer
  • vSphere administrators may not be vCloud
  • Architecture shows best practice to have two vCenters one for the Cloud Service the other for management.
  • Automation and Orchestration is more important
  • Different security levels for Cloud and vSphere Management
  • Scaling the cloud out brings additional complexity to traditional virtual BU/DR technologies
  • Most scale-outs of the cloud involve more of use case versus meeting the vCenter maximums.
    • BU/DR requirements
    • PCI Compliance (Particular Security Use-Case)
  • VCD Database is still not “officially” supported when running Oracle RAC in “Active/Active” only “Active/Standby” Configurations
  • Templates are simply “powered off” VM in VCD
  • Network Copy happens between Clusters (different storage)
  • Cloning has always been Block-File copy between host (VAAI Helps)
  • For VCD deploying vApps to the assigned tenant datastores the vApps will deploy on datastores that has the least amount of “USED” space
  • VCD requires DRS – NEVER disable it
  • Linked Clones do not have mis-alignment on NFS
  • NFS seems to be gaining more momentum for VCD deployments

The following to me seemed to make a lot of sense.  I attempted to sort it out by what I would think is needed versus highly recommended.  Need is more of vCloud is dependent on that technology.

  • High availability of certain components – applications.
    • vShield Manager > FT enabled (Fault Tolerance)
    • vCenter Server > Heartbeat (More critical because VCD uses vCenter)
  • Components Needed to use in the Cloud Architecture (VMware Specific)
    • vCenter (one for management and the other for cloud)
    • vShield Manager
    • vChargeback
  • Components Highly Recommended:
    • vCloud Service Manager
    • vCenter Update Manager
    • vCloud Connector
    • vCenter Orchestrator
  • Future Products that will be vCloud ready:
    NOTE: Due to the different API’s in each Product, VMware is playing catch up on getting some of these products “vCloud Ready”

    • vCenter Ops
    • Infrastructure Viewer
  • Skill and Knowledge increase is also needed:
    • vSphere / ESX
    • Deeper Storage Skills
    • Deeper Networking & Firewall Skills
    • Scripting (PowerCLI)
    • Workflow / Automation
    • Capacity Planning

Note: Prior it was ESX, vCenter and some scripting
it is also more about Infrastructure Management now

Cell Network Considerations:

  • Network Design of Interfaces:
    • HTTP/Console Proxy (Front-End end-user aka Portal)
    • OS Management
    • Database (Oracle or MS-SQL)
    • NFS (Transfer Service Storage L2 Network with Jumbo Frames)
    • vSphere (L2 Network with Jumbo Frames)

Note: This is not referenced in the architecture but just recommendations. These may require static or additional routes. Traditionally this has been 2 interfaces. Use VLANS if possible. The NFS and vSphere are mostly for the cloning process or Import/Export processes of VCD. This would allow the cloud to be even more scalable and efficient.

Feel Free to comment! For more information visit:
http://professionalvmware.com/2011/11/brownbag-follow-up-vcloud-architecture/

***Disclaimer: The thoughts and views expressed on VirtualNoob.wordpress.com and Chad King in no way reflect the views or thoughts of his employer or any other views of a company. These are his personal opinions which are formed on his own. Also, products improve over time and some things maybe out of date. Please feel free to contact us and request an update and we will be happy to assist. Thanks!~

%d bloggers like this: