vSphere – vCloud – The Query 1.5 API “feature” pageSize bug?

Alright, this is going to be difficult for me to really explain so I will do my best to serve it justice. First, I am not a coder and I do not know the ins and outs of the API and code. What I will attempt to explain to you is how you can reproduce this issue on your VCD instance. I also want to note this is vanilla VCD 1.5 with no updates yet. I currently do have a case with VMware opened and I have yet to resolve it.

Let’s get to the nitty gritty.

First off, I want to say that I am not 100% sure that any other queries you use produce the same affect. This issue seems to happen with only the VMadmin query.

First I would recommend reading about connecting the Rest API with Will’s blog over at VMware:

http://blogs.vmware.com/vsphere/2012/03/exploring-the-vcloud-rest-api-part-1.html

Now that you have read that and understand how to connect to the REST API I will show you an example of a basic VMadmin query.
(Note: you need to have over 128 VCD Vapps to reproduce this type of issue)

GET http://vcd.url.com/api/query?type=adminVM

This showed me that I had 333 queries returned however on the 1st page I only found 128. Now the way the script talked to VCD API was rather plain and it was basically doing this query and dumping it to a XML file. The idea was that this was similar to 1.0 API where I could get all the data I wanted and dumped into an XML file. This wasn’t the case. It seems I couldn’t get around this 128 limit. So I decided to try the next query:

Get http://vcd.instance.com/api/query?type=adminVM&pageSize=999

After running it I still got 333 queries returned but only 128 on the single page EVEN after specifying a pageSize=999 so this isn’t the end of it… let’s dig deeper. After further researching I had actually found documented proof that this was a hard setting somewhere.

Page 212 of the VCD 1.5 API Guide taken from here: http://www.vmware.com/pdf/vcd_15_api_guide.pdf

So it became obvious to me at this point that no matter what your query is it would always default to 128 objects per page. So I tried to also do the following to change this hard setting (at the recommendation of someone) located in a global.properties file in the following directory on the vCloud Director cells:

/opt/vmware/vcloud-director/etc

add/change the following: restapi.queryservice.maxPageSize=1024

I added this to the global.properties file and the VCD cells service were also restarted. Can you guess what still happened? Nothing… this didn’t change anything at all. In fact, it still remained broken. Folks, this still wasn’t the worse part about it. Lets cover the part that I believe is a true bug in the API and had someone on Twitter also comment that there is a possible bug in adminVM query.

Lets say I do a query for a pageSize=135 and my query returns 153 results. We get the usual 128 queries per page. Here is an example of the commands I used:

GET https://vcd.url.com/api/query?type=adminVM&pageSize=135&page=1&fields=name&sortAsc=name

Sort ascending gives me an alphabetical sorting of all my vApp names and I can find a Breaking point for my virtual machines (I know my ABC’s and what should be next so to speak). So I copy and paste the results into Notepad++ and it shows me 128 entries of the page size of 135 (give or take a few for other lines returned not relevant to the query. The bug as discussed is evident. However, it doesn’t show the other 7 entries it should be showing. Remember, we did the page size for 135. So now let’s take a peek at page 2.

GET https://vcd.url.com/api/query?type=adminVM&pageSize=135&page=2&fields=name&sortAsc=name

So after you run this query you will the list of the remaining 153 results. However if you take notes you will notice that it is in fact completely missing the 7 other entries. So basically your query takes the 7 it could NOT list and dumps it out to somewhere in the Cloud…. So what does this mean aside from the fact that there is a bug?

You will need to use a looping construct and not specify a page size greater then 128. (see Will’s comments below)

This is a bug and I don’t think I could make it any clearer. I wish I could’ve provided some screenshots but I think if someone does there due diligence they will see what I am talking about. If you have 2000 VCD vApps and you do a page size of 500 you would lose 372 queries between each page. No matter how you specify the page size, modify the Global.properties its just broken plain and simple. If someone would like to provide some screen shots I would be happy to put them up here to show some better detail.

If you want to discuss in further detail feel free to comment and I will follow up.

UPDATE: After reviewing with VMware on some things I found out this is actually a true but with the vCloud 1.5 API bug.  The good news is that there is a fix slated to be published in August, perhaps they will allow for a private fix if you really need it. Stay tuned. If anyone has some information aside from this please provide and I will link it! Thanks again. Also, this is not related to any type of Query parameter this is more to do with how the Query service works.

About these ads

About Cwjking

I am an IT professional working in the industry for over 10 years. Starting in Microsoft Administration and Solutions I was also a free lance consultant for small businesses. Since I first saw virtualization I have always been fascinated by the concept. I currently specialize in VMware technology. I consult daily on many different types of VMware Solutions. My current role is hands on administration, technical design, and consulting.

Posted on June 8, 2012, in Troubleshooting, vCloud and tagged , , , , , , , , . Bookmark the permalink. 4 Comments.

  1. Chad,

    I want to point out a few things as some of your assumptions based on the behavior you’re seeing in your environment are not quite correct.

    The limit of 128 entries per page is not specific to any particular query type (e.g. vAppTemplate, adminVM, etc). It is for the Query Service itself which is what you’re using to search within VCD. This limit defines the maximum entry for page and as I mentioned earlier on twitter, this is a system limit and can not be changed as you’ve confirmed with support. If you have say 640 objects that would return from a given query, this would equate to 5 pages of 128 entries per page.

    This means that if you want to retrieve all 640 entries, you still can but you need to iterate through those 5 pages. Here is a screenshot of querying for vAppTemplate which is greater than than the default 25 and as you can see there is a link to “next page” and “last page”

    http://i45.tinypic.com/fjlt1u.png

    If you perform a GET on the page2, you will see there is a link to “next page” as well as “previous page”

    http://i49.tinypic.com/13yj6v9.png

    So to retrieve all results, this can easily be done with a for loop construct which checks to see if you’ve reached all entries by either performing a count or even better, checking if “next page” still exists and this is probably even more trivial using any of the 3 SDKs from a programmatic standpoint.

    I agree that this should be better documented about the maximum and I’ll relay that back to our tech pubs team. I think in general it’s a good thing the pageSize can be configured, you definitely don’t want all entries returned, especially if you’re talking about very large VCD environments where you may have a few thousand VMs.

    I would also recommend you take a look at this blog post if you have not already which goes over in more detail about the Query Service – http://blogs.vmware.com/vsphere/2012/04/quickly-finding-objects-using-the-vcloud-api-query-service.html as well as the documentation which goes over the query parameters http://pubs.vmware.com/vcloud-api-1-5/wwhelp/wwhimpl/js/html/wwhelp.htm#href=api_prog/GUID-CDF04296-5EB5-47E1-9BEC-228837C584CE.html one of which is the “page” property for results that may span multiple pages such as in your case.

    Hopefully this makes more clear and would be great if you can update your article to reflect this information. It’ll be interesting to see what the MAX pageSize will be supported as you don’t want your payload to get too big.

    –William Lam

    • I agree, and I have updated my blog. It is a problem with the query service. However, specifying greater then 128 entries (which you can easily do) should not make your entries completely disappear going between the pages. Although 128 is the hard limit others may not choose to want to write looping or what have you to accommodate for a VMware bug. This is just my opinion, if something is supposed to work a particular way and allows for you to utilize in that particular way then it should produce the expected results or as you indicated be documented. It doesn’t produce the expected results and customers shouldn’t have to workaround that. You mention SDK and SDK is great but what many will find out when working with SDK it actually takes a completely separate support agreement.. Thought in the long run that may be a better option :).

      Either way I am glad VMware acknowledges it as a “bug” and is looking to fix it. Thanks for the comment William.

  2. Chad,

    I just want to be clear about the expected behavior of how the Query Service work. IMHO this is a VCD platform bug and not an API bug. Let’s put aside the fact you want to change the system default of 128, let’s say that is the maximum which it is right now. If you craft a query that would return more than 128 entries, the Query Service will return you the first 128 entries IF you set it to be the maximum and then provide you links to the rest of the entries on sub-sequent pages. The data is not lost or hiding, you just need to look at the other pages, this is the expected behavior and BY DESIGN. You have to remember, the API must not only support small environments of several hundreds objects but also tens of thousands if not more. You may want everything in a single payload but that may not always be the case depending on the use case.

    I just want to stress that that the behavior you’re seeing with the Query Service is by design. You definitely don’t have to use the SDKs, many users consume VCD via the REST API and it’s a pretty common practice to loop through a set of “things” and stop processing where there’s no more “things” to process, in this case pages. If you want, you can even look at the initial GET and do some math on the total number of entries, maximum entries per page and figure out how many pages you would need to read and append to your final output.

    I’m glad support has a fix coming and it’ll resolve the problem you’re facing and hopefully this gave you a better understanding of how the Query Service works.

    Thanks

    –William Lam

    • Yeah that makes sense. It is a bug and I do understand why they made that by design. You would’ve thought that the solution provided to increase the payload would’ve worked. However in my testing the entries were missing even when viewing the other pages. Only when exceeding the 128 limit though. Thanks again for the reply :).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: