Query Limits on parse.com

The first problem we hit is the 1000 object limit on parse queries. 

Here's how to get round it using jQuery promises.

get round parse.com 1000 limit


How it works

Parse allows only 1000 results max per query, but it does allow you to skip. This means that you can make multiple queries, skipping those already retrieved until you have them all. However, since the query is Async, it makes it a little tricky. However jQuery promises are a simple way to deal with asynce complexity. 

  • Here we want to retrieve all rows with the scheme name "dulux". We simply initialize Parse with our credentials, create a new Parse class called "ColorTable", and wait for the promise returned by the function "schemePromise" to be either resolved or rejected
            google.setOnLoadCallback(function () {
               
//populateParse();
               
var scheme = "dulux";
               
var parseKeys = getParseKeys();
               
Parse.initialize(parseKeys.appId, parseKeys.jsKey);
               
var ColorTable = Parse.Object.extend("ColorTable");
                schemePromise
(ColorTable, scheme).done(function (all) {
                    console
.log("scheme " + scheme + " returned " + all.length);
               
})
                   
.fail(function (error) {
                    console
.log("error getting scheme " + scheme + JSON.stringify(error));
               
});
           
});
  • schemePromise calls findChunk, passing any results so far returned. When the promise returned by findChunk is resolved, it calls itself if there are still more results to be returned.

           
function schemePromise(model, scheme, allResults, allPromise) {
               
// find a scheme at a time
               
var promise = allPromise || $.Deferred();

                findChunk
(model, scheme, allResults || [])
                   
.done(function (results, allOver) {
                   
if (allOver) {
                       
// we are done
                        promise
.resolve(results);
                   
} else {
                       
// may be more
                        schemePromise
(model, scheme, results, promise);
                   
}
               
})
                   
.fail(function (error) {
                    promise
.reject(error);
               
});
               
return promise.promise();
           
}
  • findChunk does a parse.com query, returning up to 1000 objects, concatentates the results to others so far received, and lets schemePromise know whether there are any more to get through the arguments to its promise resolution.
            function findChunk(model, scheme, allData) {
               
// we have to find in chunks since there is a limit on query size
               
// will return a promise
               
var limit = 1000;
               
var skip = allData.length;
               
var findPromise = $.Deferred();
               
var query = new Parse.Query(model);
                query
.equalTo("scheme", scheme);
                query
                   
.limit(limit)
                   
.skip(skip)
                   
.find()
                   
.then(function (results) {
                    findPromise
.resolve(allData.concat(results), !results.length);
               
}, function (results) {
                    findPromise
.reject(error);
               
});
               
return findPromise.promise();
           
}

This model will work well for any retrieval needs from Parse.com for this project by simply generalizing the query constraints. 

There is another problem though - 

error getting scheme {"code":154,"message":"Skips larger than 10000 are not allowed"}

Which means that we'll need yet another workaround if there are more than 10000 rows in the table. 

For more on parse.com see Parse.com
For help and more information join our forumfollow the blogfollow me on twitter

You want to learn Google Apps Script?

Learning Apps Script, (and transitioning from VBA) are covered comprehensively in my my book, Going Gas - from VBA to Apps script, All formats are available from O'ReillyAmazon and all good bookshops. You can also read a preview on O'Reilly

If you prefer Video style learning I also have two courses available. also published by O'Reilly.
Google Apps Script for Developers and Google Apps Script for Beginners.




Comments