When using services like Google Fusion API, sometimes you get errors because of over quota attempts – too many requests in too short a time, or the infrastructure is just too busy to service them. Google recommend you implement Exponential Backoff.
The idea is quite simple – slow down your requests by increasing amounts of random time, and eventually give up. A request that’s worth trying again will return an error 503 with various specific messages – often something like ‘backend error’.
I tend to use jQuery promises to deal with asynchronous requests. This makes implementing exponential backoff rather trivial. Here’s an extract from d3 integration with Google Fusion Tables which uses promises to make multiple simultaneous requests to fusion. To report progress, I use a traffic light system – the ‘light’ argument just defines the which div to show the progress light in.
// general deferred handler for getting json data and creating promise function getPromiseData(url,proxyUrl,light){ var deferred = $.Deferred(); var u = url; if (proxyUrl ) { u = proxyUrl+"?url="+encodeURIComponent(url); } else { // if not proxied, needs to be jSONp u = u + "&callback=?" } backOffJson (deferred,u,light); return deferred.promise(); } function backOffJson (defer,u, light,tries) { // uses exponential back off tries = tries || 0; if(light)makeLight(light,"busy"); $.getJSON(u, null, function (data) { if (data.error) { if (data.error.code == 503) { // a 503 error - so i'll back off tries ++; if (tries <= 5) { // try again var hangAroundFor = (Math.pow(2,tries)*1000) + (Math.round(Math.random() * 1000)); if (light)makeLight(light,"lemon"); setTimeout(function() { backOffJson (defer,u,light,tries); },hangAroundFor); } else { // could include some error about backoff attempts exhausted here defer.reject(data); } } else { defer.reject(data); } } else { defer.resolve(data); } }) .error(function(res, status, err) { defer.reject("error " + err + " for " + url); }); defer.promise() .done (function () { if (light) makeLight(light,"idle"); }) .fail( function () { if(light)makeLight(light,"failed"); }); return defer; } function makeLight (what,img) { d3.select('#'+what+'busy') .attr('src', "//xliberation.com/cdn/img/" +img + ".gif"); }
And that’s all there is to it. The caller can simply wait for the promise to be resolved or rejected, and the backoff will be executed a few times as per the best practice recommendation.