jQuery promises and exponential backoff

When using services like Google Fusion API, sometimes you get errors because of over quota attempts – too many requests in too short a time, or the infrastructure is just too busy to service them. Google recommend you implement Exponential Backoff.

The idea is quite simple – slow down your requests by increasing amounts of random time, and eventually give up. A request that’s worth trying again will return an error 503 with various specific messages – often something like ‘backend error’.

I tend to use jQuery promises to deal with asynchronous requests. This makes implementing exponential backoff rather trivial. Here’s an extract from d3 integration with Google Fusion Tables which uses promises to make multiple simultaneous requests to fusion. To report progress, I use a traffic light system – the ‘light’ argument just defines the which div to show the progress light in.

// general deferred handler for getting json data and creating promise


function getPromiseData(url,proxyUrl,light){
    
    var deferred = $.Deferred();
    var u = url;
   
    if (proxyUrl ) {
            u = proxyUrl+"?url="+encodeURIComponent(url);
    }
    else {
            // if not proxied, needs to be jSONp
            u = u + "&callback=?"     
    }
    
    backOffJson (deferred,u,light);
    return deferred.promise();


}
function backOffJson (defer,u, light,tries) {
    // uses exponential back off
    tries = tries || 0;
    if(light)makeLight(light,"busy");
    $.getJSON(u, null, 
        function (data) {
            if (data.error) {
                if (data.error.code == 503) {
                    // a 503 error - so i'll back off
                    tries ++;
                    if (tries <= 5) {
                        // try again
                        var hangAroundFor = (Math.pow(2,tries)*1000) + (Math.round(Math.random() * 1000));
                        if (light)makeLight(light,"lemon");
                        setTimeout(function() {
                                        backOffJson (defer,u,light,tries);
                                    },hangAroundFor);
                    }
                    else { 
                        // could include some error about backoff attempts exhausted here
                        defer.reject(data); 
                    }
                }
                else {
                  defer.reject(data);
                }
            }
            else {
                defer.resolve(data);
            }
    })
    .error(function(res, status, err) {
        defer.reject("error " + err + " for " + url);
    });
    
    defer.promise()
        .done (function () {
            if (light) makeLight(light,"idle");
        })
        .fail( function () {
            if(light)makeLight(light,"failed");
        });
        
    return defer;
}
function makeLight (what,img) {
     d3.select('#'+what+'busy')
        .attr('src', "//xliberation.com/cdn/img/" +img + ".gif"); 
}

 

And that’s all there is to it. The caller can simply wait for the promise to be resolved or rejected, and the backoff will be executed a few times as per the best practice recommendation.

About brucemcp 225 Articles
I am a Google Developer Expert and decided to investigate Google Apps Script in my spare time. The more I investigated the more content I created so this site is extremely rich. Now, in 2019, a lot of things have disappeared or don’t work anymore due to Google having retired some stuff. I am however leaving things as is and where I came across some deprecated stuff, I have indicated it. I decided to write a book about it and to also create videos to teach developers who want to learn Google Apps Script. If you find the material contained in this site useful, you can support me by buying my books and or videos.