If you use any kind of API you’ll eventually have to deal with rate limiting. Most API providers introduce rate limiting to prevent abuse if their platform, but it can be tricky to deal with – especially since it’s going to be asynchronous. In this example, I’m going to use the Vimeo API. If you’re not familiar with Vimeo, it’s a media hosting platform, a bit like Youtube but popular with Media professionals.  
This article is not intended to be an end to end example, but just an extract from a larger app showing how to handle rate limiting exceptions. Other specific parts of the app are dealt with in other articles

 

The problem

My App provides access to users Vimeo account so they can see which videos they have and optionally mark them to be included in their portfolio. Since my app uses graphql, it is my back end server that talks to the Vimeo API, and translates the queries and responses in graphQL to my app’s client, but communicates with the Vimeo REST API on behalf of the user. This allows all the Apollo caching goodies to be used with the Vimeo API, and also seamlessly integrates it with other data from my database and other artefacts on Google Cloud storage. The issue is that Vimeo has some quite strict Rate limiting that needs to be handled in the server – let’s take a look at how. 

Promisifying the Vimeo request

The first step is to simply turn the Vimeo request into a promise. I’m using the Vimeo Node library. Nothing fancy here. I use the Promise chain style here (I could have used await/async), because there are a few functions that use old-style callbacks in the Vimeo API, so they need to promisified anyway. You can convert to await/async if you prefer that style.

// turns vimeo request into a promise
const requestPromise = ({ vimeoClient, request }) => {
    // assume access token already set
    return new Promise((resolve, reject) => {
        vimeoClient.request(request, function (error, body, status_code, headers) {
            if(error) {
                reject({
                    error,
                    status_code,
                    headers
                });
            } else {
                resolve ({
                    headers,
                    body
                });
            }
        });
    });
};

Generating a Vimeo request

To make requests to Vimeo you need an Oauth2 access token. That’s covered elsewhere, so I won’t go into the details here, but to be able to retrieve the token, I need the decoded JWT for the user making the original GraphQL request. This comes through on the request header from the graphql client making the request. Since I’m using Firebase for client-side authentication, all that is taken care of by the Firebase SDK, and elsewhere we’ve gone through the Vimeo Oauth2 process to generate a token, so all that’s required now is to generate a Vimeo request and fire it off. This example gets a Vimeo user’s details from Vimeo.

 

/**
* this gets a vimeo user info item
* @param {object} pack this is the verified firebase user object that get passed with every authenticated gql request
* @return {object} a user info item from vimeo
*/
const getVimeoUserInfo = (fad) => {
    const { user, whoAmI, passedVimeoID } = fad;
    // make a vimeo request - for now the graphql definition is more than this, but Im just taking a subset
    const vimeoID = passedVimeoID || user.vimeoID;
    const request = {
        path: `/users/${vimeoID}`,
        query: {
            fields: `
            uri,
            name,
            link,
            location,
            bio,
            created_time,
            pictures,
            websites,
            metadata.connections.videos,
            metadata.connections.pictures,
            account
        `
        }
    };
    return vimeoRequest({ vob: whoAmI, userID: user.userID, request })
        .then (r => {
            const { body } = r;
            // and add an id - will help with client side caching
            if (body) {
                const id = body.uri.replace(/\/users\/(.*)/,'$1');
                if (id !== vimeoID) {
                    return errors.gqlError('vimeo api',`asked for ${vimeoID} got ${id}` , errors.codes.DB);
                }
                body.id = id;
            }        
            return body;
        });
};

Simulating errors

When developing this stuff, I like to be able to simulate errors as a I work on how to handle them. In this case, I’m going to have to deal with a number of unusual errors. The idea is to turn them on to check out the various fail paths by simulating API or network fails.

const simulateMissingToken = false;
const simulateMissingRequest = false;
const simulateFailRemoveToken = false;
const simulateRateLimitError = false;

The request

Now to the request. After getting the access token and handling various error simulations, we’re ready to hit the API. This is wrapped in the manageRateLimiting function which will hide the complexity of handling the rate limit conversatons. The log parameter allows logging of any rate limit retries and maxAttempts is how many times to try before giving up.

// do a vimeo request
const vimeoRequest = ({ vob, userID, request }) => {
    // first get the accesstoken
    return getVimeoToken(vob)
        .then(r => {

            // check for missing token
            if (!r || simulateMissingToken) {
                return errors.gqlError(
                    'no token vimeo token entry',
                    'user id ' + userID + ' simulated?' + simulateMissingToken,
                    errors.codes.AUTH
                );

            } else if(r.userID !== userID ) {
                return errors.gqlError(
                    'mismatch userid',
                    'user id ' + userID + '/' + r.userID ,
                    errors.codes.AUTH
                );

            } else {
                const vimeoClient = new Vimeo();
                vimeoClient.setAccessToken(r.tokenInfo.accessToken);
                return manageRateLimiting ({
                    vimeoClient,
                    request,
                    log: true,
                    maxAttempts: 3
                });
            }
        });
};

Manage rate limiting

The vimeo API is rather simpler than most, as it provides good tools for working with rate limiting. In particular it usefully tells you how long to wait before the next measurement windows restarts, so it the manageRateLimiting function can proceed directly to calling the request. I like to put this in a wrapper in any case, as the doRequest function is going to be recursive. 

 // handles rate limiting for requests
const manageRateLimiting = ({ vimeoClient, request, log, maxAttempts }) =>
    doRequest ({
    vimeoClient,
    request,
    log,
    maxAttempts
});

Doing the request

This function is recursive and will call itself as many times as necessary till it gets a result or gives up.  In vimeo a rate limit error is identified with an html error code of 429, and the header will contain information of why the request was rejected. We also know that the measurement window of vimeo is 60 seconds, so in case something goes badly wrong, we’ll put a maximum wait time of 61 seconds and give up if vimeo is telling us to wait any longer. The code should be fairly self-explanatory, but I’ve abstracted the handleRequest function to be able to simulate rate limit failures for testing. The waitAwhile function needs some explanation, to follow, but you can see it waits a while then recurses back into the doRequest function. If you were doing exponential backoff (where the API doesn’t give you a hint as to how long to wait, then the wait time would be calculated as an exponential based on the number of attempts so far.

const doRequest = (payload) => {
    // vimeo rate limit window is 1 minute so thats the most we should ever have to wait
    // + add an extra second for rounding// vimeo rate limit window is 1 minute so thats the most we should ever have to wait
    // + add an extra second for rounding
    const MAX_WAIT_TIME = 61 * 1000;
    const { vimeoClient, request, log, maxAttempts } = payload;

    return handleRequest(payload)
        .catch (err => {

        // unpick this error
        const { headers, error, status_code } = err;

        // if its a rate limit issue, then log and potentially retry
        if (status_code === 429 && headers) {
            /*
                this is a rate limit problem
                X-RateLimit-Limit The maximum number of API responses that the requester can make through your app in 60 secs
                X-RateLimit-Remaining The remaining number of API responses in the current 60-second period.*
                X-RateLimit-Reset A datetime value indicating when the next 60-second period begins.
            */
            const xlimit = headers['X-RateLimit-Limit'];
            const xRemaining = headers['X-RateLimit-Remaining'];
            const xReset = headers['X-RateLimit-Reset'];
            if (log) {
                console.log('rate limit problem detected ', {
                    xlimit,
                    xRemaining,
                    xReset,
                    simulateRateLimitError
                });
            }

            // after a number if attempts give up
            if (maxAttempts <= 1) {
                // if we've failed too many times then clear off
                if (log) console.log('too many attempts giving up');
                return Promise.reject('Too many attempts to defeat vimeo rate limit');
            }

            // now wait a bit and add a little to compensate for latency
            // no need for exponential backoff as the info is given in the headers

            const waitTime = new Date(xReset).getTime() - new Date().getTime() + 200;
            if (waitTime > MAX_WAIT_TIME) {
                console.log('not waiting that long - giving up after', waitTime);
                return Promise.reject('Too long to wait for next vimeo request attempt');
            }

            // try again
            if (log) console.log('waiting then retrying ', waitTime);
            return waitAwhile ({
                action: doRequest,
                params: {
                    vimeoClient,
                    request,
                    log,
                    maxAttempts: maxAttempts - 1
                },
                waitTime
            });

        } else {
            // its some other error
            return Promise.reject(err);
        }
    });
};

Waiting a while

Normally waiting a while then doing something in JavaScript is pretty simple using setTimeout, and that’s what we’ll do here. The mistake most people make at this point is simply to setTimeout and recurse back to the calling function – but that would break the Promise chain, and return back to caller before the postponed function had been executed or even scheduled, so we need a way of adding the timeout to the promise chain and keeping control in this function, only returning when all attempts at retrying have been exhausted or we have a successful response. This is easily done by promisifying setTimeout and returning its promise, which will eventually resolve with a call to doRequest. Check back to the previous paragraph to see how waitAwhile is used and added to the promise chain.

// timeout promise
const waitAwhile = ({ action, params , waitTime }) => {
  return new Promise ( resolve => {
 setTimeout (() => {
 resolve(action(params));
 }, waitTime);
 });
};

Handling the request

The last piece of the puzzle is handling the request, which I’ve abstracted away so I can inject some simulation of errors for testing. Here if we’re simulating a request, we sometimes generate some random stuff of the type we’d get back from the API if there was a rate limit error.

// handles a request - separated out to be able fake a rate limit failure for testing
const handleRequest = (params) => {
        // simulate a rate limit error for testing with a 70% chance of happening
        if(simulateRateLimitError && Math.random() < 0.7) {
            const t = new Date();
            const waitSeconds = Math.floor(Math.random()*60);
            // some random amount of time up to 60 seconds
            console.log('handling wait of', waitSeconds);
            t.setSeconds(t.getSeconds() + waitSeconds);
            return Promise.reject({ headers: {
                'X-RateLimit-Reset': t.toISOString()
            }, status_code: 429 });
        }

        // hit the api
        return requestPromise (params);
};

All together

Putting all this together, the graphQL resolver calls getVimeoUserInfo with some parameters along with some identity verification stuff and returns the data from the Vimeo API.  All calls to Vimeo are handled the exact same way. For example, here’s one that gets info about all the videos belonging to a particular user.

/**
* this gets a vimeo user video item
* @param {object} pack this is the verified firebase user object that get passed with every authenticated gql request
* @return {object} a user info item from vimeo
*/
const getVimeoUserVideosInfo = (fad) => {

    const { user, whoAmI, passedVimeoID , params } = fad;
    const { limit, offset, value } = params;
    const vimeoID = passedVimeoID || user.vimeoID;

    // make a vimeo request - for now the graphql definition is more than this, but Im just taking a subset
    const request = {
        path: `/users/${vimeoID}/videos`,
        query: {
            page: Math.floor(offset / limit) + 1,
            per_page: limit,
            sort: 'modified_time',
            direction: 'desc',
            filter: 'playable',
            filter_playable: true,
            fields: `
            files,
            download,
            uri,
            name,
            link,
            description,
            type,
            duration,
            height,
            width,
            language,
            embed,
            created_time,
            release_time,
            pictures,
            privacy,
            status
        `
        }
    };

    // if there's a search query add that too
    if (value) {
        request.query.query = value;
    }

    return vimeoRequest({ vob: whoAmI, userID: user.userID, request })
        .then (r => {
            const { body } = r;
            const { data, paging } = body;
            data.forEach(d => {
                // and add an id - will help with client side caching
                const videoID = d.uri.replace(/\/videos\/(.*)/,'$1');
                d.id = videoID;
            });
            return body;
    });
};