If you use any kind of API you’ll eventually have to deal with rate limiting. Most API providers introduce rate limiting to prevent abuse if their platform, but it can be tricky to deal with – especially since it’s going to be asynchronous. In this example, I’m going to use the Vimeo API. If you’re not familiar with Vimeo, it’s a media hosting platform, a bit like Youtube but popular with Media professionals.  
This article is not intended to be an end to end example, but just an extract from a larger app showing how to handle rate limiting exceptions. Other specific parts of the app are dealt with in other articles

 

The problem

My App provides access to users Vimeo account so they can see which videos they have and optionally mark them to be included in their portfolio. Since my app uses graphql, it is my back end server that talks to the Vimeo API, and translates the queries and responses in graphQL to my app’s client, but communicates with the Vimeo REST API on behalf of the user. This allows all the Apollo caching goodies to be used with the Vimeo API, and also seamlessly integrates it with other data from my database and other artefacts on Google Cloud storage. The issue is that Vimeo has some quite strict Rate limiting that needs to be handled in the server – let’s take a look at how. 

Promisifying the Vimeo request

The first step is to simply turn the Vimeo request into a promise. I’m using the Vimeo Node library. Nothing fancy here. I use the Promise chain style here (I could have used await/async), because there are a few functions that use old-style callbacks in the Vimeo API, so they need to promisified anyway. You can convert to await/async if you prefer that style.

Generating a Vimeo request

To make requests to Vimeo you need an Oauth2 access token. That’s covered elsewhere, so I won’t go into the details here, but to be able to retrieve the token, I need the decoded JWT for the user making the original GraphQL request. This comes through on the request header from the graphql client making the request. Since I’m using Firebase for client-side authentication, all that is taken care of by the Firebase SDK, and elsewhere we’ve gone through the Vimeo Oauth2 process to generate a token, so all that’s required now is to generate a Vimeo request and fire it off. This example gets a Vimeo user’s details from Vimeo.

 

Simulating errors

When developing this stuff, I like to be able to simulate errors as a I work on how to handle them. In this case, I’m going to have to deal with a number of unusual errors. The idea is to turn them on to check out the various fail paths by simulating API or network fails.

The request

Now to the request. After getting the access token and handling various error simulations, we’re ready to hit the API. This is wrapped in the manageRateLimiting function which will hide the complexity of handling the rate limit conversatons. The log parameter allows logging of any rate limit retries and maxAttempts is how many times to try before giving up.

Manage rate limiting

The vimeo API is rather simpler than most, as it provides good tools for working with rate limiting. In particular it usefully tells you how long to wait before the next measurement windows restarts, so it the manageRateLimiting function can proceed directly to calling the request. I like to put this in a wrapper in any case, as the doRequest function is going to be recursive. 

Doing the request

This function is recursive and will call itself as many times as necessary till it gets a result or gives up.  In vimeo a rate limit error is identified with an html error code of 429, and the header will contain information of why the request was rejected. We also know that the measurement window of vimeo is 60 seconds, so in case something goes badly wrong, we’ll put a maximum wait time of 61 seconds and give up if vimeo is telling us to wait any longer. The code should be fairly self-explanatory, but I’ve abstracted the handleRequest function to be able to simulate rate limit failures for testing. The waitAwhile function needs some explanation, to follow, but you can see it waits a while then recurses back into the doRequest function. If you were doing exponential backoff (where the API doesn’t give you a hint as to how long to wait, then the wait time would be calculated as an exponential based on the number of attempts so far.

Waiting a while

Normally waiting a while then doing something in JavaScript is pretty simple using setTimeout, and that’s what we’ll do here. The mistake most people make at this point is simply to setTimeout and recurse back to the calling function – but that would break the Promise chain, and return back to caller before the postponed function had been executed or even scheduled, so we need a way of adding the timeout to the promise chain and keeping control in this function, only returning when all attempts at retrying have been exhausted or we have a successful response. This is easily done by promisifying setTimeout and returning its promise, which will eventually resolve with a call to doRequest. Check back to the previous paragraph to see how waitAwhile is used and added to the promise chain.

Handling the request

The last piece of the puzzle is handling the request, which I’ve abstracted away so I can inject some simulation of errors for testing. Here if we’re simulating a request, we sometimes generate some random stuff of the type we’d get back from the API if there was a rate limit error.

All together

Putting all this together, the graphQL resolver calls getVimeoUserInfo with some parameters along with some identity verification stuff and returns the data from the Vimeo API.  All calls to Vimeo are handled the exact same way. For example, here’s one that gets info about all the videos belonging to a particular user.