rottler – a rate limit helper

Working with rate limits can be hard, so the purpose of rottler is to provide no only a way of testing rate limit strategy, but also an helper to throttle calls according to a rate limit

rate limits supported

Rottler supports

  • a limited number of call per period
  • a minimum delay between calls
  • a combination of both

installation

Usage

the best bits

Before diving into the detail, here’s the best bits

  • set up a rot according to your APIs rate limiting rules. This example is for an api that allows a maximum of 20 requests a minute, with at least 1 second between each one
  • loop through your data – each row in the array of data be presented in the loop at a rate that satisfies the rate limit rules
  • for non async Apps Script, you need to provide a timeout function and set synch to true
  • Apps script for forEach
  • using an iterator with Apps Script

Where to get it

API rate limit testing

One use of rottler is for testing your code that is supossed to handle rate limiting by acting as a simulated rate limited API. Let’s say you are writing some code to run against an API which has rate limits.

Instead of testing it against the real API, you can simulate the API response behavior with rottler.

Say the api limits to 10 calls per minute, with a minimum delay of 2 seconds between each call.

or more likely, the API you are simulating will be async

or you could see if it’s going to fail before even trying

or check how many you can still run in this period

or see how many have been run in this period

Alternatively, just let rottle handle your API calls

You can let just let rottle worry about waiting for the right time. This example will only run rot.use() when it knows it will fit inside the api rate limit parameters, and will wait for however long is necessary.

Applied to to api usage

Now we’ve seen how rot.use() simulates a rate limited API, but by mixing it into your app you can control when you call the api and forget all about rate limiting

events

If you need to customize behavior, you can set listeners to be triggered when any exceptions happen

and

In these cases, you might want to set options.throwError to false if you want to handle exceptions in some custom way

options

These are the constructor options

name default purpose
period 60000 period over which ratelimitis measured in ms
rate 10 max no of calls in period
delay 5 minimum wait between calls
timeout setTimeout a function that needs to do the same as setTimeout – unlikely to be needed
throwError true whether an attempt to trigger .use or .useAsync outside of rate throws an error
synch false how to handle waiting – you only need this if you plan to use the iterator method and provide a syncronous timeout via the timeout parameter
sleep a synchronous sleep function for use when synch is true. This is mainly for Apps Script, and the correct value would be Utilities.sleep
smooth false apply smoothing to wait times – see smoothing section later
smoothMinimum 0.25 minimum threshold for smoothing when smooth is turned on – see later for explanation

methods

All the options are accessible as class properties (eg rot.delay). Everything else is a method as below.

method returns purpose
entry() RottlerEntry measurement stats
sinceLast() number how many ms since last successful .use
tooSoon() boolean whether it’s too soon to try to .use
available() number how many .use are available in the current period (doesn’t account for .delay)
waitTime() number how long to wait before a .use will be successful
reset() start again and clear all measurements
rottle() Promise resolves when waitTime() is zero
use() RottlerEntry use 1 slot
useAsync() Promise async version of use()
on(name: string, func: function) what to do when a rate or a delay event occurs
off(name: string, func: function) turn off listening to the selected event

convenience time conversion

Since there’s a lot of conversions, a convenience ms to to other measures are provided as a static method, but also accessible from an instance. For example to get one day in ms

or 10 hours in ms

or can also be called as a static method

To convert back the other way, just stick ‘ms’ in front of the conversion name. For example to convert 200000ms to weeks.

It’s not rocket science, but it does help to document when instead of defining a simmer like this

You can do this

and you can interpret results like this

Here’s the full list of conversions

conversion name returns
seconds ms
minutes ms
hours ms
days ms
weeks ms
msSeconds seconds
msMinutes minutes
msHours hours
msDays days
msWeeks weeks

Quotas

Some schemes reset the counter at specific times, or allow the carrying forward of unused rate limits. However these are more about quotas (how many you can have) as opposed to rate limitations (how often you can have it), and are not supported by rottle at this time. If these or other pooled quota schemes is of interest, let me know in the issues section. We’d need to find a way to persist usage across sessions.

You can of course reset the counters during use with rot.reset() if necessary.

Smoothing

Let’s say you have a rate limit of 8 per second, and you have many of these to do. Normal behavior will be to do as many of these as quickly as possible then wait till the older ones expire. This is fine if you have less then the rate to do, but it’s probably better to evenly distribute the calls over the period if you have many to do. Smoothing will attempt to distribute calls over the rate measurement period by adjusting the delay between calls, but it will never be shorter than the specified delay parameters.

A smoothMinimum parameter is also available (normally 0.25) and it controls at what point smoothing kicks in. The point of it is to avoid unnecessary waiting when you only have a small number (the 8/second example smoothing with a minimum of 0.25 only kicks in after 2 calls in the period), but to smooth if it looks like there will be many to do. Smoothing also works when part of a transformation – here’s an example combining smoothing and a transformation iterator.

transformation

If you are using the rowIterator, you can also pass a transformation function that will be applied to each row like this

The value returned by rowIterator is also passed as input to the transformer and looks like this

property description
row the row value
index the row number
rows the complete rows array
transformation the row after the transformer has been applied
waitTime how long this row had to wait before being allowed to execute

Special Google Apps Script treatment

Server side Google Apps Script is not asynchronous. It doesn’t even have a setTimeout function, but it does syntactically support Promises, so to make all this work all we have to do is to provide a sleep function (which is synchronous), and tell rottle you’re working in synchronous mode

because Apps Script is synchronous and single threaded you can just do this

or if you prefer

Special treatment for loops

Rot is intended to be single threaded, so it’s up to you to manage threading when using it to test your rate management app.

If you need concurrence, see https://github.com/brucemcpherson/qottle which allows you to queue concurrent requests according to rate limit rules.

If you are using rottle to front calls to an API, at some point you’ll need to handle looping. Looping in an async environment is pretty complicated because the normal forEach doesn’t work, and if you use .map to create an array of promise they’ll all kick off together.

In Apps Script, which is syncronous you don’t need it – it’s as simple as this

On node, and client side it’s more complicated. However, Rottle provides a convenience static function to manage async looping. See this example. You can’t use this pattern with Apps Script V8 as it doesn’t support for-await-of.

synch option

With apps script there’s a way to use the iterator method too. You’ll have to provide a timeout function as before, and also set the synch option (if you don’t it won’t fail, but there won’t be a delay between calls)

Rottler figures out which type of iterator to provide on whether you’re using for or for await.

Transformations work in the same way for apps script as with node/javascript