Rate limiting – Firestore and Firebase cloud functions

This is one of a series of articles about Going serverless with Firebase. You many want to read about Firebase cloud functions and Custom domains and ssl with Firebase hosting before this article.

So you’ve created your cloud function and have a firestore backend. There are abuse limits built into the Firebase platform, but you’ll probably want to limit usage to ensure you keep your hosting costs under control too. In this example, there are a number of rate limits and quotas to enforce, based on the level of subscription of an account.

Here’s an extract from an example configuration, with 2 plans and 4 limits per plan

plans: {
      "a": {
        "limiters": {
          "burst": {
            "seconds": 30,
            "rate": 30
          },
          "minute": {
            "seconds": 120,
            "rate": 60,
          },
          "day": {
            "seconds": 86400,
            "rate": 2000
          },
          "dailywrite": {
            "seconds": 86400,
            "rate": 10240000,
            "type": "quota"
          }
        }
      },
      "b": {
        "limiters": {
          "burst": {
            "seconds": 30,
            "rate": 60
          },
          "minute": {
            "seconds": 120,
            "rate": 180
          },
          "day": {
            "seconds": 86400,
            "rate": 20000
          },
          "dailywrite": {
            "seconds": 86400,
            "rate": 102400000,
            "type": "quota"
          }
        }
      },

Each limiter consists of properties like this – this means that in a window of 120 seconds a maximum of 60 operations are allowed – in other words 1 every 2 seconds, or 30 a minute on average

"minute": {
            "seconds": 120,
            "rate": 60,
          },

Firestore slotlimit document

Each account is associated with a particular plan, and has a firestore document like this, with a property corresponding to each of the limits that are being imposed. It also has an expiry date to clean up unused documents when no longer active.

A slot is a window over which a rate is being measured. Calculating the current slot is simply a matter of dividing the current time by the size of the window, then each of the rates for the current slot can be checked to see if there is enough quota left to complete the proposed operation. Every operation starts with a fetch of the slot information for the current account

  return dbStore.getSlotLimit (accountId).then (pack=> {

next we get the current time

const nowSecs = new Date().getTime()/1000;

look through each rate being checked

    const ob = Object.keys(plan.limiters)
        .reduce((p,name)=>{
           const co = plan.limiters[name];

figure out the  slot based on the current time and the measurement window

const slot = Math.floor(nowSecs/co.seconds);

update the slot if we’ve moved on to a new measurement window from the last operation

p[name] = currentOb[name];
           if (!p[name] || p[name].slot !== slot) {
              p[name] = {
                used:0,
                slot:slot
              };
            }

increment the slot with the volume (if it’s a quota type – for example no. of bytes written per day) or by one if it’s a rate limited operation, and report and error if the limit is exceeded

 p[name].used += (co.type === "quota" ? volume : 1);
            manage.errify(
              p[name].used <= co.rate,
              "QUOTA",
              name + " quota/rate limit exceeded",
              pack
            );

finish off the loop

   return p;
        },{});

If we get here without recording an error – we can go ahead and write the updated rate to firestore

dbStore.setSlotLimit (accountId,ob)

otherwise report a quota violation

And that’s all there is to it.

For other articles on this topic see Going serverless with Firebase.  For more snippets, see Google Apps Scripts snippets