Upstash as an Apps Script cache platform
Upstash is a brand new service offering a serverless redis over https via a GraphQL API. Previously redis was hard to use along with Apps Script since we can’t naturally get to it with UrlFetchApp. In 3 Favourite things in one article: redis, apps script and graphQl I introduce a library to use Upstash from Apps Script, and now here’s a plugin for Apps script library with plugins for multiple backend cache platforms so we can use Upstash as a back end for caching large objects across platforms.
Benefits of using Upstash
GraphQL makes the interface really straightforward to implement and extend, and of course since redis is at the backend we can access our Upstash database from anything that can access GraphQL or a redis client. It has a pretty decent free tier to get started, and the only issue I’ve found so far is that the maximum object size it can write is 400k on the free plan, and 1mb on the paid plan. This is where bmCrusher comes in as it compresses and spreads objects across multiple records.
How to use
I like Upstash a lot, so I’ve actually added this plugin to the standard plugins for cacheservice, propertyservice and Drive that are already built in to bmCrusher, but I’ll go through how it’s built here in any case.
Just as in the other examples, it starts with a store being passed to the crusher library to be initialized. The plugin is used in exactly the same way as the Google platform plugins – like this. For details on how to sign up to upstash and get an access key for GraphQL see 3 Favourite things in one article; redis, apps script and graphQl
The plugin
We’ll be using the bmUpstash library so there’s not much to do. This plugin is already implemented in the bmCrusher library so you don’t need to do any of this, but I reproduce it here in case you are interested in seeing how to write a bmCrusher plugin
We’ll be using the exact same methods introduced in Apps script library with plugins for multiple backend cache platforms will now operate on Upstash instead of the other platforms. Here’s a refresher
Writing
All writing is done with a put method.
put takes 3 arguments
- key – a string with some key to store this data against
- data – It automatically detects converts to and from objects, so there’s no need to stringify anything.
- expiry – optionally provide a number of seconds after which the data should expire.
Reading
get takes 1 argument
- key – the string you put the data against, and will restore the data to it’s original state
Removing
Expiry
Upstash has expiry built in so if you add an expiry value when you create an item it will be automatically removed in time.
Here’s what some store entries look like in the Upstash GraphQL explorer. You can see that one of the entries has been distributed across multiple keys to deal with the maximum value size in Upstash. Getting the value will restore it to its original state.
Fingerprint optimization
Since it’s possible that an item will spread across multiple physical records, we want a way of avoiding rewriting (or decompressing) them if nothing has changed. Crusher keeps a fingerprint of the contents of the compressed item. When you write something and it detects that the data you want to write has the same fingerprint as what’s already stored, it doesn’t bother to rewrite the item.
However if you’ve specified an expiry time, then it will be rewritten so as to update its expiry. There’s a catch though. If your chosen store supports its own automatic expiration (as in the CacheService), then the new expiration wont be applied. Sometimes this behavior is what you want, but it does mean a subtle difference between different stores.
You can disable this behavior altogether when you initialize the crusher.
Formats
Crusher writes all data as zipped base64 encoded compressed, so the mime type will be text, and will need to be read by bmCrusher to make sense of it.
Links
bmCrusher
library id: 1nbx8f-kt1rw53qbwn4SO2nKaw9hLYl5OI3xeBgkBC7bpEdWKIPBDkVG0
Github: https://github.com/brucemcpherson/bmCrusher
Scrviz: https://scrviz.web.app/?repo=brucemcpherson/bmCrusher
Upstash write up: 3 Favourite things in one article redis, apps script and graphQl
Upstash console: https://console.upstash.com/login