Motivation
I’ve always been interested in ways to get multiple projects in Apps Script sharing data more easily and in a more standard way, independently of whatever platform is being used to host the data to be shared. I also wanted to extend that to Node and even web Apps. That would give a common way for each of these client types to share data.
Approach
Given that each back end storage will have entirely different rules, capacities, authentication and authorization schemes and APIS, the idea is that there is
- a standard interface – init, put, get and remove
- a standard middleware – for compression, supporting different payloads, splitting and reconstituting items that exceed the back end capabilities and so on
- plugins for different backends – some built in ones, and an easy way to add more.
Client platforms
Google Apps Script has a selection of supported plugins.
some of which are specific to Apps Script platform
and others which are generic and are/will be implemented for Node as well
All of these now have been incorporated into the main library, but the individual writes show how the plugins were built.
I’m focusing on Apps Script and Node for now. In Apps script library with plugins for multiple backend cache platforms you’ll see more details on the Apps Script clients, and later we’ll add some more.
How it works
- Intialize the plugin with various options, including picking the specific backend plugin
- put (key, value) from Apps Script or Node, where value is any stringifiable object or a string. Other cases such as binary and blobs will be supported over time.
- The crusher will compress that value, and split it up over multiple items if the backend cannot support that size, and pass these to be written to the plugin
- get(key) passes the request (could be multiple requests) to the plugin, then returns the value to its original state.
- optional expirations are used by the crusher to decide if an item should still be considered to exist. Some backends can handle automatic cleansing of expired items and physically delete the item on expiry.
Links
Apps script library with plugins for multiple backend cache platforms
- Sharing data between Apps Script and Node using Github back end – 5 minute example
- Sharing data between Apps Script and Node using Google Cloud Storage (GCS) back end – 5 minute example
- Sharing data between Apps Script and Node using Google Drive back end – 5 minute example
- Sharing data with Upstash between Node and Apps Script – end to example in 5 minutes
- The nodejs Drive API, service account impersonation and async iterators