Caching, property stores and pre-caching

I’ve written many times about various Apps Script caching techniques such as how to deal with size limits and use different back ends for more persistent caching. In this article, I’ll bring together a few of those concepts into a single library, and also introduce ‘pre-caching’.

Pre-caching

The property stores and cache stores are pretty fast, but it’s faster if the result you want is already in memory. Pre-caching writes recent caching to memory (as well as to a store/cache). Get methods first check in-memory before accessing the selected store. However, you probably want these to be fairly ephemeral, since you often share caching and stores across script instances. You also will want to limit the amount of memory to set aside for in memory caching.

Compression and multiple cache entries

This library uses all the same techniques described in Squeezing more into (and getting more out of) Cache services, but it also adds pre-caching to recently set values from memory where it can.

Usage

You use both Cache and Property Store in the same way, but you can set a few extra parameters on Caching instantiation.

Property stores

These use the Store class

  const user = new Exports.Store({
    store: PropertiesService.getUserProperties()
  })
  const script = new Exports.Store({
    store: PropertiesService.getScriptProperties()
  })

Store options.

The prefix is a string that the store adds to keys in case you want to seperate keys with the same name from each other in the property stores. I’ll explain the others later when I cover the PreCache class.

  /**
   * @constructor
   * @param {PropertiesStore} store the store to use
   * @log {Boolean} [log=false] handy for debugging
   * @param {string} [prefix=''] prefix to segregate entries from anything else in the prop store if required
   * @param {number} [evictAfter = 30000] store memory cache eviction after this time
   * @param {number} [maxLength = 100000] max bytes to hold in memory cache
   * @return {Store} 
   */

Cacher Stores

Instantiate Cacher stores like this

  const userCache = new Exports.Cacher({
    cachePoint: CacheService.getUserCache(),
  })
  const scriptCache = new Exports.Cacher({
    cachePoint: CacheService.getScriptCache(),
  })

Cacher store options

/**
 * typedef CacherOptions
 * @property {CacheService} [cachePoint=null] the cacheservice to use - if null no cachng will be done, but all methods will still work
 * @property {number} [expiry = 60*60] default expiry in seconds
 * @property  {string} [prefix='bmCachePoint] can be used to change key generation algo to partition cache entries
 * @property {boolean} [stale=false] whether to use stale cache processing
 * @property {string} [staleKey='stale'] key to use to get stale value
 * @property {boolean} [log=false] whether to log interacions
 * @property {number} [evictAfter = 30000] store memory cache eviction after this time
 * @property {number} [maxLength = 5000000] max bytes to hold in memory cache
 * @property {boolean} [reCache = false] wether to refresh itesm on access
 * @return {Cacher}
 */

As with Property Stores, evictAfter and maxLength apply to PreCache, but there are some specific parameters to control caching behavior.

• expiry – how many seconds to keep items in Cache befpre expiring them
• prefix – an optional key addition to separate from other items sharing the same cache
• stale – whether to operate stale detection
• staleKey – the key to use if stale detection is turned on
• log – useful for debugging
• reCache – if true, the expiry clock will restart for the fetched item each time on each access

Stale detection

Stalenessof data is a challenge for all types of caching. We need a more flexible handling method than just expiration times. This library provides a way to invalidate all or some of the cache entries.

Here’s how it works.

Cache key digests

When you write data to cache, you provide a key and other options to retrieve it by. When you instantiated the Cacher, you provided a prefix (or used the default one) which enables segregation of different kinds of data. The cacher uses a digest of your key and options plus the prefix behind the scenes Using a different Cacher instance with a different prefix segregates items that could share the same key

Stale and stale key

In addition to prefixes and keys, you can enable stale detection and provide a stale key.

const companyCache = new Exports.Cacher({
  cachePoint: CacheService.getUserCache(),
  stale: true,
  staleKey: 'companies'
})
const employeeCache = new Exports.Cacher({
  cachePoint: CacheService.getUserCache(),
  stale: true,
  staleKey: 'employees'
})

How does stale detection work

When stale is turned on, the cache key digest now includes whatever value is recorded against the staleKey in cache. The Cacher automatically maintains the value of staleKey. When you set a value to cache, it’s possible you want to invalidate other dependent entries, like this.

// make any items that share the same stalekey expire
employeeCache.makeStale()

// update cache for company
companyCache.set ('acme', value)

In this example, we made any cache entries to do with employees stale, because now we are dealing with another company. Since the key digest includes the latest value of staleKey when stale is enabled, the cacher will ignore any out of date, earlier entries.

Of course you could enhance the key in a single cache to reference both the company and the employee too. However, this staleness feature is handy when it’s too complex to figure out dependencies between different cache entries.

What can you write to these stores

You can set any primitive (eg boolean, number, string etc), or anything that’s Stringifyable (object, array etc). There’s not need to convert it to a string as the Store and the Cacher will do that and return it to its original state on retrieval.

Format in the property store

A property store entry looks like this. If you do want to access the property store directly (without using this library) you’ll need to JSON parse it and take the .ob property of the result. However, using store.get() will take care of all of that, and you’ll get back whatever you sent over with store.set(key, value)

{"ob":"[1,2]"}

Format in the Cacher store

Just like the property Store, the cacher will return the value in its original state. However, items are only accessible via the libary with the cacher.get() method because

• The key is a digest of your key and options, the prefix and possibly a stale value using a method private to the library. You won’t be able to find it.
• The data is compressed.
• If it’s larger than 100k (the cache property service limit), it’s spread over several cache entries and reconstituted when you do a cacher.get()

PreCacher

The PreCacher is some extra sugar on the Cacher and Store classes. It keeps an in-memory copy of whatever it writes via the .set() methods. It also respects deletion and staleness.

evictAfter

Precached items are kept in memory for the period of milliseconds you specify in evictAfter. When you .get() from a store, it first checks the value isn’t in memory, then goes to the respective Cache or Property Store to check there. Obviously in-memory is a bit faster than going out to the stores so this can give a little bit of a performance boost.

maxLength

You can control the memory usage with the maxLength parameter. This applies to the maximum length of the total of all the entries in the in-memory store at any one time. If you attempt to add a value that would cause the store to exceed this, it first evicts expired and older entries. Setting maxLength to 0 turns off PreCaching.

Getting started

I recommend you create and use an Exports script to organize your libraries and internal modules. That way you don’t need to worry about the order of script loading, nor declaring a particular library. Here’s how I use bmCacher from my testing script.

var Exports = {  
  /**
   * Unit Class 
   * @implements {bmUnitTester.Unit} 
   */
  get Unit() {
    return bmUnitTester.Unit
  },
  /**
   * Store class
   * @implements {bmPreCache.Exports.Store} 
   */
  get Store() {
    return bmPreCache.Exports.Store
  },

  /**
   * Cacher class
   * @implements {bmPreCache.Exports.Cacher} 
   */
  get Cacher() {
    return bmPreCache.Exports.Cacher
  },

  /**
   * PreCache class
   * @implements {bmPreCache.Exports.PreCache} 
   */
  get PreCache() {
    return bmPreCache.Exports.PreCache
  },

}

  const unit = new Exports.Unit({
    showErrorsOnly: true
  })
  
  // use these for general tests
  const log = false

  const user = new Exports.Store({
    store: PropertiesService.getUserProperties(),
    log,
    evictAfter: 1000
  })
  const script = new Exports.Store({
    store: PropertiesService.getScriptProperties(),
    log,
    evictAfter: 1000
  })
  const cacheOptions = {

    // 20 minutes
    expiry: 20 * 60,

    // we're allowing stale marking
    stale: true,
    staleKey: 'unit-test',

    log,

    // whether or not to refresh cache entry if its accessed
    reCache: false,
    evictAfter: 1000
  }
  const userCache = new Exports.Cacher({
    ...cacheOptions,
    cachePoint: CacheService.getUserCache(),
  })
  const scriptCache = new Exports.Cacher({
    ...cacheOptions,
    cachePoint: CacheService.getScriptCache(),
  })

Links

bmPreCache

IDE (1Rg1AKNiECIlaGMFOR3-o4Uh3QNCdU-1dOZXRRMQjM5goBiuR64ejMeXJ)

Github

testbmcacher

IDE (1iLmcg9NABXszHOyGOR5Fw2u0dHz3AfTgVjo_2hXs601_CAFZVulAVYlw)

Github

Related

SuperFetch caching: How does it work?

Fix Apps Script file order problems with Exports

Simple but powerful Apps Script Unit Test library