Creating a key digest to use for a cache key or to compare content

Creating a crypto digest of some content is a neat trick used by github (and torrent sharing sites) to know if the content of two files is identical. There are plenty of other uses for this idea, and in Apps Script, one of the most useful I find is to generate a short, but unique, key for items I’m writing to cache. Quite often, the data being written to cache is based on a set of parameters that is larger than the current limit for the Cache Service key size (250 characters)

Page Content hide

1 Creating short unique cache keys

2 Comparing values

3 A note about objects

4 Collisions

5 The code

6 Share with your network

Creating short unique cache keys

A simple way to create a short key size is to create a sha1 digest of the data that uniquely identifies the data being saved.

Here’s some examples, using my keyDigest function from my cUseful library.

Logger.log( cUseful.Utils.keyDigest ('abc'));
  Logger.log( cUseful.Utils.keyDigest ('https://ramblings.mcpher.com', 23));
  Logger.log( cUseful.Utils.keyDigest ({a:100,b:200,c:[1,2,3]}, 'some text'));
  Logger.log( cUseful.Utils.keyDigest ({a:100,b:{d:'d'},c:[1,2,3]}, true));
  Logger.log( cUseful.Utils.keyDigest (UrlFetchApp.fetch(
    'https://script.google.com/macros/s/AKfycbwZ2Hht93wTNzvRmYINYF7obaOHciBXWcP_wAiEtyGq70_x3cI/exec').getContentText(), 'lots of text',"about 100k")
	);

Results

qZk+NkcGgWq6PiVxeFDCbJzQ2J0=
aOKNCnP1JpsgZNjTOfWg5h23GvM=
XJfaTicyOzHXxbwQkexKVJU+EUY=
HROY6aXmdgdC6Ie1O8MHuroKT5k=
DwimaHi7xF2xVK2uXdxuJVX+2xI=

The keyDigest function can take any number or type of arguments and create a digest out of them.

Comparing values

Let’s say you want a quick a way to test whether a bunch of values are the same as each other. One use might be to see if an entire spreadsheet contains the same data as another. This will show true if the two ranges contain the same data.
Logger.log (cUseful.Utils.keyDigest(range1.getValues()) === cUseful.Utils.keyDigest(range2.getValues()));

Here’s some more examples

var x = 1, y = 2, z = 3;
  var a = 1 , b = 2 , c = 3;
  
  Logger.log(cUseful.Utils.keyDigest(x,y,z) === cUseful.Utils.keyDigest (a,b,c)); // true
  c = 20;
  Logger.log(cUseful.Utils.keyDigest(x,y,z) === cUseful.Utils.keyDigest (a,b,c)); // false

A note about objects

This will return true

var o1 = {a:1,b:2};
  var o2 = {a:1,b:2};
  Logger.log(cUseful.Utils.keyDigest(o1) === cUseful.Utils.keyDigest (o2)); // true

But if you reverse the order of declaration, it will return false even though they are ostensibly the same values. This is because the order of stringification by JSON.stringify is not controllable, so be careful when passing objects as key values. Of course only stringifyable objects can be used (as opposed to built in Apps Script objects).

var o2 = {b:2,a:1};
  Logger.log(cUseful.Utils.keyDigest(o1) === cUseful.Utils.keyDigest (o2)); // false

Collisions

The sha1 digest in theory could produce a collision – meaning that two different sets of data could return the same key. The chances of that happening though are at least 251 against (according to this paper). That’s 2,251,799,813,685,248 to 1. By comparison the chances of winning the UK lottery are 14,000,000 to 1 and being struck by lightning in your lifetime are 12,000 to 1.

The code

Here’s the key for the cUseful library, and it’s also on github, or below.

Mcbr-v4SsYKJP7JMohttAZyz3TLx7pV4j

/**
  * @param {[*]} arguments unspecified number and type of args
  * @return {string} a digest of the arguments to use as a key
  */
  ns.keyDigest = function () {
    // conver args to an array and digest them
    return Utilities.base64Encode (
      Utilities.computeDigest(Utilities.DigestAlgorithm.SHA_1,Array.prototype.slice.call(arguments).map(function (d) {
        return (Object(d) === d) ? JSON.stringify(d) : d.toString();
      }).join("-")));
	  }

For more like this see Google Apps Scripts Snippets