Creating a key digest to use for a cache key or to compare content

Creating a crypto digest of some content is a neat trick used by github (and torrent sharing sites) to know if the content of two files is identical. There are plenty of other uses for this idea, and in Apps Script, one of the most useful I find is to generate a short, but unique, key for items I'm writing to cache. Quite often, the data being written to cache is based on a set of parameters that is larger than the current limit for the Cache Service key size (250 characters)

Creating short unique cache keys

A simple way to create a short key size is to create a sha1 digest of the data that uniquely identifies the data being saved. 

Here's some examples, using my keyDigest function from my cUseful library. 
  Logger.log( cUseful.Utils.keyDigest ('abc'));
  Logger.log( cUseful.Utils.keyDigest ('http://ramblings.mcpher.com', 23));
  Logger.log( cUseful.Utils.keyDigest ({a:100,b:200,c:[1,2,3]}, 'some text'));
  Logger.log( cUseful.Utils.keyDigest ({a:100,b:{d:'d'},c:[1,2,3]}, true));
  Logger.log( cUseful.Utils.keyDigest (UrlFetchApp.fetch(
    'https://script.google.com/macros/s/AKfycbwZ2Hht93wTNzvRmYINYF7obaOHciBXWcP_wAiEtyGq70_x3cI/exec').getContentText(), 'lots of text',"about 100k")
  );

Results 
qZk+NkcGgWq6PiVxeFDCbJzQ2J0=
aOKNCnP1JpsgZNjTOfWg5h23GvM=
XJfaTicyOzHXxbwQkexKVJU+EUY=
HROY6aXmdgdC6Ie1O8MHuroKT5k=
DwimaHi7xF2xVK2uXdxuJVX+2xI=

The keyDigest function can take any number or type of arguments and create a digest out of them. 

Comparing values

Let's say you want a quick a way to test whether a bunch of values are the same as each other. One use might be to see if an entire spreadsheet contains the same data as another. This will show true if the two ranges contain the same data.
  Logger.log (cUseful.Utils.keyDigest(range1.getValues()) ===  cUseful.Utils.keyDigest(range2.getValues()));

Here's some more examples
  var x = 1, y = 2, z = 3;
  var a = 1 , b = 2 , c = 3;
  
  Logger.log(cUseful.Utils.keyDigest(x,y,z) === cUseful.Utils.keyDigest (a,b,c)); // true
  c = 20;
  Logger.log(cUseful.Utils.keyDigest(x,y,z) === cUseful.Utils.keyDigest (a,b,c)); // false

A note about objects

This will return true
  var o1 = {a:1,b:2};
  var o2 = {a:1,b:2};
  Logger.log(cUseful.Utils.keyDigest(o1) === cUseful.Utils.keyDigest (o2)); // true

But if you reverse the order of declaration, it will return false even though they are ostensibly the same values. This is because the order of stringification by JSON.stringify is not controllable, so be careful when passing objects as key values. Of course only stringifyable objects can be used (as opposed to built in Apps Script objects).
  var o2 = {b:2,a:1};
  Logger.log(cUseful.Utils.keyDigest(o1) === cUseful.Utils.keyDigest (o2)); // false

Collisions

The sha1 digest in theory could produce a collision - meaning that two different sets of data could return the same key. The chances of that happening though are at least 251 against (according to this paper).  That's 2,251,799,813,685,248  to 1. By comparison the chances  of winning the UK lottery are 14,000,000 to 1 and being struck by lightning in your lifetime  are 12,000 to 1. 


The code

Here's the key for the cUseful library, and it's also on github, or below.

Mcbr-v4SsYKJP7JMohttAZyz3TLx7pV4j



  /**
  * @param {[*]} arguments unspecified number and type of args
  * @return {string} a digest of the arguments to use as a key
  */
  ns.keyDigest = function () {
    // conver args to an array and digest them
    return Utilities.base64Encode (
      Utilities.computeDigest(Utilities.DigestAlgorithm.SHA_1,Array.prototype.slice.call(arguments).map(function (d) {
        return (Object(d) === d) ? JSON.stringify(d) : d.toString();
      }).join("-")));
  }

For more like this, see Google Apps Scripts snippets. Why not join our forumfollow the blog or follow me on twitter to ensure you get updates when they are available. 

You want to learn Google Apps Script?

Learning Apps Script, (and transitioning from VBA) are covered comprehensively in my my book, Going Gas - from VBA to Apps script, available All formats are available now from O'Reilly,Amazon and all good bookshops. You can also read a preview on O'Reilly

If you prefer Video style learning I also have two courses available. also published by O'Reilly.
Google Apps Script for Developers and Google Apps Script for Beginners.







Comments