Proxy magic with Google Apps Script

I’ve written a few articles on here about JavaScript proxy and reflect. I use both extensively not only in Node projects but also in Apps Script. In this article I’ll demonstrate how to extend built-in Apps Script objects. We’ll fiddle with UrlFetchApp to build in caching and graphql capabilities right into the fetch function and CacheSevices to add parsing.

Page Content hide

1 What is the point of Proxy ?

2 Getting started

2.1 Trapping property gets

2.2 Validating references properties

2.3 Encapsulating values in the proxy

3 Proxy for caching

4 Adding caching to fetch

5 Putting it all together

5.1 Caching examples

6 Implementing a graphql proxy

6.1 A gql proxy

6.2 Using the gql query

6.2.1 Query result

7 Summary

8 Related

What is the point of Proxy ?

Usually, if you want to extend an object or its methods, you’d build a wrapper with its own properties around that object. Creating a proxy for an object allows you to intercept or ‘trap’ accesses to its properties, or to modify the behavior of its methods. We’ll use this trapping to add rich functionality to Apps Script without having to build wrapper functions.

First off we’ll just create a proxy for UrlFetchApp that does nothing extra. Calling proxy.fetch will have exactly the same effect as calling UrlFetchApp.fetch. The ‘handler’ – the second argument to the Proxy constructor is empty so no trapping is done. The proxy instance created is essentially just a clone of UrlFetchApp.

Getting started

const proxy1 = () => {

  // use httpbin for initial testing
  const endpoint = "https://httpbin.org"

  // make a simple proxy of UrlFetchApp
  // in this case the handler (2nd argument) does nothing
  const proxy = new Proxy(UrlFetchApp, {})

  // check it still works
  result = JSON.parse(proxy.fetch(endpoint   "/get").getContentText())
  console.log(result)
}

This proxy does nothing

Trapping property gets

Here’s how to intercept an access to a property. We’ll be working on the fetch method, so first of we need to notice when the fetch method is accessed. At this point, we’re still not modifying any behavior – simply logging the fact that the fetch property has been accessed. The 3 arguments passed to the get function in our proxy are

target – the object being proxied. In this case, UrlFetchApp
prop – the property being accessed. In this case, ‘fetch’
receiver – this is the proxy we’re making. You seldom need to worry about this argument

We use Reflect to complete the get request, as we are not modifying any of the behaviors at this point. All that will happen is we’ll see a message confirming that an access to the fetch property has been observed.

const proxy2 = () => {

  // use httpbin for initial testing
  const endpoint = "https://httpbin.org"

  // lets intercept calls to the fetch argument 
  const proxy = new Proxy(UrlFetchApp, {
    // every time fetch is accessed this will be called first
    get(target, prop, receiver) {
      if (prop === 'fetch') {
        console.log('fetch called')
      }
      // reflect.get will return allthe same args as was passed to the proxy
      return Reflect.get(target, prop, receiver)
    }
  })

  // check it still works
  result = JSON.parse(proxy.fetch(endpoint   "/get").getContentText())
  console.log(result)
}

Logging property access

Validating references properties

One useful by product of using a proxy (I often use proxies just for this purpose) is to notice if you attempt to access a property that doesn’t exist. It’s all too easy in JavaScript to mispell a property name and get an unexpected and unnoticed ‘undefined’ result back. Here we are going to throw an error if an attempt is made to access a property that doesn’t exist on the object.

const proxy3 = () => {

  // use httpbin for initial testing
  const endpoint = "https://httpbin.org"

  // now let's validate that any args passed to the proxy exist
  const proxy = new Proxy(UrlFetchApp, {
    // every time fetch is accessed this will be called first
    get(target, prop, receiver) {
      console.log('trying property', prop)
      if (!Reflect.has(target, prop)) {
        throw `attempt to access unknown fetchapp property '${prop}'`
      }
      // reflect.get will return allthe same args as was passed to the proxy
      return Reflect.get(target, prop, receiver)
    }
  })
  // check it still works
  let result = JSON.parse(proxy.fetch(endpoint   "/get").getContentText())
  console.log(result)

  // try an invalid key
  result = JSON.parse(proxy.nonsense(endpoint   "/get").getContentText())
  console.log(result)
}

check for valid property names

Encapsulating values in the proxy

This is the first small change to the UrlFetchApp.fetch behavior we’ll make. Instead of callers to ‘fetch’ having to specify the base endpoint, we’ll build that into the proxy. This will allow, for example, different version of your api to be used with no change required anywhere other than in the proxy.

In this case, when the ‘fetch’ property is accessed, we return a modified version of the regular method. In fact, it’s returning a proxy for UrlFetchApp.fetch which traps ‘apply’. Apply is called when a method is executed. Here’s what happens.

proxy.fetch (url, options) is called
the proxy detects the attempt to find the fetch function
instead of returning the regular fetch function it returns a modified version of it
this modifed version has an ‘apply’ trap for when it is executed to call UrlFetchApp.fetch (endpoint + url, options) – in other words to extend the originally passed url

This small victory forms the basis to some of the much more complex things we’ll introduce in the next part of this article.

const proxy4 = () => {

  // use httpbin for initial testing
  const endpoint = "https://httpbin.org"

  // now intercept a function call to fetch and add the endpoint automatically
  const proxy = new Proxy(UrlFetchApp, {

    // every time fetch is accessed this will be called first
    get(target, prop, receiver) {

      // check its a good call
      if (!Reflect.has(target, prop)) {
        throw `attempt to access unknown fetchapp property '${prop}'`
      }
      // if we get a fetch call, we'd like to send it back with the endpoint encapsulated
      // so that when it's applied, it will execute my version of the function
      if (prop === 'fetch') {
        return new Proxy(target[prop], {
          // this 
          apply(func, thisArg, args) {
            // pick off the url and fiddle with the arguments
            const url = args[0] || ''
            return func.apply(thisArg, [endpoint   url].concat(args.slice(1)))
          }
        })
      }

      // reflect.get will return allthe same args as was passed to the proxy
      return Reflect.get(target, prop, receiver)
    }
  })

  // check it still works this time we don't need the endpoint
  let result = JSON.parse(proxy.fetch("/get").getContentText())
  console.log(result)

  // make sure that additional arguments still work
  result = JSON.parse(proxy.fetch("/get", {
    method: "post",
    payload: { data: "post options" },
    contentType: "application/json",
    header: {
      Authorization: "Bearer "   ScriptApp.getOAuthToken()
    }
  }).getContentText())

  console.log(result)

}

Modifying the base end point

Proxy for caching

We can build caching right into the fetch method, but before that we are going to create a proxy for the Apps Script cacheservice. Some of the features are

key prefixes automatically added to separate different data
tracking data added so we can tell how old the cache entry is
encapsulating data in an object so that multiple types of data can be saved to cache
instead of returning just the cache data, it will return an object with a value property containing the value retrieved from cache, and a cacheData property which will hold metadata about the cache entry such as when it was written

To achieve this we need to trap calls to the CacheService get, put and remove methods and to return versions of these with ‘apply’ traps to provide the behavior described above.

const getCache = ({ expiry = 100, prefix = '' }) => {
  // inject caching into fetch
  // some handy functions
  const isNU = (value) => typeof value === typeof undefined || value == null
  const getKey = (key) => {
    if (isNU(key) || key === '') throw 'cache key not specified'
    return prefix   key
  }
  const getExpiry = (value) => isNU(value) ? expiry : value
  const constructCacheEntry = (key, value, expiry) => {
    const createdAt = new Date().getTime()
    expiry = getExpiry(expiry)
    const expiresAt = createdAt   expiry * 1000
    return {
      value,
      cacheData: {
        key: getKey(key),
        createdAt,
        expiresAt,
        expiry
      }
    }
  }



  // a cache proxy
  return new Proxy(CacheService.getUserCache(), {
    // every time fetch is accessed this will be called first
    get(target, prop, receiver) {

      // check its a good call
      if (!Reflect.has(target, prop)) {
        throw `attempt to access unknown cacher property '${prop}'`
      }
      // if we get a fetch call, we'd like to send it back with the endpoint encapsulated
      // so that when it's applied, it will execute my version of the function

      if (prop === 'get') {
        return new Proxy(target[prop], {
          apply(func, thisArg, args) {
            const key = getKey(args[0])
            const result = func.apply(thisArg, [key])
            return result ? JSON.parse(result) : null
          }
        })
      } else if (prop === 'put') {
        return new Proxy(target[prop], {
          apply(func, thisArg, args) {
            const cacheEntry = constructCacheEntry(...args)
            const { key, expiry } = cacheEntry.cacheData
            func.apply(thisArg, [key, JSON.stringify(cacheEntry), expiry])
            return cacheEntry
          }
        })
      } else if (prop === 'remove') {
        return new Proxy(target[prop], {
          apply(func, thisArg, args) {
            const key = getKey(args[0])
            return func.apply(thisArg, [key])
          }
        })
      }

      // reflect.get will return allthe same args as was passed to the proxy
      return Reflect.get(target, prop, receiver)
    }
  })
}

A proxy for CacheService

Adding caching to fetch

Now we’re ready to add caching to the fetch proxy. In this case fetch will return an object with these properties

value – the value returned from either cache or by fetching it
cacheData – if it came from cache, this property will contains metadata about the age of the cache as well as other info
reponse – if it came from fetch, this will be a UrlFetchResponse

The example below shows multiple calls to the proxy with a delay to allow cached values to expire. Note that only ‘gets’ are cached, and only the url is used as the key. In a later example, we’ll enhance that to cache other kinds of requests.

const proxy6 = () => {

  // use requestbin for post testing
  const endpoint = "https://httpbin.org"

  // now add caching
  const cacher = getCache({ expiry: 3, prefix: 'proxy-demo-' })

  const proxy = new Proxy(UrlFetchApp, {

    // every time fetch is accessed this will be called first
    get(target, prop, receiver) {

      // check its a good call
      if (!Reflect.has(target, prop)) {
        throw `attempt to access unknown fetchapp property '${prop}'`
      }

      // a get will populate cache when its not found in cache
      if (prop === 'fetch') {
        return new Proxy(target[prop], {
          // this 
          apply(func, thisArg, args) {
            const options = args[1] || {}
            const shouldCache = (options.method || "get").toLowerCase() === 'get'
            const url = endpoint   (args[0] || '')
            const cacheEntry = shouldCache && cacher.get(url)
            if (cacheEntry) return cacheEntry
            // since we're oly dealing with json in this example. we'll parse the result while we're at it
            const response = func.apply(thisArg, [url].concat(args.slice(1)))
            const text = response.getContentText()
            const value = JSON.parse(text)
            if (shouldCache) cacher.put(url, value)
            return {
              value,
              response
            }
          }
        })
      }

      // reflect.get will return allthe same args as was passed to the proxy
      return Reflect.get(target, prop, receiver)
    }
  })

  // we're doing caching and parsing in the proxy now
  let result = proxy.fetch("/get")
  // this is the api result
  console.log(result)
  console.log(result.value)
  // this is the cache data if it came from cache 
  console.log(result.cacheData)

  // if we do it again we should get it from cache
  console.log ('from cache', proxy.fetch("/get").cacheData)

    // what happens if we wait a bit - it should not come from cache
  Utilities.sleep (5000)
  console.log ('not from cache', proxy.fetch("/get").cacheData)

  
}

add caching to the fetch method

Putting it all together

You’ll probably want multiple proxies for different things – for example caching and non caching versions, different endpoints etc, so we can easily generalize all this into a function that returns a fully baked proxy with everything in it we’ve looked at so far

const getProxy = ({ noCache = false, expiry = 100, prefix = 'demo', endpoint = "https://httpbin.org" }) => {

    // now add caching
  const cacher = !noCache && getCache({ expiry, prefix })

  return new Proxy(UrlFetchApp, {

    // every time fetch is accessed this will be called first
    get(target, prop, receiver) {

      // check its a good call
      if (!Reflect.has(target, prop)) {
        throw `attempt to access unknown fetchapp property '${prop}'`
      }

      // a get will populate cache when its not found in cache
      if (prop === 'fetch') {
        return new Proxy(target[prop], {
          // this 
          apply(func, thisArg, args) {
            const options = args[1] || {}
            const shouldCache = cacher && (options.method || "get").toLowerCase() === 'get'
            const url = endpoint   (args[0] || '')
            const cacheEntry = shouldCache && cacher.get(url)
            if (cacheEntry) return cacheEntry
            // since we're oly dealing with json in this example. we'll parse the result while we're at it
            const response = func.apply(thisArg, [url].concat(args.slice(1)))
            const text = response.getContentText()
            const value = JSON.parse(text)
            if (shouldCache) cacher.put(url, value)
            return {
              value,
              response
            }
          }
        })
      }

      // reflect.get will return allthe same args as was passed to the proxy
      return Reflect.get(target, prop, receiver)
    }
  })

}

Generalized function to get a proxy

Caching examples

Here’s a couple of examples showing multiple proxy instances with different characteristics

const proxy8 = () => {

  // this one with caching and everything. should be able to support posting as well
  const proxy = getProxy ({endpoint: "https://httpbin.org"}) 

  // we're doing caching and parsing in the proxy now
  let result = proxy.fetch("/post", {
    method: "post"
  })

  // this is the api result
  console.log(result)
  console.log(result.value)
  // this is the cache data if it came from cache 
  console.log('nocaching on a post', result.cacheData)

  // we can turn off caching too
  const noCache = getProxy ({noCache: true, endpoint: "https://httpbin.org"}) 
  noCache.fetch ('/get')

  // normally this would be cached - but we've turned it off
  console.log('shoud be undefined', noCache.fetch("/get").cacheData)
  
}

multiple instnaces

Implementing a graphql proxy

Nowadays many APIs offer graphQL endpoints rather than or as well as REST endpoints. For this example I’m going to use the filmID api endpoint. You need an api key for this, so you won’t be able to run it, but it will demonstrate how the previous examples can easily be put to other uses.

A gql proxy

This differs a little from the REST proxy in the previous examples

all requests to graphql are posts rather than gets. A request that updates is called a mutation, whereas a get operation is called a query
the payload is a graphql query or mutation, plus some options variables
the api I’m using here needs an api key in the header
because the query is built into the payload rather than the url, we need to digest the payload and the url to build a cache key, and because the apikey used might limit data access, we should add that too. In this example, I’m using an md5 digest to create a cache key from all that
the url for a graphql query is actually the endpoint, so instead of calling proxy.fetch (url, options), we will call proxy (graphqldefinition).
graphqldefinition is an object that looks like this {query, mutation, variables}. It’ll be the job of the apply trap in the fetch proxy to convert that into a fetch payload and options.
mutations shouldn’t be cached, but queries should

const getGqlProxy = ({
  noCache = false,
  expiry = 100,
  prefix = 'gql',
  endpoint = "https://api.xliberation.com/v1"
}={}) => {

  // now add caching
  const cacher = !noCache && getCache({ expiry, prefix })
  const apiKey = PropertiesService.getUserProperties().getProperty('fid-apikey')

  const getHeaders = (headers = {}) => ({
    ...headers,
    'x-fid-apikey': apiKey
  })

  // make a digest from arbitrary set of objects
  const digest = (...args) => {
    return Utilities.base64Encode(
      Utilities.computeDigest(
        Utilities.DigestAlgorithm.MD5,
        Utilities.newBlob(JSON.stringify({ a: Array.from(args) })).getBytes()
      ))
  }

  return new Proxy(UrlFetchApp, {

    // every time fetch is accessed this will be called first
    get(target, prop, receiver) {

      // check its a good call
      if (!Reflect.has(target, prop)) {
        throw `attempt to access unknown fetchapp property '${prop}'`
      }

      // a get will populate cache when its not found in cache
      if (prop === 'fetch') {
        return new Proxy(target[prop], {
          // this 
          apply(func, thisArg, args) {

            // calls to gql are always post
            // we'll cache queries but not mutations - im not implementing mutations for this demo
            // the key needs to be a digest of the query content, the apikey and the url
            /// args are {query, variables}
            if (!args[0]) throw `arg should be {query?, mutation?, variables}`
            const load = args[0]            
            const shouldCache = cacher && load.query
            const keys = digest({ endpoint, load })
            const cacheEntry = shouldCache && cacher.get(digest)
            if (cacheEntry) return cacheEntry

            const payload = JSON.stringify(load)
            const contentType = "application/json"
            const headers = getHeaders()
            const options = {
              headers,
              contentType,
              muteHttpExceptions: true,
              payload,
              method: "post"
            }
            // since we're oly dealing with json in this example. we'll parse the result while we're at it
            const response = func.apply(thisArg, [endpoint, options])
            const text = response.getContentText()
            const value = JSON.parse(text)
            if (shouldCache) cacher.put(digest, value)
            return {
              value,
              response
            }
          }
        })
      }

      // reflect.get will return allthe same args as was passed to the proxy
      return Reflect.get(target, prop, receiver)
    }
  })

}

The gql proxy

Using the gql query

Now we have a very simple to use proxy that can handle graphql, and has built in caching.


const queryTest = () => {

  const queries = {
    Film: `query Film ($id: ID!) {
      Film (id: $id) {
        id
        filmName
        synopsis
      }
    }
   `
  }

  const gqlProxy = getGqlProxy()

  const result = gqlProxy.fetch({
    query: queries.Film,
    variables: {
      id: 1018
    }
  })

  console.log(JSON.stringify(result.value.data))
}

Example usage

Query result

Here’s the API result from filmid

   "Film": {
        "id": "1018",
        "filmName": "Submarine",
        "synopsis": "A man wakes up in a Russian submarine, dressed in a monkey suit. He han drunk too much last night and does not remember anything.."
    }

result