In Resuscitating the Apps Script execution transcript – JavaScript Proxy and Reflect to the rescue I showed how we could use the ES6 Proxy global object to intercept calls to Apps Script services so we could log service activity. Of course Proxy allows you to intercept any object, so I wondered if there might be a way to enhance UrlFetchApp and Cache Services to do all the repetitive stuff when making calls to a GraphQL API.
Of course all this can be done by just wrapping these services in a some methods to do the prep work, but using a proxy seems a nicer solution, despite being a little more complex to get your head around at first. This is just a proof of concept, but I like where it might lead.
Approach
With an enhanced UrlFetch, I just want to just call fetch with the GraphQL query as an argument. This means the fetch has also to do these things
- Prepare the auth, other headers and standard endpoint
- Deal with error handling, parsing and structuring of the result
- Use and maintain cache to avoid unnecessary calls
The UrFetch proxy will also make calls to the cache service, so I want to enhance that to
- Stringify and parse results
- Maintain cache freshness data
- Compress content
- Build keys from query content
What is Proxy
It’s a new Global Object that appeared in ES6, and it allows you to intercept the fundamental workings of a JavaScript object to change its behavior. These intercepts are called traps.
For some basic on setting up proxies and how they work, see Resuscitating the Apps Script execution transcript – JavaScript Proxy and Reflect to the rescue
Proof of concept
For simplicity, I’m going to redo the bmSwopCx library – see Currency exchange rates library for Apps Script – optimized for the parsimonious – swop.cx for the original implementation details.
My final class will look like this, with only 4 methods and a constructor, plus some static stuff and all the the required gql queries and fragments.
class _SwopCx {
/**
* @param {options}
* @param {function} options.fetcher the fetcher function from apps script- passed over to keep lib dependency free
* @param {string} options.apiKey the apiKey
* @param {string} [options.defaultBase ="EUR"] the default base currency - this doesn't work with the free version
* @param {boolean} [options.freeTier = true] whether the apiKey is for the limited free tier
* @param {cache} [options.cache = null] the apps script cache service to use
* @param {number} [options.cacheSeconds = 3600] the lifetime for cache
*/
constructor({ fetcher, apiKey, defaultBase = "EUR", freeTier = true, cache = null, cacheSeconds = 60 * 60 }) {
}
/**
* @param {options}
* @param {function} options.symbols the comma separated list of currencies to consider
* @param {string} [options.base ="EUR"] the default base currency - this doesn't work with the free version
* @param {boolean} [options.noCache = false] whether to skip caching (if its enabled in the first place)
* @param {boolean} [options.meta = false] whether to include the metadata about the rate
*/
latest(params) {
}
/**
* @param {options}
* @param {function} options.symbols the comma separated list of currencies to consider
* @param {string} [options.base ="EUR"] the default base currency - this doesn't work with the free version
* @param {boolean} [options.noCache = false] whether to skip caching (if its enabled in the first place)
* @param {boolean} [options.meta = false] whether to include the metadata about the rate
* @param {boolean} [options.startDate = false] whether to include the metadata about the rate
*/
onThisDay(params) {
}
/**
* @param {options}
* @param {function} options.symbols the comma separated list of currencies to consider
* @param {boolean} [options.noCache = false] whether to skip caching (if its enabled in the first place)
*/
currencies(params) {
}
/**
* do a conversion without using the api as its not available in the free tier
* @param {object} swopCxResult an api response parsed as recevied by onThisDay() or latest()
* @param {object} options
* @param {string} options.from the from currency
* @param {string} options.to the to currency
* @param {number} [options.amount=1] the amount to convert
* @returns {object} result
*/
hackConvert(swopCxResult, { from, to, amount = 1 }) {
}
}
const gql = {
// all the gql queries
}
get trap with proxy
As you work through the constructor preparation, it will seem a little complex, but the key thing here is that the body of the class will be clean and simple, and the constructor should be re-usable for any GraphQL API by just modifying the endpoint and Auth.
The constructor
This means that we need to do most of the set up in the constructor by creating proxies, rather than by creating wrapper methods in the class body. This gives the added bonus of only having to expose the methods that are supposed to be public (you can’t have private variables in the version of JavaScript implemented for Apps Script yet – so all class methods are public), and not having to store arguments passed to the constructor as class members. They’ll be visible as closures of the proxy services instead.
The fetcher received as an argument by the constructur is the UrlFetchApp.fetch method. That’s what we’re going to create a proxy for, but first we need to do a bit of prep work.
Standard headers
There’s no need to keep recreating and passing the headers and endpoint to the fetcher, as in this API and GraphQL in general, they’ll be pretty standard, so let’s set them up just once
// these options are standard
const standardOptions = {
method: 'POST',
contentType: 'application/json',
muteHttpExceptions: true,
headers: {
Authorization: `ApiKey ${apiKey}`
}
}
// and the GraphQL url is standard
const url = 'https://swop.cx/graphql'
standard headers
CacheHandlers
The fetcher is also going to work with cache, so before we set up the fetch proxt, let’s go ahead and set up the proxies we’ll need for the cacheservice. Before we do that though, we’ll need a couple of static utility functions to generate keys from content, and to compress and uncompress.
Create a digest to use a key
We’ll use the body of the query and url and a prefix associated with this class. This will make sure its unique and reproducable. A digest will look something like this.
tLacvC5hoiVbhZ3h9YaUPg44Co4=
and will be used as a key to results produced by the query from which it was generated
/**
* make a digest to use as contrstuctor name content addressable cache key
*/
static digest(...args) {
// conver args to an array and digest them
const t = args.concat(['_SwopCx']).map(d => {
return (Object(d) === d) ? JSON.stringify(d) : (typeof d === typeof undefined ? 'undefined' : d.toString());
}).join("-")
const s = Utilities.computeDigest(Utilities.DigestAlgorithm.SHA_1, t, Utilities.Charset.UTF_8)
return Utilities.base64EncodeWebSafe(s)
};
digest as cache key
Zip and Unzip
I generally use lzString for compression so it’s compatible across platforms (Node and Apps Script), but since this class is only intended for Apps Script, we may as well use the zip Utilities to avoid bringing in extra libraries. The result is also encoded/decoded into/from base64 string so that the cacheservice can handle it as a regular string
/**
* zip some content - for this use case - it's for cache, we're expecting string input/output
* @param {string} crushThis the thing to be crushed
* @raturns {string} the zipped contents as base64
*/
static crush (crushThis) {
return Utilities.base64Encode(Utilities.zip([Utilities.newBlob(crushThis)]).getBytes());
}
/**
* unzip some content - for this use case - it's for cache, we're expecting string input/output
* @param {string} crushed the thing to be uncrushed - this will be base64 string
* @raturns {string} the unzipped and decoded contents
*/
static uncrush (crushed) {
return Utilities.unzip(Utilities.newBlob(Utilities.base64Decode(crushed), 'application/zip'))[0].getDataAsString();
}
zip and unzip
Cache apply handlers
We’ll need 2 handlers for the cacheservice proxy, as we need to intercept the handling of .get and .put to do some extra stuff.
cacheGetHandler
This generates a key from the request content, makes a regular cache.get call, then unzips and parses the result if there is one. Note that the arguments it receives are different to the ones regular cache.get would receive. Since it has responsibilty of generating a key, it needs the request content to make it from.
const cacheGetHandler = {
apply(targetFunction, thisArg, args) {
// call the cache get function and make the keys
const digest = _SwopCx.digest(args)
const result = targetFunction.apply(thisArg, [digest])
if (result) {
const uncrushed = _SwopCx.uncrush(result)
const r = JSON.parse(uncrushed)
r.fromCache = true;
return r
}
return null
}
}
cache get handler
cacheSetHandler
Similar to the getHandler, this also needs different arguments to the regular cache.set method so it can make a key from the request content. This adds some timestamp data, stringifies and compresses the data before calling cache.set to write it out.
const cacheSetHandler = {
apply(targetFunction, thisArg, args) {
const [data, expiry, ...keys] = args
const digest = _SwopCx.digest(keys)
const pack = {
timestamp: new Date().getTime(),
fromCache: false,
digest,
data
}
targetFunction.apply(thisArg, [digest, _SwopCx.crush(JSON.stringify(pack)), expiry])
return pack
}
}
cache set handler
cacheHandler
We’re almost ready to create the cache proxy, but first we need to create a handler for that proxy. It needs to intercept calls to cache.get and set to return proxies using the cacheGet and cacheSet handlers
const cacheHandler = {
get(target, property, receiver) {
const value = Reflect.get(target, property, receiver)
if (property === 'get') {
return new Proxy(value, cacheGetHandler)
} else if (property = 'set') {
return new Proxy(value, cacheSetHandler)
}
// just return the regular object
return value
}
}
cache handler
Cache proxy
Now we can create the cache proxy. Any accesses of the get or set property will be intercepted by cacheGet and cacheSet handlers and instead of returning the cache.get or cache.set methods for execution it will return our modified versions of them as a proxy.
// use the modifed cache object instead
const proxyCache = cache && new Proxy(cache, cacheHandler)
cache proxy
Fetcher handler
Now we can get back to creating an apply handler for the UrlFetchApp.fetch method. This will execute in place of the regular fetch method. This orchestrates the enhanced fetch workflow, including caching, error handling and response structuring
// this is the proxy will do we'll do when asked to fetch something
const fetcherHandler = {
apply(targetFunction, thisArg, args) {
const [payloadObject, params] = args
if (args.length !== 2 || typeof payloadObject !== 'object' || !payloadObject) throw new Error('invalid fetch arguments')
const options = {
...standardOptions,
payload: JSON.stringify(payloadObject)
}
const fetchArgs = [url, options]
// first check if it's in cache
let cached = !params.noCache && proxyCache && proxyCache.get(fetchArgs)
let error = null
// if it's not then we have to fetch it, using the regular fetcher
if (!cached) {
// just add the payload to the normal results
const response = targetFunction.apply(thisArg, fetchArgs)
const text = response.getContentText()
const data = text ? JSON.parse(text) : null
if (response.getResponseCode() !== 200) {
error = text
}
cached = error ? null : proxyCache.put(data, cacheSeconds, fetchArgs)
}
// standardize a response for error handling
const d = cached && cached.data && cached.data.data && cached.data.data
const gqlData = d && d[Object.keys(d)[0]]
return {
data: gqlData,
digest: cached && cached.digest,
timestamp: cached && cached.timestamp,
error: error || (!gqlData && cached),
fromCache: cached && cached.fromCache
}
}
}
fetcher handler
Fetcher proxy
Finally, create the fetcher proxy using the handler above
this.fetcher = new Proxy(fetcher, fetcherHandler)
fetcher proxy
Class methods
Now we’re good to go, and can create very simple methods that invisibly use all that stuff we’ve just set up using the fetcher proxy.
/**
* @param {options}
* @param {function} options.symbols the comma separated list of currencies to consider
* @param {string} [options.base ="EUR"] the default base currency - this doesn't work with the free version
* @param {boolean} [options.noCache = false] whether to skip caching (if its enabled in the first place)
* @param {boolean} [options.meta = false] whether to include the metadata about the rate
*/
latest(params) {
return this.fetcher({
query: gql.queries[params.meta ? 'latestMeta' : 'latest'],
variables: {
quoteCurrencies: params.symbols.split(","),
baseCurrency: this.freeTier ? null : params.base || this.defaultBase
}
}, params)
}
/**
* @param {options}
* @param {function} options.symbols the comma separated list of currencies to consider
* @param {string} [options.base ="EUR"] the default base currency - this doesn't work with the free version
* @param {boolean} [options.noCache = false] whether to skip caching (if its enabled in the first place)
* @param {boolean} [options.meta = false] whether to include the metadata about the rate
* @param {boolean} [options.startDate = false] whether to include the metadata about the rate
*/
onThisDay(params) {
if (!params.startDate) throw new Error(`onThisDay needs a startDate parameter in this format YYYY-MM-DD`)
const result = this.fetcher({
query: gql.queries[params.meta ? 'historicalMeta' : 'historical'],
variables: {
quoteCurrencies: params.symbols.split(","),
baseCurrency: this.freeTier ? null : params.base || this.defaultBase,
date: params.startDate
}
}, params)
result.data.forEach(f => f.historical = true)
return result
}
/**
* @param {options}
* @param {function} options.symbols the comma separated list of currencies to consider
* @param {boolean} [options.noCache = false] whether to skip caching (if its enabled in the first place)
*/
currencies(params) {
return this.fetcher({
query: gql.queries.currencies,
variables: {
currencyCodes: params.symbols.split(",")
}
}, params)
}
simple methods
Full class code
Although not especially relevant to the subject, I’ve included the specific GQL query data for this particular API. It may be useful if you are using this method to create your own GraphQL API handler.
class _SwopCx {
/**
* make a digest to use as contrstuctor name content addressable cache key
*/
static digest(...args) {
// conver args to an array and digest them
const t = args.concat(['_SwopCx']).map(d => {
return (Object(d) === d) ? JSON.stringify(d) : (typeof d === typeof undefined ? 'undefined' : d.toString());
}).join("-")
const s = Utilities.computeDigest(Utilities.DigestAlgorithm.SHA_1, t, Utilities.Charset.UTF_8)
return Utilities.base64EncodeWebSafe(s)
};
/**
* zip some content - for this use case - it's for cache, we're expecting string input/output
* @param {string} crushThis the thing to be crushed
* @raturns {string} the zipped contents as base64
*/
static crush (crushThis) {
return Utilities.base64Encode(Utilities.zip([Utilities.newBlob(crushThis)]).getBytes());
}
/**
* unzip some content - for this use case - it's for cache, we're expecting string input/output
* @param {string} crushed the thing to be uncrushed - this will be base64 string
* @raturns {string} the unzipped and decoded contents
*/
static uncrush (crushed) {
return Utilities.unzip(Utilities.newBlob(Utilities.base64Decode(crushed), 'application/zip'))[0].getDataAsString();
}
/**
* @param {options}
* @param {function} options.fetcher the fetcher function from apps script- passed over to keep lib dependency free
* @param {string} options.apiKey the apiKey
* @param {string} [options.defaultBase ="EUR"] the default base currency - this doesn't work with the free version
* @param {boolean} [options.freeTier = true] whether the apiKey is for the limited free tier
* @param {cache} [options.cache = null] the apps script cache service to use
* @param {number} [options.cacheSeconds = 3600] the lifetime for cache
*/
constructor({ fetcher, apiKey, defaultBase = "EUR", freeTier = true, cache = null, cacheSeconds = 60 * 60 }) {
if (!apiKey) throw new Error('apiKey property not provided - goto fixer.io to get one and pass to constructor')
if (!fetcher) throw new Error('fetcher property not provided- pass urlfetchapp.fetch to constructor')
// these options are standard
const standardOptions = {
method: 'POST',
contentType: 'application/json',
muteHttpExceptions: true,
headers: {
Authorization: `ApiKey ${apiKey}`
}
}
// and the GraphQL url is standard
const url = 'https://swop.cx/graphql'
// using these handlers
// the get and set functions are enhanced to make a digest from a set of keys
// so the arguments are not in the normal order
// also decorates the data with timestamps etc and compresses/decompresses it
const cacheGetHandler = {
apply(targetFunction, thisArg, args) {
// call the cache get function and make the keys
const digest = _SwopCx.digest(args)
const result = targetFunction.apply(thisArg, [digest])
if (result) {
const uncrushed = _SwopCx.uncrush(result)
const r = JSON.parse(uncrushed)
r.fromCache = true;
return r
}
return null
}
}
const cacheSetHandler = {
apply(targetFunction, thisArg, args) {
const [data, expiry, ...keys] = args
const digest = _SwopCx.digest(keys)
const pack = {
timestamp: new Date().getTime(),
fromCache: false,
digest,
data
}
targetFunction.apply(thisArg, [digest, _SwopCx.crush(JSON.stringify(pack)), expiry])
return pack
}
}
const cacheHandler = {
get(target, property, receiver) {
const value = Reflect.get(target, property, receiver)
if (property === 'get') {
return new Proxy(value, cacheGetHandler)
} else if (property = 'set') {
return new Proxy(value, cacheSetHandler)
}
// just return the regular object
return value
}
}
// use the modifed cache object instead
const proxyCache = cache && new Proxy(cache, cacheHandler)
// this is the proxy will do we'll do when asked to fetch something
const fetcherHandler = {
apply(targetFunction, thisArg, args) {
const [payloadObject, params] = args
if (args.length !== 2 || typeof payloadObject !== 'object' || !payloadObject) throw new Error('invalid fetch arguments')
const options = {
...standardOptions,
payload: JSON.stringify(payloadObject)
}
const fetchArgs = [url, options]
// first check if it's in cache
let cached = !params.noCache && proxyCache && proxyCache.get(fetchArgs)
let error = null
// if it's not then we have to fetch it, using the proxy fetcher
if (!cached) {
// just add the payload to the normal results
const response = targetFunction.apply(thisArg, fetchArgs)
const text = response.getContentText()
const data = text ? JSON.parse(text) : null
if (response.getResponseCode() !== 200) {
error = text
}
cached = error ? null : proxyCache.put(data, cacheSeconds, fetchArgs)
}
// standardize a response for error handling
const d = cached && cached.data && cached.data.data && cached.data.data
const gqlData = d && d[Object.keys(d)[0]]
return {
data: gqlData,
digest: cached && cached.digest,
timestamp: cached && cached.timestamp,
error: error || (!gqlData && cached),
fromCache: cached && cached.fromCache
}
}
}
// the idea here is add some functionality that's specific to this this class to the fetcher
this.fetcher = new Proxy(fetcher, fetcherHandler)
this.defaultBase = defaultBase
this.freeTier = freeTier
}
/**
* @param {options}
* @param {function} options.symbols the comma separated list of currencies to consider
* @param {string} [options.base ="EUR"] the default base currency - this doesn't work with the free version
* @param {boolean} [options.noCache = false] whether to skip caching (if its enabled in the first place)
* @param {boolean} [options.meta = false] whether to include the metadata about the rate
*/
latest(params) {
return this.fetcher({
query: gql.queries[params.meta ? 'latestMeta' : 'latest'],
variables: {
quoteCurrencies: params.symbols.split(","),
baseCurrency: this.freeTier ? null : params.base || this.defaultBase
}
}, params)
}
/**
* @param {options}
* @param {function} options.symbols the comma separated list of currencies to consider
* @param {string} [options.base ="EUR"] the default base currency - this doesn't work with the free version
* @param {boolean} [options.noCache = false] whether to skip caching (if its enabled in the first place)
* @param {boolean} [options.meta = false] whether to include the metadata about the rate
* @param {boolean} [options.startDate = false] whether to include the metadata about the rate
*/
onThisDay(params) {
if (!params.startDate) throw new Error(`onThisDay needs a startDate parameter in this format YYYY-MM-DD`)
const result = this.fetcher({
query: gql.queries[params.meta ? 'historicalMeta' : 'historical'],
variables: {
quoteCurrencies: params.symbols.split(","),
baseCurrency: this.freeTier ? null : params.base || this.defaultBase,
date: params.startDate
}
}, params)
result.data.forEach(f => f.historical = true)
return result
}
/**
* @param {options}
* @param {function} options.symbols the comma separated list of currencies to consider
* @param {boolean} [options.noCache = false] whether to skip caching (if its enabled in the first place)
*/
currencies(params) {
return this.fetcher({
query: gql.queries.currencies,
variables: {
currencyCodes: params.symbols.split(",")
}
}, params)
}
/**
* do a conversion without using the api as its not available in the free tier
* @param {object} swopCxResult an api response parsed as recevied by onThisDay() or latest()
* @param {object} options
* @param {string} options.from the from currency
* @param {string} options.to the to currency
* @param {number} [options.amount=1] the amount to convert
* @returns {object} result
*/
hackConvert(swopCxResult, { from, to, amount = 1 }) {
// check that the result is a valid one
if (!Array.isArray(swopCxResult)) throw new Error('input swopCxResult should be an array')
// and that it contains both from and to rates
const rateFrom = swopCxResult.find(f => from === f.quoteCurrency)
const rateTo = swopCxResult.find(f => to === f.quoteCurrency)
if (!rateFrom) throw new Error('from currency not found in swopCxResult')
if (!rateTo) throw new Error('to currency not found in swopCxResult')
if (rateFrom.baseCurrency !== rateTo.baseCurrency) throw new Error('base currencies mst match for conversion to work')
const rate = rateTo.quote / rateFrom.quote
const result = rate * amount
const { historical, date } = rateFrom
return {
rate,
historical: historical || false,
date,
result,
to,
from,
amount
}
}
}
const gql = {
get frags() {
return {
fragCurrencyType:
`fragment fragCurrencyType on CurrencyType {
code
name
numericCode
decimalDigits
active
}`,
fragRate:
`fragment fragRate on Rate {
date
baseCurrency
quoteCurrency
quote
}`,
fragMeta:
`fragment fragMeta on Rate {
meta {
sourceShortNames
sourceNames
sourceIds
sources {
id
shortName
name
}
rateType
calculated
calculationShortDescription
calculationDescription
calculation {
pathRate
weight
sourceRates {
sourceId
date
baseCurrency
quoteCurrency
quote
flipped
fetched
source {
id
shortName
name
}
}
}
}
}`
}
},
get queries() {
return {
currencies:
`query ($currencyCodes:[String!]){
currencies(currencyCodes: $currencyCodes) {
...fragCurrencyType
}
}
${this.frags.fragCurrencyType}`,
latest:
`query ($baseCurrency: String, $quoteCurrencies:[String!]) {
latest (baseCurrency: $baseCurrency, quoteCurrencies: $quoteCurrencies) {
...fragRate
}
}
${this.frags.fragRate}`,
latestMeta:
`query ( $baseCurrency: String, $quoteCurrencies:[String!]) {
latest (baseCurrency: $baseCurrency, quoteCurrencies: $quoteCurrencies) {
...fragRate
...fragMeta
}
}
}
${this.frags.fragRate}
${this.frags.fragMeta},
`,
historical:
`query ($date: Date!, $baseCurrency: String, $quoteCurrencies:[String!]) {
historical (date:$date, baseCurrency: $baseCurrency, quoteCurrencies: $quoteCurrencies) {
...fragRate
}
}
${this.frags.fragRate}`,
historicalMeta:
`query ($date: Date!, $baseCurrency: String, $quoteCurrencies:[String!]) {
historical (date:$date, baseCurrency: $baseCurrency, quoteCurrencies: $quoteCurrencies) {
...fragRate
...fragMeta
}
}
}
${this.frags.fragRate}
${this.frags.fragMeta}
`
}
}
}
var Fx = (options) => new _SwopCx(options)
the whole thing
Summary
This library is interchangable with the original Currency exchange rates library for Apps Scrip-optimized for the parsimonious -swop.cx and won’t offer any execution benefits over it, except that the code in this one is now a reusable model for any other GQL API library access.
Links
bmSwopProxy : IDE
bmSwopProxy library: 18ncSp3–MSV1_svV4mnEPJGNepoe06pZfWR1gT1Mi0G9dVrE8a8PYlPw
bmSwopProxy github: https://github.com/brucemcpherson/bmSwopCxProxy
Currency exchange rates library for Apps Scrip-optimized for the parsimonious -swop.cx
Resuscitating the Apps Script execution transcript – JavaScript Proxy and Reflect to the rescue