This is a follow on from the article on Pull libraries inline to your Apps Script project (which you should probably read fiest) to explain a little about how it works.

Page Content hide

1.1 Why do we need to know what’s in the library’s global space

1.2 The library namespace constructor

1.3 Discovering what’s in the library global space via an AST

1.3.1 Getting the Ast body

1.3.2 Make the exports

1.4 Exporting

1.4.1 Classes

1.5 Creating instances

1.6 Accessing each instance

1.7 Versions

1.8 Main project code

1.9 Upgrading

1.9.1 versionTreatment

1.13.2 Example converter and reverter

1.14.2 Super simple cipher library for Apps Script encryption and decryption

1.14.3 Obfuscate text with Apps Script

1.14.4 Import, export and mix container bound and standalone Apps Script projects

1.14.5 Add-on for decrypting columns in Google Sheets

1.14.6 4 ways to customize scrviz info card behavior

The Script API

It uses the Script API to get and update Apps Script project content, but the key to making it work is to be able to detect which variables and functions are exposed in the global space of the library being pulled in. They can be similarly exposed in the refactored version pulled into the main project.

Why do we need to know what’s in the library’s global space

A project that uses a library will access variables and functions in that library via libraryusersymbol.variablename and libraryusersymbol.function name. These are exposed via some magic in the Apps Script library system.

When the library is pulled in to the project, we need to still be able to access exactly the same variables and functions in the same way, so we need a away of identifiying them.

The library namespace constructor

Each imported library is wrapped in a namespace. Here’s how a library is wrapped in a namespace when it’s pulled into a project. For the moment it doesn’t really matter what the library is doing. I’ve pulled out all the code except the global space declarations.

//--project:1SohIo0pSlXSt2SKNf3iX2ZcbzE_N0F4BocnKpGuaXgyxx1X6s1nDhjZK (bmVizzyCache) version:6
//  imported by bmImportLibraries at Sat, 12 Mar 2022 15:36:47 GMT
function __bmVizzyCache_v6 () {
	
	//.... the code - all the files in the libray concatenated
	
}

library namespace wrapper

First, all the scripts in the library are concatenated, then wrapped in a namespace as above – named after the library name and its version.

//--project:1SohIo0pSlXSt2SKNf3iX2ZcbzE_N0F4BocnKpGuaXgyxx1X6s1nDhjZK (bmVizzyCache) version:6
//  imported by bmImportLibraries at Sat, 12 Mar 2022 15:36:47 GMT
function __bmVizzyCache_v6 () {
  
  //--script file:src/cache
  
  const Cache = (() => {
  
    // .... the code
	 
  }
  
//--end:src/cache

//--script file:vizzycache
  var VizzyCache = {
				   
    // .... the code

  }
  
  
//--end:vizzycache

//--script file:src/compress
  const Compress = (() => {
	  
	  // .... the code

  })();
  
//--end:src/compress

//--script file:src/settings
  const QueryDefinition = {
  
	  // .... the code
  };
  
//--end:src/settings

//--script file:node/lzstring
  var LZString = (() => {
   // .... the code 
  })()
  
  
//--end:node/lzstring

//--script file:test/code
  function test() {
   // .... the code 
  }
  
//--end:test/code

//--script file:src/gasser.gs
  
  const Gasser = (() => {
	 // .... the code 
  
  })()
  
//--end:src/gasser.gs

//--script file:classes/GasManifests
  class GasManifests {
   
  
//--end:classes/GasManifests

//--script file:classes/GasManifest
  class GasManifest {
	// the code
  }
  
//--end:classes/GasManifest

//--script file:classes/GitData
  
  const types = ["shaxs", "files", "owners", "repos"];
  
  class GitData {
   	// the code
  }
  
//--end:classes/GitData

//--script file:classes/GitFile
  const decorations = [];
  
  class GitFile {
    // the code
  }
  
//--end:classes/GitFile

//--script file:classes/GitOwner
 
  class GitOwner {
    // the code
  }

//--end:classes/GitOwner

//--script file:classes/GitRepo
  class GitRepo {
  	// the code
  }
  
  
//--end:classes/GitRepo

//--script file:classes/GitShax
  class GitShax {
   	// the code
  }
  
  
//--end:classes/GitShax

 
}
//--end project:bmVizzyCache

concatenated scripts

Discovering what’s in the library global space via an AST

You can use apiData.functionSet.values() to retrieve the name of the functions returned when you interrogated the library script with the Script API, but you can’t get the variable names. To do that, I parse the library code using a JavaScript parser (Acorn).

  /**
   * compile a source file
   * @param {File} file the file to be parsed
   * @return {ParsedFile} {ast, file}
   */
  getAst(file) {
    return {
      ast: acorn.parse(file.source, {
        sourceType: "script",
        ecmaVersion: 12,
        allowReserved: true
      }), // parseScript
      file
    }
  }

parsing the source code

This creates an Abstract Syntax Tree (Ast), which is a tree representation of all the tokens in the source code.

Getting the Ast body

Next we’ll need to extract the Program body from the AST

  /**
   * get interesting body
   * @param {ParsedFile} the file
   * @param {string} the type 'FunctionDeclaration'|'VariableDeclaration
   * @returns {object[]} the wualifying bodies
   */
  getBody(ast, type) {
    return (ast.type === 'Program' ? ast.body.filter(f => f.type === type) : [])
  }

get the program body

Make the exports

Since we are only interested in the global space declarations, we can pick them up from the top branch of the AST body without bothering to traverse the entire tree. Functions and variables are defined slightly differently so we can use the same function to pick up the names in the global space, but with an accessor passed as an argument

  /**
   * pull out the name of the variable/funciton to be exported
   */
  makeExports(ast, type, accessor) {
    return this.getBody(ast, type).map(c => accessor(c))
  }

getting the list of exports

We can now create the list of exports like this

  /**
   * compile the files, and get the names of the exports for each file
   * @param {File[]} files the files to be parsed
   * @return {object[]} 
   */
  getExports(params, files) {
    return files.map(this.getAst)
      .map(({ file, ast }) => ({
        file,
        ast,
        functions: this.makeExports(ast, 'FunctionDeclaration', (f) => f.id.name),
        variables: this.makeExports(ast, 'VariableDeclaration', (f) => f.declarations[0].id.name),
        classes: params.exportClasses ? this.makeExports(ast, 'ClassDeclaration', (f) => f.id.name) : []
      }))
  }

pulling out the exports

Note that inlining will expose a little more than libraries do – libraries only expose variables defined as var and functions. Inlining exposes const,let and var. I have a TODO to add an option to suppress const and let exports to make it completely backwards compatible.

Exporting

Now the wrapped namespace constructor can return this list as its properties, so the final constructor function looks like this

 //--project:1SohIo0pSlXSt2SKNf3iX2ZcbzE_N0F4BocnKpGuaXgyxx1X6s1nDhjZK (bmVizzyCache) version:6
//  imported by bmImportLibraries at Sat, 12 Mar 2022 15:36:47 GMT
function __bmVizzyCache_v6 () {
	
  //.... the code - all the files in the libray concatenated

  // the exports
  return {
    Cache,
    VizzyCache,
    Compress,
    QueryDefinition,
    LZString,
    test,
    Gasser,
    types,
    decorations
  }
  
}

namespace constructor with exports

Classes

You’ll notice that I haven’t exposed the classes defined in the library by default. This is because libraries don’t, and I want to replicate (rather than enhance) the library experience. However, with full access to the ast there’s quite a few nice enhancements you could add. If you’ve inlined your libraries, and you want to be able to access the classes in them, just add the exportClasses: true option like this:

  const files = limporter.getInlineProjectFiles({ 
    scriptId, 
    versionTreatment: 'respect',
    exportClasses: true
   })

export classes

Creating instances

When traversing the tree of dependent libraries, it’s possible that some of them reference the same library. There’s no point in creating the same namespace constructors multiple times, so the script ‘_bmlimport/__bmlimporter_gets’ in the amended project contains a function to distributes instances of each namespace. Remember that in the original library version, each library would be its own instance, so these must remain discrete. Here’s a more complex version that will create a bunch of imported library namespaces

var __bmlimporter_gets = {
  get __bmPreFiddler_v30 () { 
    return new __bmPreFiddler_v30 () 
  },
  get __bmFiddler_v25 () { 
    return new __bmFiddler_v25 () 
  },
  get __bmLibraryReporter_v0 () { 
    return new __bmLibraryReporter_v0 () 
  },
  get __bmLibraryTracking_v7 () { 
    return new __bmLibraryTracking_v7 () 
  },
  get __bmLibraryReporter_v14 () { 
    return new __bmLibraryReporter_v14 () 
  },
  get __cGoa_v37 () { 
    return new __cGoa_v37 () 
  },
  get __cUseful_v130 () { 
    return new __cUseful_v130 () 
  },
  get __bmCrusher_v21 () { 
    return new __bmCrusher_v21 () 
  },
  get __bmUpstash_v6 () { 
    return new __bmUpstash_v6 () 
  },
  get __cGcsStore_v13 () { 
    return new __cGcsStore_v13 () 
  }
}

_bmlimport/__bmlimporter_gets

Accessing each instance

Because each library namespace is isolated from each other, we can assign the imported library instance to the same user symbol that was used in the original library reference.

Here’s an example of an instance of the library being included in a namespace

//--project:13JUFGY18RHfjjuKmIRRfvmGlCYrEkEtN6uUm-iLUcxOUFRJD-WBX-tkR (bmPreFiddler) version:30
//  imported by bmImportLibraries at Sat, 12 Mar 2022 17:01:13 GMT
function __bmPreFiddler_v30 () {
  
  //---Inlined Instance of library 13EWG4-lPrEf34itxQhAQ7b9JEbmCBfO8uE4Mhr99CHi3Pw65oxXtq-rU(bmFiddler version 25)
  const bmFiddler = __bmlimporter_gets.__bmFiddler_v25
  
  //---Inlined Instance of library 1D_lWK-jU53wxMA2-NxSjiyu7Uze_GDDqBKTsQnCgPhyUmmSLv0bfTNPX(bmLibraryReporter version 14)
  const bmLibraryReporter = __bmlimporter_gets.__bmLibraryReporter_v14
  //--script file:Code
  /**
   * some usefule fromts for fiddler
   * this  needs spreadsheet scopes,whereas fiddler is dependency free
   */
  function PreFiddler() {
  
    // track library usage
    Trackmyself.stamp()
    
    const getss = ({ id }) => {
      return id ? SpreadsheetApp.openById(id) : SpreadsheetApp.getActiveSpreadsheet()
    }
  
    // open a sheet
    const getSheet = ({ id, sheetName, createIfMissing = false }) => {
  
      const ss = getss({ id })
      let sheet = ss.getSheetByName(sheetName)
      if (!sheet && createIfMissing) {
        sheet = ss.insertSheet(sheetName)
      }
      return sheet
    }
  
    // open a fiddler and assign a sheet
    const getFiddler = ({ id, sheetName, createIfMissing }) => {
      return new bmFiddler.Fiddler(getSheet({ id, sheetName, createIfMissing }))
    }
  
    return {
      getFiddler,
      getSheet,
      getss,
      Fiddler: bmFiddler.Fiddler
    }
  
  }
  
//--end:Code

//--script file:Trackmyself
  // tracking usage of library snippet to a centralized store
  var Trackmyself = ((trackingOptions) => {
    const track  = bmLibraryReporter.Trackmyself
    
    // so we can get reports 
    return {
      exportUsage: (options = {}) => track.exportUsage({...trackingOptions,...options}),
      currentUserUsage: (options = {}) => track.currentUserUsage({...trackingOptions,...options}),
      stamp: ()=>track.stamp(trackingOptions)
    }
    
  })({
    name: 'bmPreFiddler',
    version: '30',
    failSilently: true,
    singleStamp: true
  })
  
  
//--end:Trackmyself

  return {
    PreFiddler,
    Trackmyself
  }
}
//--end project:bmPreFiddler

referencing library instance

Versions

It may be that you access libraries that themselves access other libraries and so on recursively. If you are respecting or updating deployments (the versions specified by the libraries, or using the latest deployed versions), then you might end up with multiple versions of the same library.

This information is in itself handy as it will probably prompt you to go and upgrade the library references, but let’s assume that we want to keep the versions exactly as referenced. We have to handle multiple versions of the same code. In this example, we have 2 different versions of the bmLibraryReporter library

var __bmlimporter_gets = {
  get __bmPreFiddler_v30 () { 
    return new __bmPreFiddler_v30 () 
  },
  get __bmFiddler_v25 () { 
    return new __bmFiddler_v25 () 
  },
  get __bmLibraryReporter_v0 () { 
    return new __bmLibraryReporter_v0 () 
  },
  get __bmLibraryTracking_v7 () { 
    return new __bmLibraryTracking_v7 () 
  },
  get __bmLibraryReporter_v14 () { 
    return new __bmLibraryReporter_v14 () 
  },
  get __cGoa_v37 () { 
    return new __cGoa_v37 () 
  },
  get __cUseful_v130 () { 
    return new __cUseful_v130 () 
  },
  get __bmCrusher_v21 () { 
    return new __bmCrusher_v21 () 
  },
  get __bmUpstash_v6 () { 
    return new __bmUpstash_v6 () 
  },
  get __cGcsStore_v13 () { 
    return new __cGcsStore_v13 () 
  }
}

multiple versions

In this case, each library namespace references its own instance of a different version of the library, allowing multiple versions to coexist in the same project

function __bmPreFiddler_v30 () {
  
 
  
  //---Inlined Instance of library 1D_lWK-jU53wxMA2-NxSjiyu7Uze_GDDqBKTsQnCgPhyUmmSLv0bfTNPX(bmLibraryReporter version 14)
  const bmLibraryReporter = __bmlimporter_gets.__bmLibraryReporter_v14
  
  /// code....
  
  }

//--project:13EWG4-lPrEf34itxQhAQ7b9JEbmCBfO8uE4Mhr99CHi3Pw65oxXtq-rU (bmFiddler) version:25
//  imported by bmImportLibraries at Sat, 12 Mar 2022 17:01:13 GMT
function __bmFiddler_v25 () {
  
  //---Inlined Instance of library 1D_lWK-jU53wxMA2-NxSjiyu7Uze_GDDqBKTsQnCgPhyUmmSLv0bfTNPX(bmLibraryReporter version 0)
  const bmLibraryReporter = __bmlimporter_gets.__bmLibraryReporter_v0
  
  //// code
  
  
 }

different version of the same library

Main project code

The main project files are replicated untouched (other than the manifest file which is merged with each of the libraries referenced). The final thing that needs to be handled is the global level reference to the imported libraries referenced by the main project (as opposed to the recursively resolved libraries).

  //---Inlined Instance of library 13JUFGY18RHfjjuKmIRRfvmGlCYrEkEtN6uUm-iLUcxOUFRJD-WBX-tkR(bmPreFiddler version 30)
  const bmPreFiddler = __bmlimporter_gets.__bmPreFiddler_v30
  
  //---Inlined Instance of library 1v_l4xN3ICa0lAW315NQEzAHPSoNiFdWHsMEwj2qA5t9cgZ5VWci2Qxv2(cGoa version 37)
  const cGoa = __bmlimporter_gets.__cGoa_v37
  
  //---Inlined Instance of library 1nbx8f-kt1rw53qbwn4SO2nKaw9hLYl5OI3xeBgkBC7bpEdWKIPBDkVG0(bmCrusher version 21)
  const bmCrusher = __bmlimporter_gets.__bmCrusher_v21

_bmlimport/__globals

These declarations are found in _bmlimport/__globals

The project now looks like this

Upgrading

The import process can upgrade and harmonize the versions of libraries it pulls in using the versionTreatment parameter, which has these options

versionTreatment

This describes which version of the referenced libraries to pull, and can take these values

respect – Respect the version mentioned in the manifest referencing the library. This is the default.
upgrade – Upgrade to the latest deployed version. Note that the deployment record for the new IDE is checked here. It doesn’t have access to libraries deployed using the old IDE, so if it can’t find a new IDE deployment it’ll fail with an error like this. If it does, then you could consider doing a sloppyupgrade (see later) as a workaround if you can’t deploy using the new IDE.

Error: Couldn’t find any deployments for 1v_l4xN3ICa0lAW315NQEzAHPSoNiFdWHsMEwj2qA5t9cgZ5VWci2Qxv2:(cGoa) (upgrade:upgrade to the latest deployed versions) – maybe they were deployed with old IDE?

head – Use the latest code of the library whether or not it’s been deployed. This is equivalent to development mode. These will generate console messages like this.

Modified reference to 1U6j9t_3ONTbhTCvhjwANMcEXeHXr4shgzTG0ZrRnDYLcFl3_IH2b2eAY:(cCacheHandler) from version 18 to head ((head:use the latest version whether or not deployed))

sloppyupgrade – The same as upgrade, but if it can find no deployed versions or the deployed version is lower than the one in the manifest it will do a head. This could generate a console message like this

Sloppy upgrade of reference to library 1dajqLysdKo8IoqddtEaGhtUUlSbtSQ1Agi2K5cXSUm0DxXfLYouSO9yD:(bmRottler) from version 10 to head (use the latest deployed version if there is one – otherwise use the head)

The manifest

By bringing in all these libraries, it’s possible that they have dependencies (other than libraries) such as oauth scope requirements. The importing process merges all the library manifests (and upgrades the run time to v8 by default), so the final manifest will contain a melange of all the library manifest requirements.

Reverting

It may be necessary to revert to the library version of a project. During import, it creates a file called __bmlimport/manifest.json. This is just a copy of the original manifest. Therefore it’s very straightforward to revert to the library version, simply by copying the contents into your appsscript.json, then deleting everything in __bmlimport/*. The library also provides a .revert() method that does all that for you.

Oddities

If one of the libraries you are importing is not really Apps Script, but something taken from webpack style code, then the exports from the library are not visible as normal to the JavaScript parser, so it can’t be properly exported. Here’s what one of these looks like.

(function (global, factory) {
	typeof exports === 'object' && typeof module !== 'undefined' ? factory(exports) :
	typeof define === 'function' && define.amd ? define(['exports'], factory) :
	(factory((global.acorn = {})));
}(this, (function (exports) { 
	
	/// the code
	
})));

code imported from webpack etc..

The key here is the argument ‘global’, which receives ‘this’ as the argument value. When it is in a library ‘this’ refers to the global space of the library, but when it’s inlined, then ‘this’ refers to the global space of the project, which means it won’t be detected in the correct scope by the inline parser.

The simple workaround is to expose the entry point explicitly in the library. In this case, you can see that the entry point passed to the factory is ‘global.acorn’, and already know that global is in fact the ‘this’ of the library. So we can propery expose the entry point with a var declaration.

	typeof exports === 'object' && typeof module !== 'undefined' ? factory(exports) :
	typeof define === 'function' && define.amd ? define(['exports'], factory) :
	(factory((global.acorn = {})));
}(this, (function (exports) { 
	
	/// the code
	
})));
var acorn = this.acorn

exposing the entry point

This won’t affect the library at all, but it will make that entry point visible when it’s inlined