This piece of work to create inline libraries was both challenging and a lot of fun to produce, and it’s something I’ve wanted to get round to for a long time. Apps Script libraries are a great way to reuse work you and others have done, but you have to be careful that they don’t get out of date. Libraries that refer to other libraries are complex to keep up to date, and worst of all they may disappear or their permissions change at any time.
Google recommend you don’t use them in Add-ons, and although they focus on ‘load efficiency’ (I did a study on this a few years back and found absolutely no evidence of a measurable load penalty for libraries: see Measuring library load speed ), the above are probably better reasons to bundle all the code you use in your app or Add-on.
This builds on the work from Import, export and mix container bound and standalone Apps Script projects and I release it as a library (which you can of course use to inline it to your own project if you want!). Library references at the end of the article
Capabilities of Inline libraries
I’ve tried to make this as painless and applicable as widely as possible, so it provides the following capabilities
- Recursive libraries are supported – so you can inline libraries which in turn reference other libraries. All the code for all libraries will be imported into your project
- Multiple versions of the same library can co-exist. This means that if your app uses library A, B, and C and library B references library D version X, and library C references library D version Y both versions will be retained.
- No changes are needed to the userSymbol reference to the library whether it’s inline or a library, even if different versions are referenced, and all the same functions and variables are exposed
- The manifest files of project and all the underlying libraries are merged so that any oauthScopes, Advanced Services and so on they need will be specified in the main project manifest.
- Because libraries can run with different runtimes, some syntax might be valid (for example in legacy Apps Script) but invalid in V8, and visa versa, so would cause problems when inlined to a single project. Therefore the manifest by default will be upgraded to V8, but you can to choose a alternative runtime.
- Library versions can be upgraded when imported – more of that later.
- The original manifest is saved in the project, so you can simply either manually reinstate it and delete the inlined files to revert to using libraries or use the .revert() method which will do it for you.
How to use
Once you’ve installed the bmLimport library, the options are the same as those for the bmImportScript library described here Import, export and mix container bound and standalone Apps Script project
After initial preparation, there’s only 3 steps.
- get an instance
- execute a function to inline the files and pepare the manifest
- write out the updated project
Here’s an example converter you can use to inline any of your projects. Just modify the scriptIds
Preparation
This library uses the Apps Script API to pull code from Scripts to which you have access. The Apps Script API is neither an Advanced Service nor a built in service of Apps Script so we first have to enable it in the cloud project associated with the Apps Script project you’re running this from.
Officially, you’d create (or reuse) a cloud project you’ve created in the cloud console, turn on the Apps Script API, and then change the GCP project in your Apps Script project settings to the project number of the GCP project. You may also have to enable billing when you do that (although the use of the API is currently free)
Unofficially though, I’ve found you can just let it fail and you’ll get a link that’ll take you to your Apps Script managed project and use that to get the secret cloud project, then turn on the API in that one. You should also remember the project ID somewhere as it’s always hidden in the Apps Script settings. It also means you don’t have to enable billing which is nice. (Since writing this I believe this loophole has been somewhat closed – although you do get a link to the apps script console project on failure, the console directs you to your last opened non Apps Script project. If anybody finds a workaround – let me know!)
Just to be clear – you don’t have to do any of this for the projects you are converting. Just one time for the the converter script.
Scopes
Once you’ve enabled the API, you’ll need these scopes in your appsscript.json – the same as required for bmImportScript
For manipulating the project content
For examining library deployment info
Instantiation
The constructor requires the same configuration as described in Import, export and mix container bound and standalone Apps Script projects.
Here’s an example instantiation. Here I’m using caching, but if there’s a chance that there’s been a recent upgrade to some of the libraries, then just omit that property
The inlined library project files
The parameters are
scriptId
The input project scriptId
noCache
Whether to disable cache – default false. Cache can be useful to defeat the ScriptAPI rate limits if you are doing repeated testing. Only relevant if you instantiated with a cacheService
versionTreatment
This describes which version of the referenced libraries to pull, and can take these values
-
respect – Respect the version mentioned in the manifest referencing the library. This is the default.
-
upgrade – Upgrade to the latest deployed version. Note that the deployment record for the new IDE is checked here. It doesn’t have access to libraries deployed using the old IDE, so if it can’t find a new IDE deployment it’ll fail with an error like this. If it does, then you could consider doing a sloppyupgrade (see later) as a workaround if you can’t deploy using the new IDE.
Error: Couldn’t find any deployments for 1v_l4xN3ICa0lAW315NQEzAHPSoNiFdWHsMEwj2qA5t9cgZ5VWci2Qxv2:(cGoa) (upgrade:upgrade to the latest deployed versions) – maybe they were deployed with old IDE?
-
head – Use the latest code of the library whether or not it’s been deployed. This is equivalent to development mode. These will generate console messages like this.
Modified reference to 1U6j9t_3ONTbhTCvhjwANMcEXeHXr4shgzTG0ZrRnDYLcFl3_IH2b2eAY:(cCacheHandler) from version 18 to head ((head:use the latest version whether or not deployed))
-
sloppyupgrade – The same as upgrade, but if it can find no deployed versions or the deployed version is lower than the one in the manifest it will do a head. This could generate a console message like this
Sloppy upgrade of reference to library 1dajqLysdKo8IoqddtEaGhtUUlSbtSQ1Agi2K5cXSUm0DxXfLYouSO9yD:(bmRottler) from version 10 to head (use the latest deployed version if there is one – otherwise use the head)
Writing the updated project
The sapi property of your bmLimport instance contains a handle to a bmImportScript instance, so you can do everything you can do in Import, export and mix container bound and standalone Apps Script projects.
Typically you’ll just do this
A converted project
Here’s the gasgit project before and after inlining
Here’s what’s happened
- The original source files are retained unmodified
- The referenced library is examined and all the libraries it references are recursively examined. The code for all the required libraries are pulled into a folder called _bmlimport/ and named according to their project name and version.
- A new appsscript.json is created merging all the library manifests with the original one.
- The original appsscript.json is moved to _bmlimport/manifest.json
- reference to the top level library is added to _bmlimport/__globals
- A factory for producing instances of the library is added to _bmlimport/_bmlimporter_gets
I won’t go into the details of how the underlying functions are exposed and how the library code is wrapped up in this article, but will leave that to another article later.
How to revert
If you want to revert the project to use libraries again, you can simply restore the manifest from _bmlimport/manifest.json, overwriting the appsscript.json, then delete all files in _bmlimport/*
However, the library has a revert() method to do all that for you. Here’s an example revert script
There’s also a refresh method which will revert your project to library mode, and re-inline the upgrade libraries. Here’s an example of that.
Some thoughts on workflow
You are of course writing destructively to a script project so ALWAYS TAKE A BACKUP if you plan to write back to the same project.
Another approach is to create a new project (you can use limporter.sapi.createProject({ title: ‘some name’}) and write the inlined project to that. Using a workflow like this, you could have the concept of a development version (Using libraries) and a dist version (with the libraries inlined) which you overwrite from the development version using bmLimporter.
I’ve noticed also that the IDE can get occassionally get confused when you change the content of a project it has open. Best to close it and open it again before executing the inlined code (or run the converter with the target project closed)
Another benefit of inlining is that you debug more effectively since all the code being executed is part of your project, so you can set breakpoints and look at what each of the libraries are doing. You can always revert to libraries once you’re done if you want.
Testing
It’s quite tough to test something like this on anything other than your own libraries, so I went to the list of libraries discovered by Find an Apps Script library id in 10 seconds with scrviz
There’s almost 300 of them, so I inlined them all into a test project. You can see the list in the limports tab of this spreadsheet along with the results of the inlining. The import-files column either shows the number of files that were added to the output project, or an error message. All the errors are as expected, and are one of these
- Error: circular reference to… this is where a library references itself. This is not supported and applied to 4 of the 300
- Error: The caller does not have permission… the library is owned by someone else and hasn’t been shared publicly so I couldn’t test it
- Error: Requested entity was not found… the library no longer exists
- Error: Request contains an invalid argument... the libraryId (and therefore the appsscript,json from which it was extracted) is not a valid
- Error: Syntax error: SyntaxError: Unexpected token ‘class’… a reserved word has been used as a variable name, which isn’t allowed in V8
- Error: Service not found: urlshortener v1... library references an Apps Script service that no longer exists
Note that if any errors are thrown, the project is not written at all to avoid rubbish being written to the project
Please let me know of any other exceptions you come across via issues in the github repo for this project
Oddities
If one of the libraries you are importing is not really Apps Script, but something taken from webpack style code, then the exports from the library are not visible as normal to the JavaScript parser, so it can’t be properly exported. Here’s what one of these looks like.
The key here is the argument ‘global’, which receives ‘this’ as the argument value. When it is in a library ‘this’ refers to the global space of the library, but when it’s inlined, then ‘this’ refers to the global space of the project, which means it won’t be detected in the correct scope by the inline parser.
This won’t affect the library at all, but it will make that entry point visible when the library is inlined.
Next
I think idea of inline libraries this can form the basis of module management for Apps Script. We know that importing and exporting are not supported (and maybe they won’t be – who knows), but for now we can use this as a kind of npm/yarn to pull in referenced libraries plus all the underlying libraries they themselves reference.
The next step will probably be to allow some kind of configuration file – like package.json in node – that can specify the rules about library inclusion without relying on the Apps Script library management and deployments to trigger inlining. Would love to hear your thoughts via the github repo.
Links
The library
bmLimporter 18vZDGy1YTRsMSraL2Rj2haSrJzRegIUv2mCgMwDHFu5uobx6chyXnQV8
Example converter and reverter
testBmLimporter 11ktzauPNsZ2jq2EO4dhO9T0Rs-3qBn_lqwMNZc4osv1Myi7qD7dLd2j2
All are also on github in their respective repos at github.com/brucemcpherson
Also worth reading Import, export and mix container bound and standalone Apps Script projects.