Making sense of gwt rpc dependency service


THIS POST IS NOW PRETTY MUCH OBSOLETE. AS OF APRIL 2016, THE DEPENDENCY SERVICE SEEMED TO DISAPPEAR. IM NOW USING A HACKED AUTOCOMPLETE SERVICE TO DISCOVER DEPENDENCIES. WILL POST ON THIS AT A FUTURE DATE.

Thanks to some great detective work by Spencer Easton, it seems that it is possible to figure out systematically which libraries a particular script is using. This is important because we need to ensure that the library sources are properly uploaded to Git along with the projects that use them. Up till now I've been using Get GAS library info, which is fine but there is maintenance around keeping that up to date.

The key to this is the GWT RPC Service and the Wire Protocol it uses. It is close to unintelligible from the docs I could find about it, but Brian Slesinsky has done some reverse engineering using Chrome Dev tools which gives some pointers as to what is needed.

Spencer Easton figured out that the Apps Script IDE is using GWT RPC to find out library dependencies, so hacking around with that seems worth looking at. 

This is part of the gasGit project which is on gitHub here. You can also find it as a standalone library on github here.

As usual, I'm creating a library to simplify this - so far it looks like this (hover to view - see Dynamic height github gadget for Google Site for how this works)



I'm calling it like this
function t() {

  var ds = new DependencyService().setKey("1jw6jaQaCHI1LYVUlxfk1jKQNWTq9j60xoHP5dxV5AwdqI7B22mVJKwfA");

  var result = ds.getDependencies();
  
  if(result.success) {
    Logger.log(JSON.stringify(result.data));
  }
  else {
    Logger.log(JSON.stringify(result));
  }
  
}

Spencer and I have figured out how to extract private libraries, so for every referenced library in a project we can figure this out.

{
development: false,
identifier: "cDependencyService",
key: "Me90hDkr73ajS2dd-CDc4V6i_d-phDA33",
sdc: "3f412eceec31aa38",
library: "cDependencyService",
version: "11",
known: true
}

And for all the enabled Google services we can figure this out

{
identifier: "Drive",
library: "drive",
version: "v2"
}

For help and more information join our community,  follow the blog,  follow me on twitter, or follow me on g+

What is this format anyway

The raw format you get back from a GWT query looks something like this. It's kind of like JSON but not quite. That long list of numbers is actually a map. To avoid repeating strings, there is a position map - so the key to this is to read the strings via the position map. The positions point to (base 1), the string that should be at that each position. Sometimes its a 0 or a negative number. I don't know what this represents so I just ignore them. Essentially each string in the array towards the end appears only once. You use the position map to sort them into the right order and duplicate them if necessary.

 {
        content=//OK[
            'HgYSFYl',
            90,
            89,
            88,
            87,
            86,
            56,
            23,
            22,
            -17,
            17,
            85, etc...
  
            [
                "e",
                "1j",
                "9",
                "a",
                "cDataHandler",
                "i",
                "Mj61W-201_t_zC9fJg1IzYiz3TLx7pV4j",
                "j",
                "c8d678a2b50616e1",
                "b",
                "c",
                "5",
                "cPoll",
                "MhwYWpncx2kNyR0sckR24mqi_d-phDA33",
        etc..
            ],
            1,
            7
        ],

Identifying custom libraries.

Since writing this, I've discovered that the format actually changes fairly regularly, so the detail of below won't be exactly correct at various points in time, but I'll leave it here to give the general picture.

Take a look at the source code which is commented with the latest reverse engineered format. 

Starting from the end of the mapped data, each custom library block can be identified by a "9" and google services by an "8". There can be differences between blocks- the reason for which i haven't fully figured out - but the multiple formats are dealt with by the cDependency class.

A typical block looks like this - don't forget it starts from the end. I may publish a more detailed spec of what I've discovered if there is any interest.

example2nd and subsequent libslocationassigned propertynotes
"10",p-10version
-9,p-9ignore
"b",p-8check=b
"cUseful",p-7library
"ecf6642f980ce639",p-6sdc
"j",p-5check=j
"Mcbr-v4SsYKJP7JMohttAZyz3TLx7pV4j",p-4key
"i",p-3check=i
"cUseful",p-2identifier
-5,p-1ignore
"9",pmarker for end of dependency section

For more on drive SDK see Using Drive SDK

You want to learn Google Apps Script?

Learning Apps Script, (and transitioning from VBA) are covered comprehensively in my my book, Going Gas - from VBA to Apps script, All formats are available from O'ReillyAmazon and all good bookshops. You can also read a preview on O'Reilly

If you prefer Video style learning I also have two courses available. also published by O'Reilly.
Google Apps Script for Developers and Google Apps Script for Beginners.

Comments