Measuring library load speed

There's a lot of talk about whether or not to use libraries because of performance issues. When you just load the library once, it probably doesn't matter, but if you are using Html service, every time a server function is called, the server script is initialized - loading all the source and the libraries again. So certainly in theory, this feels like it should be an important discriminator.

There are certainly other reasons for not using libraries in add-ons, such as creating unnecessary dependencies on libraries which might disappear - so I think the advice is absolutely solid. 

Is performance really the problem ?

It's called out as a massive no-no in the Add-on documentation like this. But is loading from libraries really a lot worse than uploading the same amount of source code as part of the script ? 


Also, since google.script.run calls are asynchronous, you can  run many of them at the same time - but the other question I wanted to find the answer to was whether it becomes counterproductive with a certain number of calls in progress - do multiple parallel calls interfere with each other?

So I decided to test all that.

The results


Findings

In summary this means that I find no real penalty for loading libraries. In fact, with parallelism the library versions seem to perform a little better than when all the code is local. I can't explain it but it seems consistent. 

The chart shows the results of 3 flavors of test.
  • Where all the source was local in a container bound project. I copied in the source of 3 chunky libraries into the source code.
  • Where the the libraries were accessed as libraries from a container bound project.
  • Where the the libraries were accessed as libraries from an add-on.

For each example above I called a server function 100 times using various levels of parallelism. That means I allowed 1,5,10 and 20 instances of the server call to run at the same time in each of the tests. The timings are the average round trip time for a single call to the server functions (the light blue, purple and green)  and the  yellow, red and blue are the overall time to run the 100 calls.

The server functions did nothing, aside from returning to the client so the round tip time could be measured. 

  ns.provoke = function (stats) {
    stats.receivedByServer = new Date().getTime();
    return stats;
  };

Waking up the server function should have provoked all the code to be loaded from libraries or locally each time. Since these calls were asynchronous, I did multiple simultaneous server calls to see the effect  of that too. I discovered that the average round trip call for more than 10 parallel threads outweighs the elapsed time benefit for running more things at the same time. So between 5 and 10 seems about the optimum number of threads to be trying to manage. If the server was doing something more substantial, its possible that this balance could change.

Server time versus client time. 

As little wrinkle, I noticed that the server time is out of synch with the client time by about half a second. Originally I had planned to use the difference between the time the server stared running its function 
  ns.provoke = function (stats) {
    stats.receivedByServer = new Date().getTime();
    return stats;
  };

and the time the client requested it as the measurement (receivedByServer - initiatedByClient), to remove some of the variability an internet connection might introduce
  stat = {
    initiatedByClient:new Date().getTime(),
    receivedByServer:0,
    receivedByClient:0
  };

but because of this out of sync problem I used the complete round trip time (receivedByClient - initiatedByClient).
  .withSuccessHandler ( function (updated) {
     updated.receivedByClient = new Date().getTime();
        //...

   })

as the receivedByServer was actually later than the subsequent receivedByClient, meaning that the server time is running a little ahead of the client time.


 The code for the addon version of the test follows if you want to play around with it, and is also on github

The code

You can load whichever libraries you like to test. However it needs the cUseful library (copied locally for the local test, or used as a library for the library test). For details, see A functional approach to fiddling with sheet data. This follows my usual pattern for a sidebar app/spreadsheet add-on. It's also on github.

Addon.gs
/**
 * test if libraries slow things down
 */
function onOpen () {
  return SpreadsheetApp
  .getUi()  
  .createAddonMenu()
  .addItem('go', 'kickoff')
  .addToUi();
}


function onInstall () {
  return onOpen();
}
/**
* given an array of .gs file names, it will get the source and return them concatenated for insertion into htmlservice
* like this you can share the same code between client and server side, and use the Apps Script IDE to manage your js code
* @param {string[]} scripts the names of all the scripts needed
* @return {string} the code inside script tags
*/
function requireGs (scripts) {
    return '<script>\n' + scripts.map (function (d) {
      // getResource returns a blob
      return ScriptApp.getResource(d).getDataAsString();
    })
    .join('\n\n') + '</script>\n';
}
/**
* given an array of .html file names, it will get the source and return them concatenated for insertion into htmlservice
* like this you can share the same code between client and server side, and use the Apps Script IDE to manage your js code
* @param {string[]} scripts the names of all the scripts needed
* @return {string} the code inside script tags
*/
function requireJs (scripts) {
    return '<script>\n' + scripts.map (function (d) {
        return HtmlService.createHtmlOutputFromFile(d+".js").getContent();
    })
    .join('\n\n') + '</script>\n';
}

function kickoff () {

  SpreadsheetApp.getUi() // Or DocumentApp or FormApp.
      .showSidebar(
        HtmlService.createTemplateFromFile('index')
        .evaluate()      
        .setSandboxMode(HtmlService.SandboxMode.IFRAME)
        .setTitle('library load testing')
      );
  
}


Server.gs

function expose (namespace , method) {
   return this[namespace][method]
  .apply(this,Array.prototype.slice.call(arguments,2));
}

/**
 * runs on the server side
 * for testing whether libraries have an effect
 * @namespace Server
 */
var Server = (function (ns) {

  /**
   * doesnt do anything except provoke a script load 
   * and return some stats
   * @param {object} stats stats object
   * return {object} stats updated stats object
   */
  ns.provoke = function (stats) {
    stats.receivedByServer = new Date().getTime();
    return stats;
  };
  
  /**
   * writes the stats to a sheet
   * @param {[object]} the stats
   * @param {number} elapsed pverall elapsed time
   * @param {string} where to write the sheet name
   * @return {void}
   */
  ns.log = function (stats,elapsed, sheetName) {
    
    //where to write the data
    var ss = SpreadsheetApp.getActiveSpreadsheet();
    
    // create the sheet if necessary
    var sheet = ss.getSheetByName(sheetName) || ss.insertSheet(sheetName);
    var fiddleRange =  sheet.getDataRange();
    
    // get a fiddle and set the data
    var fiddler = new cUseful.Fiddler()
    .setData (stats);
    
    // clear and write the result
    fiddleRange
    .getSheet()
    .clearContents();
 
    fiddler
    .getRange(fiddleRange)
    .setValues(fiddler.createValues());
    
    // analyze the results on a new sheet
    var sheet = ss.getSheetByName('summary') || ss.insertSheet('summary');
    
    fiddler
    .setValues(sheet.getDataRange().getValues())
    .getData().push ( {
      "name":sheetName,
      "start to finish":stats.reduce(function(p,c) {
        return p+c.receivedByClient-c.initiatedByClient;
      },0)/stats.length,
      elapsed:elapsed
    });
  
    fiddler.setData(fiddler.getData());
    
  
    var fiddleRange =  sheet.getDataRange();

    // clear and write the result
    fiddleRange
    .getSheet()
    .clearContents();
 
    fiddler
    .getRange(fiddleRange)
    .setValues(fiddler.createValues());
    
    return stats;
  };
  
  return ns;
  
}) (Server || {});

Client.gs
/**
* runs on the client side
* for testing whether libraries have an effect
* @namespace Client
*/
var Client = (function (ns) {
  
  /**
  * schedule another poke
  * @param {function} func a func to pass to log
  * @return {object} stat
  */
  ns.schedule = function (func) {
    
    var stat;
    // throttle the max absolute and parallel number of requests
    if (App.globals.stop || App.globals.stats.length >= App.globals.MAX_REQUESTS) {
      
      // we're all done
      if (App.globals.log && (!inProgress_().length || App.globals.stop)) {
        ns.log(func);
      }
      
    }
    else if (inProgress_().length < App.globals.divs.parallel.value) {
      stat = {
        initiatedByClient:new Date().getTime(),
        receivedByServer:0,
        receivedByClient:0
      };
      stat.index = App.globals.stats.push (stat)-1;
    }

    return stat;
  };
  
  function inProgress_ () {
    return App.globals.stats.filter(function (d) {
      return !d.receivedByServer;
    });
  }
  /**
   * fill up the request queue initially
   */
  ns.startPoking = function () {
    
    // finish anything still running
    if (inProgress_().length) {
      App.reportMessage('aborting...');
      App.globals.stop = true;
      ns.schedule (ns.startPoking);
    };
    
    App.reportMessage('starting');
    App.globals.stop = false;
    App.globals.stats = [];
    App.globals.start = new Date().getTime();
    while (ns.provoke()) {
      
    }
  };

  /**
  * provoke a server activity and time it
  * @return {object} the provoked item
  */
  ns.provoke = function () {
    
    // kick off some pokes
    
    var stat = ns.schedule();
    if (stat) {

      google.script.run
      
      .withFailureHandler ( function (err) {
        App.reportMessage (err);
        App.stop = true;
      })
      
      /**
      * called back from server
      * will record time recevied back
      * and potentially push another
      */
      .withSuccessHandler ( function (updated) {
        // store result
        updated.receivedByClient = new Date().getTime();
        App.reportMessage('got ' + updated.index);
        App.globals.stats[updated.index]=updated;
        
        // rey to do some more
        ns.provoke();
      })
      
      .expose('Server','provoke', stat);
    }                      
    
    return stat;
  };
  
  /**
  * log results
  * @param {function} [func] something to run on completion
  */
  ns.log = function (func) {
    
    google.script.run
    
    .withFailureHandler ( function (err) {
      App.reportMessage (err);
    })
    
    
    .withSuccessHandler ( function (stats) {
      App.reportMessage ('did ' + App.globals.stats.length + ' pokes');
      if (func) { 
        func();
      }
    })
    
    .expose(
      'Server', 
      'log', 
      App.globals.stats , 
      new Date().getTime() - App.globals.start ,
      App.globals.prefix+ App.globals.divs.parallel.value
    );                    
    
  };
  
  return ns;
})(Client || {});



main.js.html
window.addEventListener("load", function () {

  // set up the app
  App.init();

});

styles.css.html
<!-- This CSS package applies Google styling; it should always be included. -->
<link rel="stylesheet" href="https://ssl.gstatic.com/docs/script/css/add-ons1.css">

<style>
.content {
  padding:8px;
}

.message {
  color:red;
}


.aside {
  font-size:.7em;
  color:gray;
}

td {
  padding:2px;
}

</style>

index.html
<!DOCTYPE html>
<!-- styles -->
<?!= HtmlService.createHtmlOutputFromFile('styles.css').getContent(); ?>

<h2>measure library effect</h2>
<div class="content">

  <div class="block">
    <label for="parallel">How many instances to run at once</label>
    <select class="block" id="parallel">
    <option value=1>1</option>
    <option value=5>5</option>
    <option value=10>10</option>
    <option value=20 selected>20</option>
    </select>
  </div>  

  <div class="block">
    <button class="action" id="test">Test</button>
    <button onclick="google.script.host.close()">Close</button>
  </div>

</div>

<div class = "block">
  <div id="message" class="message"></div>
</div>

<!-- javascript. -->
<?!= requireJs(['main']); ?>
<?!= requireGs(['Client','Server','App']); ?>

App.gs
var App = (function (ns) {
 
  // static for use on both client and server
  ns.globals = {
    stats:[],
    MAX_REQUESTS:100,
    log:true,
    stop:false,
    start:0,
    prefix:'addon-library-parallel-'
  };
  
  // for use on client side.
  ns.init = function () {
    ns.globals.divs = {
      message:document.getElementById('message'),
      parallel:document.getElementById('parallel'),
      test:document.getElementById('test')
    };
    
    ns.listeners();

  };
  
  /**
  * report a message
  * @param {string} message the message
  */
  ns.reportMessage = function (message) {
    ns.globals.divs.message.innerHTML = message;
  };
  
  /** 
   * add listeners
   */
  ns.listeners = function () {
    
    ns.globals.divs.parallel.addEventListener (
      "change", function (e) {
        Client.startPoking();
      },false);
    
    ns.globals.divs.test.addEventListener (
      "click", function (e) {
        Client.startPoking();
      },false);
    
  };
  
  return ns;
  
}) (App || {});

For more like this, see Google Apps Scripts snippets. Why not join our forumfollow the blog or follow me on twitter to ensure you get updates when they are available. 

You want to learn Google Apps Script?

Learning Apps Script, (and transitioning from VBA) are covered comprehensively in my my book, Going Gas - from VBA to Apps script, available All formats are available now from O'Reilly,Amazon and all good bookshops. You can also read a preview on O'Reilly

If you prefer Video style learning I also have two courses available. also published by O'Reilly.
Google Apps Script for Developers and Google Apps Script for Beginners.






Comments