Running things in parallel using HTML service was a brief intro on how to run a number of things at once, orchestrating executing using Google Apps Script HTML service. Here’s how to set it up. You’ll need to run this as a container bound script, since we are going to visualize the activity in a spreadsheet sidebar.

Orchestration profile

The action is controlled by an orchestration profile. Here’s the one for the video in Running things in parallel using HTML service
 
Explanation
  • Tasks are ordered into arrays of things that can be run in parallel. So the example above is allowing a1,b1,a2 to be run together followed by ‘reduction’ when they are completed.
  • Each task should have a name (to identify it on the progress chart), and a functionName that points to a Google Apps Script function to be run. The skip property can be set to true if you want to omit anything from actually running. This allows you to run subsets of the profile without having to make a new one.
  • You’ll find the debug property invaluable. It’s very difficult to debug html service since it is sanitized by caja. The code you see chrome developer tools is nothing like the code you wrote. Also its hard to get visibility into what’s happening on the GAS side since the execution log gets overwritten. If you turn debug on, you’ll see lots of stuff in the console window of the developer tools showing you whats running, using what options, and what data it’s using.
  • Options can be anything you want. This is passed over to the Google Apps Function that is to be run. In addition to options, the data from the previous function block is automatically passed over too. This is the mechanism for passing results between one stage and another
Here’s the code for this demo – in fact it doesn’t do anything – just demonstrates the principle.
The first block – the map functions
 

The second block – a reduce function. This is a very common operation – so the same reduce function can probably be re-used for most applications. Note that it get passed any options from its profile, along with all the data from each of the map operations executed in the first block. It’s job is simply to consolidate each of these results into one.

 

Here’s what a completed progress sidebar looks like. Note that each task shows how long it took to complete – This example shows that we got 23 seconds of compute time done in 10 seconds through parallel running. All the bars are green so nothing failed. You’ll notice the bars changing colors at different stages of execution.

A more complicated example

If you use Database abstraction with google apps script libraries from this site you’ll know that they present a common interface to a number of back ends. This means that testing them against the same data set and queries is complicated and I usually hit quota limitations – so I have to break them up and so on. Using this method automates all that, runs all the tests at the same time, and also helps simulates multiuser access. Her’s a video of a test of 5 back ends simultaneously.
You’ll notice that there were multiple map and reduce steps in this example. Here’s what happened
  • Multiple threads to create a sets of test data
  • A reduce operation to combine testdata into one dataset
  • Multiple threads to execute the same test on each of the database backends
  • A reduce operation to combine the test results into one dataset
  • A logging operation to output the test results to a spreadsheet. v
In this example, we got 123 seconds worth of computing done in 56 seconds. You’ll also notice that I also recorded one of the database update threads (SHEET) in action in the video as it used the in focus spreadsheets as its back end database.
The only difference between the demo example and this one is the profile and the map modules. I used the same reduction function throughout. Here’s the result of a chunkier test. This one would have failed if I’d done the operations sequentially as it breaks the Apps Script 6 minute quota, but with parallel running it all got done in a couple of minutes
Here’s the profile for running this. 
Most of the content is the various parameters for setting up different kind of database access.
 
and the functions that are called
block 0
 

block 1

 

block 2

 

block 3

Repeat the reduction function

block 4

 

Setting up your own version of this

I recommend a structure like this

Code.gs
This is all about setting up the sidebar and kicking of execution. The only change you need to make here is to call the function that creates the profile for your job.
 
profiles.gs
These are the profiles for whatever you want to run. The demo profile looks like this. My dbProfile() example was shown earlier.
 
process.gs
App script functions that are referenced in your profile. For the demo example, you’ll need the first 3 functions
 
asyncService.html
This is the sidebar template. You may want to play around with this, but the results and canvas divs are needed as are the two include references
 
canvas.js.html
This is the template Javascript for rendering the progress bars. You shouldn’t need to change this at all unless you want to play with the colors or bar sizes.
 
async.js.html
This orchestrates everything
 
dbTests.gs
These are specific to my use case for database testing. If you want to see the code for that let me know and I’ll share it with you. In this module you would write the functions specific to your application.

For more snippets like this see Google Apps Scripts snippets