Running things in parallel using HTML service was a brief intro on how to run a number of things at once, orchestrating executing using Google Apps Script HTML service. In Some hints on setting up parallel running profiles I showed how to set up some complex profiles. We’ll use them as the basis for this example, so you should read that first.

In Summarizing emails to a sheet I showed how to search your mail and write a summary of the results to a sheet using Database abstraction with google apps script. The problem is that Gmail is kind of slow and rate limited, so you end up not being able to get enough message details within the 6 minute deadline. We can use Running things in parallel using HTML service to split the work up and run it all in parallel. Here’s how  – (I recommend putting all these kind of thing in the same script since they are all very similar and use the same structure for their functions).
This example is  to demonstrate the principle of how to run things in parallel. With a rate limited service such as this, running more than a couple of threads can be counter productive. For this particular service, you could consider running search (query, start, max) and running it a few times with different parameters. However in the Dealing with rate limited services, you’ll find a way of building upon the section below to sequence chunks of parallel workloads that have a rate limited restriction.

The profiles

The first task is to get all the threads matching the search text in your email. We’ll write the getTheThreads() function later
 

Now we need to split the threads into some number of chunks to be worked on in parallel. We’ll write the getMessages() function later

 

Next, a reduction to bring all the results together. As usual we’ll use the common function reduceTheResults(), that we’ve used in all the examples for that

Finally we’ll log all the results in a spreadsheet. We can use a common logTheResults() function for that too

Now we create a single profile to sequence all of those components

The only change we need to make now is to call this function to set up the run profile

The executors

Now we need to write the couple of new functions mentioned in these profiles that will be called by the htmlservice
This one will search the email for threads that match the search term

and this one will be run in parallel, processing chunks of the total messages

Here’s a snap of the run – We got 1026 seconds of processing over 371 seconds, and more importantly – managed to process something we wouldn’t have been able to inside of a 6 minute limit

Less is sometimes more

Normally throwing more parallel threads can get things done quicker, but in this case – more threads mean more backoffs due to rate limiting – since with GmailApp, the limiting is for a user. In fact, when we reduce the number of threads to two – it still gets the job done in roughly the same elapsed time, even though there are less threads working on it. This is because of the increased backing off necessary. In fact if you can get away with it  (this one was too large), run it one thread – and use this to synchronize tasks that can run one after the other as a way of avoiding execution time quota problems.

Authorization

GmailApp needs authorization, so its worth running a small test first to force an authorization dialog, as well as to test your executor functions. That can easily be done by emulating a couple of execution steps in a simple function like this.