Twt is a SuperFetch plugin to easily access to the Twitter v2 API.  SuperFetch is a proxy for UrlFetchApp with additional features – see SuperFetch – a proxy enhancement to Apps Script UrlFetch for how it works and what it does.

This is another in my series on SuperFetch plugins.

Page Content hide
1 Motivation

Motivation

This is a follow on from SuperFetch plugin – Twitter client for Apps Script – Search and Get but will cover the counts method. This counts the number of occurences of a given query in a period.

Script and Twitter Preparation

You’ll need the bmSuperFetch and cGoa libraries and if you want to run some tests, the bmUnitTest library – details at the end of the article.

Unlike SuperFetch plugin – Twitter client for Apps Script – Search and Get which can use either Twitter Oauth2 User flow or the less secure OAuth2 and Twitter API – App only flow for Apps Script (giving subtly different results), the counts method only accepts queries with the App flow. This is because it accesses purely public data without any specific user filtering.

Instantiation

There’s a few things you need to initialize before you can start running your script

Goa

First you’ll need a Goa to provide a token service – see OAuth2 and Twitter API – App only flow for Apps Script for how to set up. You can have multiple Goas in the same app, so you you can have both User and App only flow enabled.

  // we can use this for user flow token service for normal searching etc.
// this one won't work with .counts()
const goa = makeTwitterGoa()

// for counts we'll use a different service
// app only oauth
const goaApp = makeTwitterGoaAppOnly()


// here's how to make an app only token service
// this is the one you'll need for .counts()
// SETTINGS
/*
{
twitterAppOnly: {
propertyService: PropertiesService.getScriptProperties(),
name: 'twitterAppOnly'
}}
*/
const makeTwitterGoaAppOnly = () => {

const { twitterAppOnly } = SETTINGS
return cGoa.make(
twitterAppOnly.name,
twitterAppOnly.propertyService,
)
}




get an app only Goa

SuperFetch

You need to include the plugins you need, and an instaniated SuperFetch instance. If you don’t need caching, then just omit the cacheService property (but I highly recommend caching with this API). If you are using both types of OAuth2  tokenService make a superFetch instance for each.

  // import required modules
const { Plugins, SuperFetch } = bmSuperFetch
const { Twt } = Plugins

// for .counts() we need an app only Oauth2 token service service
const superFetchApp = new SuperFetch({
fetcherApp: UrlFetchApp,
tokenService: goaApp.getToken,
cacheService: CacheService.getScriptCache(),
})
superfetch instance

Twt instance

​This will be the handle for accessing Twitter.

  
// clone and user the app only superfetch instance
const twtApp = new Twt({
superFetch: superFetchApp
})

twt instance

There are various other standard SuperFetch Plugin parameters that I’ll deal with later, but the ones of most interest are:

superFetch (required)

The instance you created earlier

noCache property (optional)

In this instance, I’m turning caching off for now.

showUrl property (optional)

Normally this would be false, but if you want to see you how your calls to the twt client are translated to native Rest calls, you can add this and set it to true.

Calling structure

The twt plugin uses closures heavily. A closure is a function which carries the references to its state (the lexical environment) with it.  This encapsulation is very handy for creating reusable short cuts to standard queries, as we’ll cover later.

Counting

Here’s how you’d get the recent count of tweets that match a particular query . The query is constructed using the same syntax as you’d use in the normal web  client. The full details are here.

  // get lastest tweets about apps script
// we need app only auth for counting
const data = twtApp.tweets.counts("Apps Script").throw().data
simple counts
Responses

All responses have exactly the same format. You’ll get back a standard SuperFetch Response, which looks like this

/**
* This is the response from the apply proxied function
* @typedef PackResponse
* @property {boolean} cached whether item was retrieved from cache
* @property {object||null} data parsed data
* @property {number} age age in ms of the cached data
* @property {Blob|| null} blob the recreated blob if there was one
* @property {boolean} parsed whether the data was parsed
* @property {HttpResponse} response fetch response
* @property {function} throw a function to throw on error
* @property {Error || string || null} the error if there was one
* @property {number} responseCode the http response code
* @property {string} url the url that provoked this response
* @property {string} pageToken optional restart point passed back in the page paramter
*/
console.log(Object.keys(bunch))
[ 'response',
'data',
'error',
'parsed',
'blob',
'age',
'cached',
'responseCode',
'url',
'throw',
'pageToken' ]
Pack response
Twitter API data

The response from the Twitter API will be in the data property of the SuperFetch response.

console.log(Object.keys(data))	
[ 'items']
results from twitter API

Actually the twitter response will have been massaged a bit so that every API call has exactly the same format. There is 1 property of interest

  • items – an array of standard data from the Twitter API
Data items

The items are an array of basic data matching the search criteria

 
console.log(data.items)

/* looks like this by hour
[{ end: '2022-06-23T19:00:00.000Z',
start: '2022-06-23T18:00:00.000Z',
tweet_count: 5 }, ... ]
*/
typical basic tweet data
Closures

You can specify the search query in a number of ways, including creating a query closure.

// simple query
twtApp.tweets.counts("Apps Script").throw().data


// is equivalent to
const query = twtApp.tweets.query({query: "Apps Script"})
query.counts().throw().data
alternative query structures
Additional parameters

You can enhance the results by adding parameters – these are documented in the API reference detail.

The most interesting one is granuarity – (minute, hour or day) – hour is the default

 console.log(twtApp.tweets.counts("Apps Script", { granularity: 'day' }).throw().data.items)

// you can also create a reusable closure
const dayQuery = twtApp.tweets.query({fields: { granularity: 'day' }})
console.log(dayQuery.counts("Apps Script").data.items)
fields
Using .throw

If SuperFetch encounters an error it’ll be in the error property of the response

  • you can handle this yourself by checking the response error property.
  • The plugin can automatically throw an error on your behalf if you can add the .throw() method to any request. All SuperFetch plugins have the same error handling approach. Think of it as a built in try/catch.
  // error handling
// data will be in result.data
const hResult = tweets.counts (queryString)
if (hResult.error) {
// handle it yourself
console.log(hResult.error)
}
// throw an error if there is one
// data will still be in result.data
const iResult = tweets.counts (queryString).throw()
throw() error handling

Caching

Caching is built into SuperFetch  (for details see SuperFetch – a proxy enhancement to Apps Script UrlFetch) so you get caching out of the box. For more details on caching with this api see SuperFetch plugin – Twitter client for Apps Script – Search and Get

Recent and All
There are 2 levels of searching counting depending on your account permissions
  • recent – tweets from the last 7 days
  • all – all the twitter archive

The plugin supports both, but – from the API docs regarding the ‘all’ endpoint.

“This endpoint is only available to those users who have been approved for Academic Research access.

The full-archive Tweet counts endpoint returns the count of Tweets that match your query from the complete history of public Tweets; since the first Tweet was created March 26, 2006.”

Recent

This is the default, and you never need to specify it. For completeness it looks like this

twtApp.tweets.recent.counts("Apps Script").throw().data.items
.recent is the default
All

Only Academic researchers have access to this, and most of us will get this error – which strangely enough is not an accurate description of the error anyway.

 console.log(twt.tweets.all.counts("Google Apps Script").throw().data.items) // error
/*
Error: {
"client_id":"xxxxx",
"detail":"When authenticating requests to the Twitter API v2 endpoints, you must use keys and tokens from a Twitter developer App that is attached to a Project. You can create a project via the developer portal.",
"registration_url":"https://developer.twitter.com/en/docs/projects/overview",
"title":"Client Forbidden",
"required_enrollment":"Standard Basic","
reason":"client-not-enrolled",
"type":"https://api.twitter.com/2/problems/client-forbidden"
}
*/
all returns an error for non researcher permission accounts

Unit testing

I’ll use Simple but powerful Apps Script Unit Test library to demonstrate calls and responses. It should be straightforward to see how this works and the responsese to expect from calls. These tests demonstrate in detail each of the topics mentioned in this article, and a few others, and could serve as a useful crib sheet for the plugin

  unit.section(() => {
// some of the api endpoint require credential OAuth
const goa = makeTwitterGoaAppOnly()
// we need a different tokenService
const superFetch = new SuperFetch({
fetcherApp: UrlFetchApp,
tokenService: goa.getToken,
cacheService: CacheService.getUserCache(),
missingPropertyIsFatal: true,
showUrl: true
})
const twt = new Twt({
superFetch,
noCache: true
})
const tn = twt.ref({ noCache: true }).tweets
const tc = twt.ref({ noCache: false }).tweets
const query = "Apps Script"

const cQuery = tc.query({ query })
const nQuery = tn.query({ query })

const { actual: counts } = unit.not(null, nQuery.counts().throw(), {
description: 'counts per hour for query'
})
unit.is('number', typeof counts.data.items[0].tweet_count, {
description: 'count is a number'
})
const total = (items) => items.reduce((p, c) => p c.tweet_count, 0)
const { actual: seed } = unit.not(null, cQuery.counts().throw(), {
description: 'seeding cache'
})
unit.is(false, seed.cached, {
description: 'seed wasnt cached'
})
unit.is(total(counts.data.items), total(seed.data.items), {
description: 'same tweet total'
})
unit.is(true, cQuery.counts().throw().cached, {
description: 'seed is cached'
})
unit.is(seed.data, cQuery.counts().throw().data, {
description: 'cache matches seed'
})
const {actual: days} = unit.not (null ,tn.counts(query, { granularity: 'day' }) ,{
description: 'daily counts'
})
unit.is (8 ,tn.counts(query, { granularity: 'day' }).data.items.length ,{
description: '8 day inclusive count'
})
}, {
description: "counts",
skip: skipTest.tweetCounts
})
tests

Links

bmSuperFetch: 1B2scq2fYEcfoGyt9aXxUdoUPuLy-qbUC2_8lboUEdnNlzpGGWldoVYg2

IDE

GitHub

bmUnitTester: 1zOlHMOpO89vqLPe5XpC-wzA9r5yaBkWt_qFjKqFNsIZtNJ-iUjBYDt-x

IDE

GitHub

cGoa library 1v_l4xN3ICa0lAW315NQEzAHPSoNiFdWHsMEwj2qA5t9cgZ5VWci2Qxv2

IDE

Github

Related

Twitter API docs https://developer.twitter.com/en/docs/twitter-api

Twitter Developer profile https://developer.twitter.com/en/portal

file conversion

Convert any file with Apps Script

The Drive API offers a whole range of conversions between mimeTypes, but it's a little fiddly to figure out exactly ...
Superfetch plugin

Caching, property stores and pre-caching

I've written many times about various Apps Script caching techniques such as how to deal with size limits and use ...
document AI add-on

State management across CardService, HtmlService and Server side Add-ons

Motivation I've been working on a CardService Add-on lately which also uses HtmlService, and also runs quite a few things ...
Secret Manager

SuperFetch Plugin: Cloud Manager Secrets and Apps Script

Smg is a SuperFetch plugin to access the Google Cloud Secrets API. SuperFetch is a proxy for UrlFetchApp with additional ...
Superfetch plugin

SuperFetch caching: How does it work?

SuperFetch is a proxy for UrlFetchApp with additional features such as built-in caching – see SuperFetch – a proxy enhancement ...
superfetch tank drive

SuperFetch plugins: Tank events and appending

Tank and Drv are SuperFetch plugins to emulate streaming and use the Drive REST API with Apps Script. SuperFetch is ...
superfetch tank drive

SuperFetch Plugins: Apps Script streaming with Tank and Drive

Tank and Drv are SuperFetch plugins to emulate streaming and use the Drive REST API with Apps Script. SuperFetch is ...
superfetch tank apps script streaming

SuperFetch Tank Plugin: Streaming for Apps Script

Tank is a SuperFetch plugin to emulate streaming with Apps Script. SuperFetch is a proxy for UrlFetchApp with additional features ...
superfetch drive plugin logo

SuperFetch plugin – Google Drive client for Apps Script – Part 1

Drv is a SuperFetch plugin to access the Google Drive API. SuperFetch is a proxy for UrlFetchApp with additional features ...
Superfetch plugin twitter

SuperFetch – Twitter plugin for Apps Script – Get Follows, Mutes and blocks

Twt is a SuperFetch plugin to easily access to the Twitter v2 API. SuperFetch is a proxy for UrlFetchApp with ...
Superfetch plugin twitter

SuperFetch plugin – Twitter client for Apps Script – Counts

Twt is a SuperFetch plugin to easily access to the Twitter v2 API. SuperFetch is a proxy for UrlFetchApp with ...
goa twitter oauth2 apps script

OAuth2 and Twitter API – App only flow for Apps Script

I covered how to handle the somewhat more complex OAUTH2 authorization flow for the Twitter v2 API (OAuth 2.0 Authorization ...
Superfetch plugin twitter

SuperFetch plugin – Twitter client for Apps Script – Search and Get

Twt is a SuperFetch plugin to easily access to the Twitter v2 API. SuperFetch is a proxy for UrlFetchApp with ...
Goa Oauth2 for Apps Script

Apps Script Oauth2 library Goa: tips, tricks and hacks

Motivation Goa is a library to support OAuth2 for Apps Script connecting to a variety of services, using a variety ...
goa twitter oauth2 apps script

Apps Script Oauth2 – a Goa Library refresher

It's been a few years since I first created the Goa library. Initially it was mainly to provide OAuth2 authorization ...
SuperFetch

SuperFetch plugin – Firebase client for Apps Script

Frb is a SuperFetch plugin to easily access a Firebase Real time database. SuperFetch is a proxy for UrlFetchApp with ...
SuperFetch

SuperFetch plugin – iam – how to authenticate to Cloud Run from Apps Script

SuperFetch is a proxy for UrlFetchApp with additional features - see SuperFetch - a proxy enhancement to Apps Script UrlFetch ...
SuperFetch

SuperFetch – a proxy enhancement to Apps Script UrlFetch

I've written a few articles about JavaScript proxying on here, and I'm a big fan. I also use a lot ...
SuperFetch

Apps script caching with compression and enhanced size limitations

Motivation Caching is a great way to improve performance, avoid rate limit problems and even save money if you are ...

Simple but powerful Apps Script Unit Test library

Why unit testing? There are many test packages for Node (my favorite is ava) and there are also a few ...