In Borrowing an access token from Apps Scripts I demonstrated the simplest way to get a token to use with a Google API, by borrowing one from Apps Script. However to access APIS that have no equivalent in Apps Script, we have to construct an OAUTH2 flow. That can be a fairly complicated business, but there are a few libraries around to help with that. For these examples, we’ll use my cGoa library, as described in OAuth2 for Apps Script in a few lines of code All the demo code is available on github. Note that I’m using the same code for all these examples, so the final source on github will have evolved in some details from the writeup here.
Using the Service Account
There are a number variations of Oauth2 authorization flows. The simplest is the Service Account. It’s simpler because it doesn’t need any user callback interaction. It still uses an access token, which expires after an hour, and needs credentials to be able to get that token in the first place or to refresh it, but the cGoa library takes care of all that for you. The Service account is best used when the resource required belongs to the project, rather than to an individual, and is generally used by an application that is server based. The application is permitted to access the resource because it has the credentials required to get an access token. In this example, I’ll use the Cloud Vision API, which is a brand new API from Google, still in beta, which can analyze images for location, what they are, and face detection. It doesn’t have an Apps Script equivalent, so we need to get an OAuth2 flow. This API needs to have billing enabled (meaning you may have to pay something to use it), since it uses the Google Cloud Platform.
If you prefer, there is a summary video version of this post here
Creating a project
I could use the dev console project associated with my Apps Script webapp, but since I need to enable billing, I’d like to centralize that to a single cloud project. You can go to the developers console and use one of your existing projects, or create a new one, and enable billing for it.
Enabling the API
In the API section of the developers console, enable the cloud vision API
Getting credentials
In the Borrowing an access token from Apps Scripts example, Apps Script helpfully took care of managing credentials for us, but now we need to create them in the developer console. This example will use a service account.
Create a new service account
Give it a name and download the JSON file
Move the file to somewhere on Drive. I tend to keep all my service account files in the same directory, and rename them to include the name I used for the service account – in this case cloud vision. Goa knows how to read these kind of files, so you don’t even have to bother looking inside. Just get the file Id. We’ll need that later.
Find out what scopes are needed
The API explorer can help with discovering what scopes are needed for this API. Select the method (s) you are interested in using and click the oauth2 button. From this you can see some of the scopes you’ll need to access the API. You’ll see a dialog like this.
Include cGoa
You’ll need the cGoa library to manage the interactions with the Google token infrastructure. It’s here
MZx5DzNPsYjVyZaR67xXJQai_d-phDA33
or if you prefer, you can get the source from github.
Set up one off function to store your credentials.
All you have to provide is the propertiesservice to use, the scope, the file id you downloaded from the dev console, and a name by which to refer to this package in the future. I’ve used the same name as I used in the dev console ‘cloudvision’. Once you’ve run this once, you can delete it – it wont be needed again.
function oneOffSetting() {
// used by all using this script
var propertyStore = PropertiesService.getScriptProperties();
// service account for cloud vision
cGoa.GoaApp.setPackage (propertyStore ,
cGoa.GoaApp.createServiceAccount (DriveApp , {
packageName: 'cloudvision_serviceaccount',
fileId:'0B92ExxxxxxxWXkzNXM',
scopes : cGoa.GoaApp.scopesGoogleExpand (['cloud-platform']),
service:'google_service'
}));
}
The example
This simple example will provide a couple of folders with images in them and send them off to the cloudvision api for annontation of ‘what they are’ in one case, and face detection in another. Here’s my two folders of images – pets and faces, along with the cloud vision results rendered using Html Service.(I wont be doing that in this service account example, but I will in the next example using a webapp Oauth2 authorization flow)
Pets
Faces
The expression detection part doesn’t seem to work very well yet. but I guess it will improve.
Getting the service token
Assuming that you’ve run the once off function, the processing is as simple as this one line. Goa will get a new token, or refresh an existing one as necessary.
/**
* demonstrate using service account
* getting/refreshing service token is pretty much a one liner
*/
function usingServiceAccount () {
logVision (Goth.getToken('cloudvision_serviceaccount'));
}
For simplicity I’ve centralized the getting of tokens to this namespace in this project. I recommend you do the same, as you’ll see when we come to implement the web Oauth2 flow in future examples. Like this, you don’t need to worry about the mechanics of how Goa talks to the OAUTH2 infrastucture.
/**
* namespace to deal with oauth
* @namespace Oauth
*/
var Goth = (function(ns) {
/**
* get a goa or fail
* @param {string} packageName the package name
* @param {PropertiesService} [props=ScriptProperties]
* @return {Goa} a goa
*/
ns.getGoa = function (packageName,props) {
return cGoa.GoaApp.createGoa(
packageName,
props || PropertiesService.getScriptProperties()
).execute();
}
/**
* get a token or fail
* @param {string} packageName the package name
* @param {PropertiesService} [props=ScriptProperties]
* @return {Goa} a goa
*/
ns.getToken = function (packageName,props) {
var goa = ns.getGoa (packageName, props);
if (!goa.hasToken()) {
throw 'no token retrieved';
}
return goa.getToken();
}
return ns;
}) (Goth || {});
Using the Cloudvision API
We’ve seen that this app simply calls the logVision function , passing an access token retrieved by Goa. Here’s the full code. Note that I’m using cache. Since this is a paid for service, best to avoid using it if we’ve already done the same query!
/**
* log out the result of both tests
* @param {string} accessToken the access token
*/
function logVision (accessToken) {
// i have some animal images in this folder
// findout what they are of
var result = Pull.pets (accessToken);
Logger.log ( JSON.stringify(result.map(function(d) {
return {
name:d.file.fileName,
annotation:d.annotation
};
})));
// some faces
var result = Pull.faces (accessToken);
// just summarize
Logger.log ( JSON.stringify(result.map(function(d) {
return {
name:d.file.fileName,
annotation:d.annotation
};
})));
}
function cloudVisionAnnonate (accessToken, folderId , type, maxResults) {
// the API end point
var endPoint = "https://vision.googleapis.com/v1/images:annotate";
// get the images and encode them
var folder = DriveApp.getFolderById(folderId);
if (!folder) {
throw 'cant find image folder ' + folderId;
}
// get all the files in the folder
var iterator = folder.searchFiles("mimeType contains 'image/'");
var files = [];
while (iterator.hasNext()) {
var file = iterator.next();
files.push({
fileName:file.getName(),
b64:Utilities.base64Encode(file.getBlob().getBytes()),
type:file.getMimeType(),
id:file.getId()
});
}
if (!files.length) {
Logger.log ('couldnt find any images in folder ' + folder.getName());
}
// now create the post body
maxResults = maxResults || 1;
type = type || "LABEL_DETECTION"
var body = {
requests: files.map (function (d) {
return {
features: [{
"type":type || "LABEL_DETECTION",
"maxResults":maxResults
}],
image: {
content: d.b64
}
}
})};
// can cost money so use cache
var cache = CacheService.getScriptCache();
var cacheKey = files.map(function(d) {
return d.id;
})
.concat ([type,maxResults])
.join("_")
var cacheData = cache.get(cacheKey);
var result;
if (!cacheData) {
// do the cloud vision request
var response = UrlFetchApp.fetch ( endPoint, {
method: "POST",
payload: JSON.stringify(body),
contentType: "application/json",
headers: {
Authorization:'Bearer ' + accessToken
}
});
// objectify the result
result = JSON.parse(response.getContentText());
cache.put (cacheKey , response.getContentText());
}
else {
result = JSON.parse(cacheData);
}
return files.map (function (d,i) {
return {
file:d,
annotation:result.responses[i]
}
});
}
I’ve centralized which folders to pull the images from for this example into this namespace. As this is developed in future examples, this will be replaced by a filepicker dialog.
var Pull = (function (ns) {
/**
* do the pets image analysis
* @param {string} accessToken the accessToken
* @param {string} folderId the folder id with the images
* @return {object} the pets result
*/
ns.pets = function (accessToken,folderId) {
return cloudVisionAnnonate (
accessToken, folderId || '0B92ExLh4POiZYzZZT3d0aU9VV1U',
'LABEL_DETECTION' , 3
);
};
/**
* do the faces image analysis
* @param {string} accessToken the accessToken
* @return {object} the pets result
*/
ns.faces = function (accessToken,folderId) {
// some people images here
return cloudVisionAnnonate (
accessToken,
folderId || '0B92ExLh4POiZZDNKcVN0QTJJMGs',
'FACE_DETECTION', 1
);
};
return ns;
})({});
I’ll be covering how to render this stuff using html service in a later post, but for now here’s a very small snippet of the data returned by these functions.
[{
"name": "terrier.jpg",
"annotation": {
"labelAnnotations": [{
"mid": "/m/068hy",
"description": "pet",
"score": 0.99145305
}, {
"mid": "/m/0bt9lr",
"description": "dog",
"score": 0.9912262
}, {
"mid": "/m/04rky",
"description": "mammal",
"score": 0.96640766
}]
}
}, {
"name": "cute-dogs.jpg",
"annotation": {
"labelAnnotations": [{
"mid": "/m/068hy",
"description": "pet",
"score": 0.98858243
}, {
"mid": "/m/0bt9lr",
"description": "dog",
"score": 0.98743832
}, {
"mid": "/m/04rky",
"description": "mammal",
"score": 0.96527207
}]
}
}, {
"name": "stuffed.jpg",
"annotation": {
"labelAnnotations": [{
"mid": "/m/0jbk",
"description": "animal",
"score": 0.80946273
}, {
"mid": "/m/0kmg4",
"description": "teddy bear",
"score": 0.71301168
}, {
"mid": "/m/0138tl",
"description": "toy",
"score": 0.60209852
}]
}
}]
[{
"name": "frumpy.jpg",
"annotation": {
"faceAnnotations": [{
"boundingPoly": {
"vertices": [{
"x": 196
}, {
"x": 490
}, {
"x": 490,
"y": 339
}, {
"x": 196,
"y": 339
}]
},
"fdBoundingPoly": {
"vertices": [{
"x": 229,
"y": 88
}, {
"x": 445,
"y": 88
}, {
"x": 445,
"y": 304
}, {
"x": 229,
"y": 304
}]
},
"landmarks": [{
"type": "LEFT_EYE",
"position": {
"x": 284.49878, etc.............
Why not join our forum, follow the blog or follow me on Twitter to ensure you get updates when they are available.