Apps Script (advanced level) | posted on 16th Oct 2017 |
Github pageviews
You can get things like number of stars and watchers etc from the GitHub REST API repository data, but pageviews are not tracked there. It is possible to get pageviews from another part of the API, but only the last 14 days. If you want to track pageviews over time, you can try to do it with an Apps Script trigger that records the moving pageview count in a sheet.
I already have all the components I need to mash this up pretty quickly, and they are all covered elsewhere on this site
- A functional approach to fiddling with sheet data will give me a Fiddler to play around with sheet data easily
- Github service for Goa examples to authenticate with github
- Drive SDK and GitHub a library to access gitHub
I’ll need the cUseful, cGoa and cGitJson libraries.
- 1EbLSESpiGkI3PYmJqWh3-rmLkYKAtCNPi1L2YCtMgo2Ut8xMThfJ41Ex
- 1v_l4xN3ICa0lAW315NQEzAHPSoNiFdWHsMEwj2qA5t9cgZ5VWci2Qxv2
- 1_4RfsIW57fdzWh7T38O9IfGdTRgYbOSyC5PvsOm3a4GU1sxllw8blEUl
Credentials an authentication
Follow the steps for setting up Github authentication in Apps Script as described in Step by Step – Apps Script Oauth2 authentication with Github and Github service for Goa examples.
As a reminder, my one off credentials setup looks like this using the credentials from the github console.
cGoa.GoaApp.setPackage (PropertiesService.getScriptProperties() , { clientId : "b........9", clientSecret : "634.....ff", scopes : [ 'gist', 'repo' ], service: 'github', packageName: 'git' });
And my one off webApp to kick off the authentication process is
function doGet(e) { // running as the user running the app cGoa.GoaApp.userClone('git', PropertiesService.getScriptProperties() , PropertiesService.getUserProperties()); var goa = cGoa.GoaApp.createGoa('git',PropertiesService.getUserProperties()).execute(e); // it's possible that we need consent - this will cause a consent dialog if (goa.needsConsent()) { return goa.getConsent(); } // if we get here its time for your webapp to run and we should have a token, or thrown an error somewhere if (!goa.hasToken()) throw 'something went wrong with goa - did you check if consent was needed?'; }
Run the oneoff script, publish doGet, run it once, copy the redirect url to the github console and we’re good to go. All that can be deleted if you want.
Tracking Sheet
My tracking sheet will look like this, and be updated daily by a triggered script. Each repo will have a row for each day that pageview stats were found.
Open the tracking sheet, or create it
function maintainGitPageViews () { // get the master sheet var masterss = SpreadsheetApp.openById(MASTERID); var fiddler = new cUseful.Fiddler(masterss.getSheetByName(MAINTAINGITPVNAME) || masterss.insertSheet(MAINTAINGITPVNAME) );
Get a git handle
var git = new cGitJsonApi .cGitJsonApi(SETTINGS) .setAccessToken( getAccessToken('git'));
Where SETTINGS for me are. Set up your equivalent GitHub settings here
var SETTINGS = { git: { committer: { "name": "Bruce McPherson", "email": "bruce@mcpher.com" }, userAgent: "brucemcpherson" } };
Get all your repos
// get all my repos var result = git.getMyRepos(); if (!result.success) { throw 'failed to get all the repos ' + JSON.stringify(result); } var items = result.data;
Get all the pageViews for each repo
result.data.map (function (d) { var traffic = git.getTrafficViews (SETTINGS.git.userAgent , d.name ); return { traffic:traffic.success ? traffic.data : null, id:d.id, name:d.name, fullName:d.full_name }; })
Create fiddler items records for each pageview observation
.reduce (function (p,c) { if (c.traffic) { c.traffic.forEach (function (traffic) { traffic.views.forEach (function (observation) { p.push ({ id:c.id, name:c.name, fullName:c.fullName, timestamp:observation.timestamp, uniques:observation.uniques, count:observation.count }); }); }); } else { Logger.log ("no traffic for " + c.fullName); } return p; },[])
Update the fiddler by replacing or inserting observations
// now we have to find and replace or insert .forEach (function (item) { // if this is a blank sheet, we'll need to first create some columns if (!fiddler.getNumRows()) { fiddler.setData ([item]); } // find match for this timestamp and replace it var matches = fiddler.selectRows ("id", function (value, props) { return item.id === value && props.row.timestamp === item.timestamp; }); if (matches.length === 1) { // replace fiddler.getData()[matches[0]] = item; } else if (!matches.length) { // insert fiddler.insertRows (undefined , 1 , item); } else { throw 'ambiguous match for ' + JSON.stringify (item); } });
Write out the updated fiddler
fiddler.dumpValues(); }
Scheduling
You can trigger this to run as often as you want, but at least once every two weeks. Even with a few hundred repos, it’s pretty quick so every night is probably just fine. Now you can summarize and chart by summing each unique repo.
For more on drive SDK see Using Drive SDK
For help and more information join our community, follow the blog or follow me on Twitter