Tracking github pageviews

Apps Script (advanced level) posted on 16th Oct 2017


Github pageviews

You can get things like number of stars and watchers etc from the GitHub REST API repository data, but pageviews are not tracked there. It is possible to get pageviews from another part of the API,  but only the last 14 days. If you want to track pageviews over time, you can try to do it with an Apps Script trigger that records the moving pageview count in a sheet. 

I already have all the components I need to mash this up pretty quickly, and they are all covered elsewhere on this site 
I'll need the cUseful, cGoa and cGitJson libraries.
  • 1EbLSESpiGkI3PYmJqWh3-rmLkYKAtCNPi1L2YCtMgo2Ut8xMThfJ41Ex
  • 1v_l4xN3ICa0lAW315NQEzAHPSoNiFdWHsMEwj2qA5t9cgZ5VWci2Qxv2
  • 1_4RfsIW57fdzWh7T38O9IfGdTRgYbOSyC5PvsOm3a4GU1sxllw8blEUl


Credentials an authentication

Follow the steps for setting up Github authentication in Apps Script as described in Step by Step - Apps Script Oauth2 authentication with Github and Github service for Goa examples

As a reminder, my one off credentials setup looks like this using the credentials from the github console
  cGoa.GoaApp.setPackage (PropertiesService.getScriptProperties() , { 
    clientId : "b........9",
    clientSecret : "634.....ff",
    scopes : [
      'gist',
      'repo'
    ],
    service: 'github',
    packageName: 'git'
  });

And my one off webApp to kick off the authentication process is
function doGet(e) {

  // running as the user running the app
  cGoa.GoaApp.userClone('git', PropertiesService.getScriptProperties() , PropertiesService.getUserProperties());
  var goa = cGoa.GoaApp.createGoa('git',PropertiesService.getUserProperties()).execute(e);
  
  // it's possible that we need consent - this will cause a consent dialog
  if (goa.needsConsent()) {
    return goa.getConsent();
  }
  
  // if we get here its time for your webapp to run and we should have a token, or thrown an error somewhere
  if (!goa.hasToken()) throw 'something went wrong with goa - did you check if consent was needed?';
  
}

Run the oneoff script, publish doGet, run it once, copy the redirect url to the github console and we're good to go. All that can be deleted if you want.

Tracking Sheet

My tracking sheet will look like this, and be updated daily by a triggered script. Each repo will have a row for each day that pageview stats were found.

Open the tracking sheet, or create it

function maintainGitPageViews () {
  
  // get the master sheet
  var masterss = SpreadsheetApp.openById(MASTERID);
  var fiddler = new cUseful.Fiddler(masterss.getSheetByName(MAINTAINGITPVNAME) || masterss.insertSheet(MAINTAINGITPVNAME) );

Get a git handle
var git = new cGitJsonApi
    .cGitJsonApi(SETTINGS)
    .setAccessToken( getAccessToken('git'));

Where SETTINGS for me are. Set up your equivalent GitHub settings here
var SETTINGS = {
  git: {
    committer: {
      "name": "Bruce McPherson",
      "email": "bruce@mcpher.com"
    },
    userAgent: "brucemcpherson"
  }
};


Get all your repos
 
// get all my repos
  var result = git.getMyRepos();
  if (!result.success) {
    throw 'failed to get all the repos ' + JSON.stringify(result);
  }
  var items = result.data;

Get all the pageViews for each repo
 
result.data.map (function (d) {
    var traffic = git.getTrafficViews  (SETTINGS.git.userAgent , d.name );
 
    return  { 
      traffic:traffic.success ? traffic.data : null,
      id:d.id,
      name:d.name,
      fullName:d.full_name
    };
    
  })

 Create fiddler items records for each pageview observation
 
.reduce (function (p,c) {

    if (c.traffic) {
      c.traffic.forEach (function (traffic) {
        traffic.views.forEach (function (observation) {
          p.push ({
            id:c.id,
            name:c.name,
            fullName:c.fullName,
            timestamp:observation.timestamp,
            uniques:observation.uniques,
            count:observation.count
          });
        });
      });
    }
    else {
      Logger.log ("no traffic for " + c.fullName);
    }
    return p;
  },[])

Update the fiddler by replacing or inserting observations
 
// now we have to find and replace or insert
  .forEach (function (item) {
  
    // if this is a blank sheet, we'll need to first create some columns
    if (!fiddler.getNumRows()) {
      fiddler.setData ([item]);
    }
    // find match for this timestamp and replace it
    var matches = fiddler.selectRows ("id", function (value, props) {
      return item.id === value && props.row.timestamp === item.timestamp;
    });
    
    if (matches.length === 1) {
      // replace
      fiddler.getData()[matches[0]] = item;
    }
    else if (!matches.length) {
      // insert
      fiddler.insertRows (undefined , 1 , item);
    }
    else {
      throw 'ambiguous match for ' + JSON.stringify (item);
    }
    
  });

Write out the updated fiddler
  fiddler.dumpValues();
  
}

Scheduling


You can trigger this to run as often as you want, but at least once every two weeks. Even with a few hundred repos, it's pretty quick so every night is probably just fine. Now you can summarize and chart by summing each unique repo.

You want to learn Google Apps Script?

Learning Apps Script, (and transitioning from VBA) are covered comprehensively in my my book, Going Gas - from VBA to Apps script, All formats are available from O'ReillyAmazon and all good bookshops. You can also read a preview on O'Reilly

If you prefer Video style learning I also have two courses available. also published by O'Reilly.
Google Apps Script for Developers and Google Apps Script for Beginners.

Comments