Tracking github pageviews

Apps Script (advanced level)

posted on 16th Oct 2017

Github pageviews

You can get things like number of stars and watchers etc from the GitHub REST API repository data, but pageviews are not tracked there. It is possible to get pageviews from another part of the API,  but only the last 14 days. If you want to track pageviews over time, you can try to do it with an Apps Script trigger that records the moving pageview count in a sheet. 
I already have all the components I need to mash this up pretty quickly, and they are all covered elsewhere on this site 

I’ll need the cUseful, cGoa and cGitJson libraries.

Credentials an authentication

Follow the steps for setting up Github authentication in Apps Script as described in Step by Step – Apps Script Oauth2 authentication with Github and Github service for Goa examples. 

As a reminder, my one off credentials setup looks like this using the credentials from the github console.

cGoa.GoaApp.setPackage (PropertiesService.getScriptProperties() , { 
    clientId : "b........9",
    clientSecret : "634.....ff",
    scopes : [
      'gist',
      'repo'
    ],
    service: 'github',
    packageName: 'git'
  });

And my one off webApp to kick off the authentication process is

function doGet(e) {

  // running as the user running the app
  cGoa.GoaApp.userClone('git', PropertiesService.getScriptProperties() , PropertiesService.getUserProperties());
  var goa = cGoa.GoaApp.createGoa('git',PropertiesService.getUserProperties()).execute(e);
  
  // it's possible that we need consent - this will cause a consent dialog
  if (goa.needsConsent()) {
    return goa.getConsent();
  }
  
  // if we get here its time for your webapp to run and we should have a token, or thrown an error somewhere
  if (!goa.hasToken()) throw 'something went wrong with goa - did you check if consent was needed?';
  
}

Run the oneoff script, publish doGet, run it once, copy the redirect url to the github console and we’re good to go. All that can be deleted if you want.

Tracking Sheet

My tracking sheet will look like this, and be updated daily by a triggered script. Each repo will have a row for each day that pageview stats were found.

Open the tracking sheet, or create it

function maintainGitPageViews () {
  
  // get the master sheet
  var masterss = SpreadsheetApp.openById(MASTERID);
  var fiddler = new cUseful.Fiddler(masterss.getSheetByName(MAINTAINGITPVNAME) || masterss.insertSheet(MAINTAINGITPVNAME) );

Get a git handle

var git = new cGitJsonApi
    .cGitJsonApi(SETTINGS)
    .setAccessToken( getAccessToken('git'));

Where SETTINGS for me are. Set up your equivalent GitHub settings here

var SETTINGS = {
  git: {
    committer: {
      "name": "Bruce McPherson",
      "email": "bruce@mcpher.com"
    },
    userAgent: "brucemcpherson"
  }
};

Get all your repos

// get all my repos
  var result = git.getMyRepos();
  if (!result.success) {
    throw 'failed to get all the repos ' + JSON.stringify(result);
  }
  var items = result.data;

Get all the pageViews for each repo

result.data.map (function (d) {
    var traffic = git.getTrafficViews  (SETTINGS.git.userAgent , d.name );
 
    return  { 
      traffic:traffic.success ? traffic.data : null,
      id:d.id,
      name:d.name,
      fullName:d.full_name
    };
    
  })

Create fiddler items records for each pageview observation

.reduce (function (p,c) {

    if (c.traffic) {
      c.traffic.forEach (function (traffic) {
        traffic.views.forEach (function (observation) {
          p.push ({
            id:c.id,
            name:c.name,
            fullName:c.fullName,
            timestamp:observation.timestamp,
            uniques:observation.uniques,
            count:observation.count
          });
        });
      });
    }
    else {
      Logger.log ("no traffic for " + c.fullName);
    }
    return p;
  },[])

Update the fiddler by replacing or inserting observations

// now we have to find and replace or insert
  .forEach (function (item) {
  
    // if this is a blank sheet, we'll need to first create some columns
    if (!fiddler.getNumRows()) {
      fiddler.setData ([item]);
    }
    // find match for this timestamp and replace it
    var matches = fiddler.selectRows ("id", function (value, props) {
      return item.id === value && props.row.timestamp === item.timestamp;
    });
    
    if (matches.length === 1) {
      // replace
      fiddler.getData()[matches[0]] = item;
    }
    else if (!matches.length) {
      // insert
      fiddler.insertRows (undefined , 1 , item);
    }
    else {
      throw 'ambiguous match for ' + JSON.stringify (item);
    }
    
  });

Write out the updated fiddler

fiddler.dumpValues();
  
}

Scheduling

You can trigger this to run as often as you want, but at least once every two weeks. Even with a few hundred repos, it’s pretty quick so every night is probably just fine. Now you can summarize and chart by summing each unique repo.

For more on drive SDK see Using Drive SDK

For help and more information join our community,  follow the blog or follow me on Twitter