In Node focus explorer without excel I showed a d3.js app that could navigate a google site using tags and nodes. This was actually pretty generalized, so it was a minor modification to add blogger post navigation capability also.
Again we’ll use Google Apps Script and adapt Analyzing site content with GAS  to take data from blogger posts rather than Google Site content (actually .. as well as .. but more of that later).

Getting the blogger API key

Unlike Google Sites, you do need to get an API key to access the blogger API. You can apply for one here. A day or two later, someone sends you a mail with details of your key.

The API data

This is a straightforward REST api, but it does have a limit of 20 posts, so you have to call it repeatedly until you have them all as follows
 var cb = new mcpher.cBrowser();
    var p, items=[];
    
    while (!p || p.nextPageToken) {
      p = JSON.parse(cb.get(initBlogger(eArgs,source) + ( p ? '&pageToken='+p.nextPageToken : '')));
      if (p && p.items) items = items.concat (p.items);
      if (!p) break;
    }

Storing your API key securely

 

One of the challenges about publishing code where there is private information is how to hide it. I use Database abstraction and Google Apps Script and store stuff like this on my private MongoDb store, but you may find it easier to use something l like your property store. Here’s how I do it.
// my private store
  var myKey = myStuff.getMyStuff(api, myStuff.myStuffDb());
  if (myKey.status.code == "good") {
    return myKey.result;
  }
  else {
    throw ("unable to find your key for " + api);
    return null;
  }
}

Creating the dataset

Retrieving the data takes the same structure as for Google Sites, except its a little simpler. There is no need for recursion since the blog posts are flat. In fact – you can see from the blogger and sites version below there is a very minor difference to get the data from different properties, and to recurse or not.

function blogChildren(tags,parent,pages,a) {

var resultsArray = a || [];
for (var i =0;i<pages.length;i++) {
// process this page
var page = pages[i];
resultsArray.push( {
parent: parent,
name: page.title,
key: page.id,
url: page.url,
title: page.title,
tags: addCounts(cloneObject(tags),getTextFromHtml(page.content))
} );
// and its children
}
return resultsArray;
}


function logChildren(tags,parent,pages,a) {

  var resultsArray = a || [];
  for (var i =0;i<pages.length;i++) {
     // process this page
     var page = pages[i];
     resultsArray.push( { 
         parent: parent.getName (),
         name: page.getName (), 
         key: page.getName () + "_" + resultsArray.length, 
         url: page.getUrl(), 
         title: page.getTitle(),
         tags: addCounts(cloneObject(tags),page.getTextContent())
     } );
     // and its children
    logChildren (tags,pages[i], pages[i].getChildren(), resultsArray);
  }
  return resultsArray;
}

The only other wrinkle is that the blog content comes as html, so it needs to be rendered as plain text to avoid the html markup being considered as linkable data. I found this neat solution for that on stackOverflow by Cory Goldfeder who continually comes up with great stuff like this.


Function getTextFromHtml(html) {
  return getTextFromNode(Xml.parse(html, true).getElement());
}

function getTextFromNode(x) {
  switch(x.toString()) {
    case 'XmlText': return x.toXmlString();
    case 'XmlElement': return x.getNodes().map(getTextFromNode).join('');
    default: return '';
  }
}

Scheduling data updates

Now I have a Google site version and a blogger version. Scheduling GAS showed how to add a nightly update to write the data to Google Drive. I’ll just update that to write the blogger version to Google Cloud store (for more on that see Cloud Storage and Apps Script) and here is the result for the blogger. You can navigate in the same way as in the Google site version, except this time you get to both. You can write it to Drive if you prefer by changing the parameter from gcs to drive, but then you’ll need to find somewhere to host it. Using cloud storage I can host it in place from where I write it.
 
tagoutput :'gcs',  // change this to drive if you want to write to drive instead
 
The navigator  takes an argument that is the name of the input data (the default is the data for this site). So you can use it as is, once you have created a data set as described here and in the Google site version.
 
Finally, I’ve built in a proxy to allow the data to come from some other place if you don’t have a site to host your data on. For details see Apps Script as a proxy. If you don’t want to use a proxy, you’ll need to change a couple of lines to get the data directly instead of via a proxy.
 
It is embedded below or you can try it out here.

Combining data sets

This is where it gets real interesting – now you can combine multiple sites or blogs into a single navigation app. I have scheduled a nightly trigger that creates a dataset combing the sites and blogger data, giving us this (it’s a bit much but once you start focusing in it can be useful). It will take a while to settle down as there’s a huge amount of content between the two sources. You can run it here
 

The code

Here is the Google Apps Script code to generate the data

Here is the web app code


<!DOCTYPE html>
<html lang="en">
 <head>
  <meta charset="utf-8" />
  <title>d3.js from Excel - ramblings.mcpher.com</title>
   <base href="https://storage.googleapis.com/xliberation.com/">
  <!-- Always force latest IE rendering engine (even in intranet) & Chrome Frame
  Remove this if you use the .htaccess -->
  <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" />
 <script type="text/javascript" src="https://d3js.org/d3.v3.min.js"></script>
  <script type="text/javascript" src="https://www.google.com/jsapi"></script>
  <script type="text/javascript">
   google.load("jquery", "1");
   google.setOnLoadCallback(function() {
        initialize().then (
            function (control) {
                doTheTreeViz(control);
            }
        );
   });
  </script>
 <style>

   body {
    position: relative;
    font-family: "Helvetica Neue", Helvetica, sans-serif;
    margin: 1em auto 4em auto;
    width: 960px;
   }

   h1 {
    font-size: 64px;
    font-weight: 300;
    letter-spacing: -2px;
    margin: .3em 0 .1em 0;
   }

   h2 {
    margin-top: 2em;
   }

   h1, h2 {
    text-rendering: optimizeLegibility;
   }

   h2 a {
    color: #ccc;
    left: -20px;
    position: absolute;
    width: 740px;
   }

   footer {
    font-size: small;
    margin-top: 8em;
   }

   header aside {
    margin-top: 20px;
   }

   header aside, footer aside {
    color: #636363;
    text-align: right;
   }

   aside {
 float:right;
 display:inline;
 font-size: small;
   }
   .attribution {
    font-size: small;
    margin-bottom: 2em;
   }

   body > p, li > p {
    line-height: 1.5em;
   }

   body > p {
    width: 720px;
   }

   body > blockquote {
    width: 640px;
   }

   li {
    width: 680px;
   }

   pre, code, textarea {
    font-family: "Menlo", monospace;
   }

   code {
    line-height: 1em;
   }

   textarea {
    font-size: 100%;
   }

   body > pre {
    border-left: solid 2px #ccc;
    padding-left: 18px;
    margin: 2em 0 2em -20px;
   }

   .html .value, .javascript .string, .javascript .regexp {
    color: #756bb1;
   }

   .html .tag, .css .tag, .javascript .keyword {
    color: #3182bd;
   }

   .comment {
    color: #636363;
   }

   .html .doctype, .javascript .number {
    color: #31a354;
   }

   .html .attribute, .css .attribute, .javascript .class, .javascript .special {
    color: #e6550d;
   }

   svg {
    font: 10px sans-serif;
   }

   .axis path, .axis line {
    fill: none;
    stroke: #000;
    shape-rendering: crispEdges;
   }

   sup, sub {
    line-height: 0;
   }

   q:before, blockquote:before {
    content: "?";
   }

   q:after, blockquote:after {
    content: "?";
   }

   blockquote:before {
    position: absolute;
    left: 2em;
   }

   blockquote:after {
    position: absolute;
   }

  </style>
  <style>
   #chart {
    height: 600px;
   }
   .node rect {
    cursor: move;
    fill-opacity: .9;
    shape-rendering: crispEdges;
   }
   .node text {
    pointer-events: none;
    text-shadow: 0 1px 0 #fff;
   }
   .link {
    fill: none;
    stroke: #000;
    stroke-opacity: .2;
   }
   .link:hover {
    stroke-opacity: .5;
   }

   circle.node-dot {
    fill: DarkSlateGray;
    stroke: SlateGray;
    stroke-width: 1px;
   }


   line.link {
    fill: none;
    stroke: SlateGray;
    stroke-width: 1.5px;
   }

   marker#defaultMarker {
    fill: SlateGray;
   }

   line.link.defaultMarker {
    stroke: SlateGray;
   }

   circle {
    fill: #ccc;
    stroke: #333;
    stroke-width: 1.5px;
   }

   text {
    pointer-events: none;
   }

   text.shadow {
    stroke: #fff;
    stroke-width: 3px;
    stroke-opacity: .8;
   }

  </style> <script>
  function doTheTreeViz(control) {

    var svg = control.svg;

    var force = control.force;
    force.nodes(control.nodes)
        .links(control.links)
        .start();

    // Update the links
    var link = svg.selectAll("line.link")
        .data(control.links, function(d) { return d.unique; } );
 
   // Enter any new links
    link.enter().insert("svg:line", ".node")
        .attr("class", "link")
        .attr("x1", function(d) { return d.source.x; })
        .attr("y1", function(d) { return d.source.y; })
        .attr("x2", function(d) { return d.target.x; })
        .attr("y2", function(d) { return d.target.y; })
      .append("svg:title")
        .text(function(d) { return d.source[control.options.nodeLabel] + ":" + d.target[control.options.nodeLabel] ; });
    
    // Exit any old links.
    link.exit().remove();


  // Update the nodes
    var node = svg.selectAll("g.node")
        .data(control.nodes, function(d) { return d.unique; });

    node.select("circle")
        .style("fill", function(d) {
            return getColor(d);
        })
        .attr("r", function(d) {
            return getRadius(d);
        })

  // Enter any new nodes.
    var nodeEnter = node.enter()
      .append("svg:g")
        .attr("class", "node")
        .attr("transform", function(d) { return "translate(" + d.x + "," + d.y + ")"; })
        .on("dblclick", function(d){
            control.nodeClickInProgress=false;
            if (d.url)window.open(d.url);
        })
        .on("click", function(d){
            // this is a hack so that click doesnt fire on the1st click of a dblclick
            if (!control.nodeClickInProgress ) {
                control.nodeClickInProgress = true;
                setTimeout(function(){
                    if (control.nodeClickInProgress) { 
                        control.nodeClickInProgress = false;
                        if (control.options.nodeFocus) {
                            d.isCurrentlyFocused = !d.isCurrentlyFocused;
                            doTheTreeViz(makeFilteredData(control));
                        }
                    }
                },control.clickHack); 
            }
        })
        .call(force.drag);

    nodeEnter
      .append("svg:circle")
        .attr("r", function(d) {
            return getRadius(d);
        })
        .style("fill", function(d) {
            return getColor(d);
        })
      .append("svg:title")
        .text(function(d) { return d[control.options.nodeLabel]; });

   if (control.options.nodeLabel) {       
       // text is done once for shadow as well as for text
        nodeEnter.append("svg:text")
            .attr("x", control.options.labelOffset)
            .attr("dy", ".31em")
            .attr("class", "shadow")
            .style("font-size",control.options.labelFontSize + "px")
            .text(function(d) {
                return d.shortName ? d.shortName : d.name;
            });
        nodeEnter.append("svg:text")
            .attr("x", control.options.labelOffset)
            .attr("dy", ".35em")
            .attr("class", "text")
            .style("font-size",control.options.labelFontSize + "px")
            .text(function(d) {
                return d.shortName ? d.shortName : d.name;
            });
    }

    // Exit any old nodes.
    node.exit().remove();
    control.link = svg.selectAll("line.link");
    control.node = svg.selectAll("g.node");
    force.on("tick", tick);



    if (control.options.linkName) {
        link.append("title")
            .text(function(d) {
                return d[control.options.linkName];
        });
    }


    function tick() {
        link.attr("x1", function(d) { return d.source.x; })
            .attr("y1", function(d) { return d.source.y; })
            .attr("x2", function(d) { return d.target.x; })
            .attr("y2", function(d) { return d.target.y; });
        node.attr("transform", function(d) {
            return "translate(" + d.x + "," + d.y + ")";
        });

    }
 
    function getRadius(d) {
     var r = control.options.radius * (control.options.nodeResize ? Math.sqrt(d[control.options.nodeResize]) / Math.PI : 1);
     return control.options.nodeFocus && d.isCurrentlyFocused ? control.options.nodeFocusRadius  : r;
    }
    function getColor(d) {
        return control.options.nodeFocus && d.isCurrentlyFocused ? control.options.nodeFocusColor  : control.color(d.group) ;
    }

   }

function makeFilteredData(control,selectedNode){
    // we'll keep only the data where filterned nodes are the source or target
    var newNodes = [];
    var newLinks = [];

    for (var i = 0; i < control.data.links.length ; i++) {
        var link = control.data.links&#91;i&#93;;
        if (link.target.isCurrentlyFocused || link.source.isCurrentlyFocused) {
            newLinks.push(link);
            addNodeIfNotThere(link.source,newNodes);
            addNodeIfNotThere(link.target,newNodes);
        }
    }
    // if none are selected reinstate the whole dataset
    if (newNodes.length > 0) {
        control.links = newLinks;
        control.nodes = newNodes;
    }
    else {
        control.nodes = control.data.nodes;
        control.links = control.data.links;
    }
    return control;
    
    function addNodeIfNotThere( node, nodes) {
        for ( var i=0; i < nodes.length; i++) {
            if (nodes&#91;i&#93;.unique == node.unique) return i;
        }
        return nodes.push(node) -1;
    }
}
function organizeData(control) {

    for (var i=0; i < control.nodes.length; i ++ ) {
        var node = control.nodes&#91;i&#93;; 
        node.unique = i;
        node.isCurrentlyFocused = false;
    }
    
    for (var i=0; i < control.links.length; i ++ ) {
        var link = control.links&#91;i&#93;;
        link.unique = i;
        link.source = control.nodes&#91;link.source&#93;;
        link.target = control.nodes&#91;link.target&#93;;
    }
    return control;
}


function initialize () {
   
    var initPromise = $.Deferred();

    getTheData().then( function (data) {  
        var control = {};
        control.data = data;
        control.divName = "#chart";

        control.options = $.extend({
            stackHeight : 12,
            radius : 5,
            fontSize : 14,
            labelFontSize : 8,
            nodeLabel : null,
            markerWidth : 0,
            markerHeight : 0,
            width : $(control.divName).outerWidth(),
            gap : 1.5,
            nodeResize : "",
            linkDistance : 80,
            charge : -120,
            styleColumn : null,
            styles : null,
            linkName : null,
            nodeFocus: true,
            nodeFocusRadius: 25,
            nodeFocusColor: "black",
            labelOffset: "5",
            gravity: .05,
            height : $(control.divName).outerHeight()
        }, control.data.d3.options);
    
        
        var options = control.options;
        options.gap = options.gap * options.radius;
        control.width = options.width;
        control.height = options.height;
        control.data = control.data.d3.data;
        control.nodes = control.data.nodes;
        control.links = control.data.links;
        control.color = d3.scale.category20();
        control.clickHack = 200;
        organizeData(control);
    
        control.svg = d3.select(control.divName)
            .append("svg:svg")
            .attr("width", control.width)
            .attr("height", control.height);
        
    
        // get list of unique values in stylecolumn
        control.linkStyles = &#91;&#93;;
        if (control.options.styleColumn) {
         var x;
         for (var i = 0; i < control.links.length; i++) {
          if (control.linkStyles.indexOf( x = control.links&#91;i&#93;&#91;control.options.styleColumn&#93;.toLowerCase()) == -1)
           control.linkStyles.push(x);
         }
        } 
        else
         control.linkStyles&#91;0&#93; = "defaultMarker";
        
        control.force = d3.layout.force().
            size(&#91;control.width, control.height&#93;)
            .linkDistance(control.options.linkDistance)
            .charge(control.options.charge)
            .gravity(control.options.gravity);
    
    
       initPromise.resolve(control);
    });
    return initPromise.promise();
}

function getTheData() {
    var dataPromise = getTheRawData();
    var massage = $.Deferred();
    dataPromise.done ( function (data) {
        // need to massage it
        massage.resolve ( dataMassage (data));    
    })
    .fail (function (error) {
        console.log (error);
        massage.reject(error);
    });
    return massage.promise();
}

function dataMassage(data) {
    //some basic options

    var options = { radius:2.5, nodeLabel:"label", 
                    nodeResize:"count", gravity:0.1, charge:-200, linkDistance:180, height:800,
                    nodeFocus:true}, nodes =&#91;&#93; ,links = &#91;&#93;;
    
    // simple linking tag to name
    for ( var i=0;i<data.data.length;i++) {
        for ( var j=0; j < data.data&#91;i&#93;.tags.tagmap.length ; j++ ) {
            var c =0;
            for ( var k = 0 ; k < data.data&#91;i&#93;.tags.tagmap&#91;j&#93;.counts.length ; k++) {
                c += data.data&#91;i&#93;.tags.tagmap&#91;j&#93;.counts&#91;k&#93;;
            }
            if (c) {
                // we have an interesting link - store it
                var node = 
                    {   name: data.data&#91;i&#93;.key, 
                        count: c,
                        group: data.data&#91;i&#93;.tags.tagmap&#91;j&#93;.name,
                        linkCount: 0,
                        label: data.data&#91;i&#93;.title,
                        shortName: data.data&#91;i&#93;.name,
                        url:data.data&#91;i&#93;.url 
                    };
                var sk = findOrAddNode (node, nodes);
                var node = 
                    {   name: data.data&#91;i&#93;.tags.tagmap&#91;j&#93;.name,
                        count: c,
                        group: data.data&#91;i&#93;.tags.tagmap&#91;j&#93;.name,
                        linkCount: 0,
                        label: data.data&#91;i&#93;.tags.tagmap&#91;j&#93;.name,
                        shortName: data.data&#91;i&#93;.tags.tagmap&#91;j&#93;.name,
                        url:data.data&#91;i&#93;.url 
                    };
                var tk = findOrAddNode (node, nodes);
                links.push ( { source: sk, target: tk, depth:1, linkName: nodes&#91;sk&#93;.url });
            }    
        }
    }
    return { d3: { options : options , data : { nodes: nodes, links: links } }};
}


function findOrAddNode(node,nodes) {
    node.linkCount++;
    for ( var i=0;i<nodes.length;i++) {
        if ( nodes&#91;i&#93;.name === node.name ) { 
            nodes&#91;i&#93;.count += node.count;
            return i;
        }
    }    
    // no match
    return nodes.push(node) - 1 ;
}

function getParameterByName(name) {
    name = name.replace(/&#91;\&#91;&#93;/, "\\\&#91;").replace(/&#91;\&#93;&#93;/, "\\\&#93;");
    var regex = new RegExp("&#91;\\?&&#93;" + name + "=(&#91;^&#&#93;*)"),
        results = regex.exec(location.search);
    return results == null ? "" : decodeURIComponent(results&#91;1&#93;.replace(/\+/g, " "));
}
// modify with your proxy and dataurl
// take the raw data and prepare it for d3
function getTheRawData() {
    // here's a php proxy to make jsonP
     var proxyUrl="https://script.google.com/macros/s/AKfycbzMhwJy-OAR28YTBxO1AbVdSQQFI101X3UDzY-1yc9lUgBfpSc/exec";
    // blogplay.json - blog only
    // allplay.json - both
    // play.json - site
    // var fileid = getParameterByName('data') || '0B92ExLh4POiZTFgwcWtXUG1qVU0';
    var dataUrl = getParameterByName('data') || "https://storage.googleapis.com/xliberation.com/dump/blogplay.json";

    // promise will be resolved when done
    return getPromiseData(dataUrl,proxyUrl);
}
// no need to touch this
// general deferred handler for getting json data through proxy and creating promise
function getPromiseData(url,proxyUrl){
  var deferred = $.Deferred();
  var u = proxyUrl+"?url="+encodeURIComponent(url);

  $.getJSON(u, null, 
        function (data) {
            if (Math.floor(data.responseCode/100 ) !== 2) {
              throw 'error getting data ' + data.result;
            }
            // data returned by apps script proxy is encoded json.
            deferred.resolve(JSON.parse(data.result));
    })
    .error(function(res, status, err) {
        deferred.reject("error " + err + " for " + url);
    });
  
  return deferred.promise();
}

</script></head><body><div>d3.js force directed node focus:<strong>Site Tag Explorer</strong><span style="color:darslateblue;"> <small>click a node to focus, double click to visit page</small></span><aside><small>
   <a href='http://ramblings.mcpher.com'>ramblings.mcpher.com</a><br>
    works best on Chrome<br> ackowledgements:<a href='http://bost.ocks.org/mike/'>Mike Bostok for d3.js</a></small>
   </aside>
</div>
  <div id="chart">

  </div>

 </body>
</html>

For more on this , see GAS and sites and d3.js and this excelramblings blogpost