In Node focus on Blogger posts and Node focus explorer without excel I showed how you might browse sites by connected topic, rather than hierarchically. This used a D3.js force diagram as the navigation tool.
I got to thinking about ‘interactive concept browsing’, and wondered if the d3 work could be tweaked to be a kind of hybrid list of pages, with connections through topics. Here’s what it looks like, using the posts from the Excel liberation blog.
And the live version is here
As in the previous explorer a number of the same techniques are used for retrieving the data, you can also focus in on particular topics or pages by clicking or visit the blog page by double clicking. In addition hovering will show connections between the concepts.
Hovering
Fixed positioning
This kind of approach is possible because you can mix fixed positioning and dynamic positioning in D3. simply setting d.fixed to true in your data will keep that item fixed, whilst the others are calculated relative to each other and the fixed items. So in this way you can build up a list of pages that stay where they are, whilst the topic nodes float around as needed.
Moving fixed items
But of course you may want to move these items, perhaps to group them in some way. Well you can. When you drag a page to a new position just stays there. In this example I wanted to focus on the page ‘Publishing Google Apps Script snippets’, so i pulled it to the center of the diagram. Anything else I open up will now be centered around that item I moved.
You can navigate to the underlying pages by double clicking on any page node. I think this provides an interesting alternative to normal site navigation, and of course the data does not have to be a single site – it can be from many related sources.
The Data
Just like in Node focus on Blogger posts, the navigation data is being created each night by a Google Apps Script crawling the Excel Liberation blog and this site. It’s very straightforward – an array of topics, each one looking like this, which contain an array of pages that mention that topic. The concept explorer makes connections between topics by finding pages that are common between each topics pages array. In the case of the blogger data, the name and the title are the same. When we get to data generated from Google Sites, the name is different. Name and Title are both used in the concept explorer in different contexts. The key is a unique value that will be used to match pages and topics, and the count is used to size the node. In this case the count is the number of times the ‘cdataset’ topic is mentioned in all the following pages (potentially more than once per page)
"name": "cdataset",
"count": 87,
"key": "cdataset",
"pages": [
{
"name": "Sankey diagrams direct from Excel - update",
"key": "7899286130266944386",
"title": "Sankey diagrams direct from Excel - update",
"url": "https://ramblings.mcpher.com/sankeyexcel2/"
},
{
"name": "Connections in electoral data - D3 and VBA follow on from oUseful post",
"key": "861843895324588546",
"title": "Connections in electoral data - D3 and VBA follow on from oUseful post",
"url": "https://ramblings.mcpher.com/connections-in-electoral-data-d3-and/"
},
{
"name": "Mashing up electoral data - follow on from oUseful post",
"key": "2877216136096344149",
"title": "Mashing up electoral data - follow on from oUseful post",
"url": "https://ramblings.mcpher.com/mashing-up-electoral-data-follow-on"
},
You can see it would be straightforward to generate this kind of data from other sources, including crawling other websites and combining the results. Here are the combined results for this site, or for this site combined with the blogger. If you can massage your data into the same shape, you can just pass its url as follows to visualize it
https://storage.googleapis.com/xliberation.com/googlecharts/d3concept.html?data=https%3A%2F%2Fstorage.googleapis.com%2Fxliberation.com%2Fdump%2Fsite
The GAS code is here.
The Code to write the HTML app is below.
<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8" /> <title>d3.js concept browser - ramblings.mcpher.com</title> <base href="https://storage.googleapis.com/xliberation.com/"> <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" /> <link rel="stylesheet" type="text/css" href="cdn/css/d3direct.css"> </head><body>d3.js force directed node focus:<strong>Site Concept Browser</strong> <small>click a node to focus, double click to visit page</small><aside><small> <a href='http://ramblings.mcpher.com'>ramblings.mcpher.com</a> ackowledgements:<a href='http://bost.ocks.org/mike/'>Mike Bostok(d3.js)</a></small> </aside> </body> </html>
Return to Gas and Sites to read more topics