Driver Import.io

This is the the driver for DB.IMPORTIO described in Database abstraction with google apps script 

The library reference is MJWmIWvhOOsv401piLKzeNai_d-phDA33
 

Reference

This driver came about following this post from +Martin Hawksey about import.io web scraping, and it uses his library to access importio. This is a little different from the others - it only has a query() method. Import.io allows you do set up some web scraping, and this driver will pick up the data associated with a particular scraping instance. Of course caching is built in just as with the other drivers.

The siloId is the importio data set reference, and in this example i've stored my importio restAPIkey and my user id in my user properties like this, so you should head over to importio and get your own.

{"restAPIKey":"3N4k1+hJgy0xxxxxxxxxxBd5lgoEYy/7K60V1899gK32ljKs6Ojhu8YQ==","userId":"3be1fccf-xxxxx19d1079"}

Here's how to get a handler.

  var importioHandler = new cDataHandler.DataHandler
    ('caff10dc-3bf8-402e-b1b8-c799a77c3e8c',cDataHandler.dhConstants.DB.IMPORTIO,
      undefined,'myimportio',JSON.parse(UserProperties.getProperty("importioKeys")));
 

The interesting thing about this of course is that, used in conjunction with the other databases it makes it easy to copy from impoty.io to a range of back ends in a few lines. Here's a session that populates each of parse.com, fusion tables, scriptdb, a json file on google drive, a spreadsheet, properties service and orchestrate.io from data taken from an import.io scraper query.

function driveTest4() {

// this takes some data from import.io scraper and replicates it to all known back end databases
  
  var importioHandler = new cDataHandler.DataHandler
    ('caff10dc-3bf8-402e-b1b8-c799a77c3e8c',cDataHandler.dhConstants.DB.IMPORTIO,
      undefined,'myimportio',JSON.parse(UserProperties.getProperty("importioKeys")));  
  var importioData = importioHandler.query({searchterm: "avengers 2"},undefined,1);
  if (importio.handleCode <0) throw (JSON.stringify(importioData));
  
  // write to a sheet after deleting all
  var sheetHandler = new cDataHandler.DataHandler('avengers',cDataHandler.dhConstants.DB.SHEET,undefined, sheetId);  
  
  var result = sheetHandler.remove ();
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  var result = sheetHandler.save (importioData.data);
  if (result.handleCode <0) throw (JSON.stringify(result));
   
  // write to a json file ON google drive
  var driveHandler = new cDataHandler.DataHandler('avengers.json',cDataHandler.dhConstants.DB.DRIVE,undefined,'/datahandler/driverdrive'); 
  
  var result = driveHandler.remove ();
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  var result = driveHandler.save (importioData.data);
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  // write to a parse.com file
  var parseHandler = new cDataHandler.DataHandler
    ('avengers',cDataHandler.dhConstants.DB.PARSE,undefined,'myparse',JSON.parse(UserProperties.getProperty("parseKeys"))); 
  var result = parseHandler.save (importioData.data);
  if (result.handleCode <0) throw (JSON.stringify(result));

  // write to a orchestrate.com file
  var orchestrateHandler = new cDataHandler.DataHandler
    ('avengers',cDataHandler.dhConstants.DB.ORCHESTRATE,undefined,'myorchestrate',JSON.parse(UserProperties.getProperty("orchestrateKeys"))); 
    
  var result = orchestrateHandler.remove ();
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  var result = orchestrateHandler.save (importioData.data);
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  // write to a scriptdb database
  var dbHandler = new cDataHandler.DataHandler
    ('avengers',cDataHandler.dhConstants.DB.SCRIPTDB,undefined,'mydb',ScriptDb.getMyDb()); 
  var result = dbHandler.save (importioData.data);
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  // write to a fusion  table
  
  //---create a new fusion table .. only need to do this once

  var fusionHandler = new cDataHandler.DataHandler(undefined,cDataHandler.dhConstants.DB.FUSION,undefined, 'avengers');               
  if(!fusionHandler.isHappy()) throw ('failed to create fusion table');
  
  // pick up the name assigned
  var fusionId = fusionHandler.getTableName();
  Logger.log(fusionId);
  
  
  // ifwe know the id..
  //var fusionId = "1hNZzMOjLcy3etuFXqkul2HIKGfEMCtJ47GaZrDm0";
  //var fusionHandler = new cDataHandler.DataHandler(fusionId,cDataHandler.dhConstants.DB.FUSION,undefined, 'avengers');   
  
  var result = fusionHandler.remove ();
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  var result = fusionHandler.save (importioData.data);
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  // write to a property
  var propertiesHandler = new cDataHandler.DataHandler('avengers',cDataHandler.dhConstants.DB.PROPERTIES,undefined, 'myproperties',UserProperties);  
  
  var result = propertiesHandler.remove ();
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  var result = propertiesHandler.save (importioData.data);
  if (result.handleCode <0) throw (JSON.stringify(result));
  

}


Cleaning up column headings

Another quirk about this (and spreadsheets) is that you can find strange column headings that don't work for databases. Importio.query() will clean these up and return headings that work with all databases.



The code

For help and more information join our forum,follow the blog or follow me on twitter .


Comments