Driver Import.io

This is the the driver for DB.IMPORTIO described in Database abstraction with google apps script

 

The library reference is 

MJWmIWvhOOsv401piLKzeNai_d-phDA33

Page Content hide
1 Reference
2 Cleaning up column headings
3 The Code

Reference

This driver came about following this post from +Martin Hawksey about import.io web scraping, and it uses his library to access importio. This is a little different from the others – it only has a query() method. Import.io allows you do set up some web scraping, and this driver will pick up the data associated with a particular scraping instance. Of course caching is built in just as with the other drivers.
 
The siloId is the importio data set reference, and in this example i’ve stored my importio restAPIkey and my user id in my user properties like this, so you should head over to importio and get your own.
 
{“restAPIKey”:”3N4k1+hJgy0xxxxxxxxxxBd5lgoEYy/7K60V1899gK32ljKs6Ojhu8YQ==”,”userId”:”3be1fccf-xxxxx19d1079″}
 
Here’s how to get a handler.
 
  var importioHandler = new cDataHandler.DataHandler
    ('caff10dc-3bf8-402e-b1b8-c799a77c3e8c',cDataHandler.dhConstants.DB.IMPORTIO,
      undefined,'myimportio',JSON.parse(UserProperties.getProperty("importioKeys")));
 
The interesting thing about this of course is that, used in conjunction with the other databases it makes it easy to copy from impoty.io to a range of back ends in a few lines. Here’s a session that populates each of parse.com, fusion tables, scriptdb, a json file on google drive, a spreadsheet, properties service and orchestrate.io from data taken from an import.io scraper query.
 
function driveTest4() {
 
// this takes some data from import.io scraper and replicates it to all known back end databases
  
  var importioHandler = new cDataHandler.DataHandler
    (‘caff10dc-3bf8-402e-b1b8-c799a77c3e8c’,cDataHandler.dhConstants.DB.IMPORTIO,
      undefined,’myimportio’,JSON.parse(UserProperties.getProperty(“importioKeys”)));  
  var importioData = importioHandler.query({searchterm: “avengers 2”},undefined,1);
  if (importio.handleCode <0) throw (JSON.stringify(importioData));
  
  // write to a sheet after deleting all
  var sheetHandler = new cDataHandler.DataHandler(‘avengers’,cDataHandler.dhConstants.DB.SHEET,undefined, sheetId);  
  
  var result = sheetHandler.remove ();
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  var result = sheetHandler.save (importioData.data);
  if (result.handleCode <0) throw (JSON.stringify(result));
   
  // write to a json file ON google drive
  var driveHandler = new cDataHandler.DataHandler(‘avengers.json’,cDataHandler.dhConstants.DB.DRIVE,undefined,’/datahandler/driverdrive’); 
  
  var result = driveHandler.remove ();
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  var result = driveHandler.save (importioData.data);
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  // write to a parse.com file
  var parseHandler = new cDataHandler.DataHandler
    (‘avengers’,cDataHandler.dhConstants.DB.PARSE,undefined,’myparse’,JSON.parse(UserProperties.getProperty(“parseKeys”))); 
  var result = parseHandler.save (importioData.data);
  if (result.handleCode <0) throw (JSON.stringify(result));
 
  // write to a orchestrate.com file
  var orchestrateHandler = new cDataHandler.DataHandler
    (‘avengers’,cDataHandler.dhConstants.DB.ORCHESTRATE,undefined,’myorchestrate’,JSON.parse(UserProperties.getProperty(“orchestrateKeys”))); 
    
  var result = orchestrateHandler.remove ();
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  var result = orchestrateHandler.save (importioData.data);
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  // write to a scriptdb database
  var dbHandler = new cDataHandler.DataHandler
    (‘avengers’,cDataHandler.dhConstants.DB.SCRIPTDB,undefined,’mydb’,ScriptDb.getMyDb()); 
  var result = dbHandler.save (importioData.data);
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  // write to a fusion  table
  
  //—create a new fusion table .. only need to do this once
 
  var fusionHandler = new cDataHandler.DataHandler(undefined,cDataHandler.dhConstants.DB.FUSION,undefined, ‘avengers’);               
  if(!fusionHandler.isHappy()) throw (‘failed to create fusion table’);
  
  // pick up the name assigned
  var fusionId = fusionHandler.getTableName();
  Logger.log(fusionId);
  
  
  // ifwe know the id..
  //var fusionId = “1hNZzMOjLcy3etuFXqkul2HIKGfEMCtJ47GaZrDm0”;
  //var fusionHandler = new cDataHandler.DataHandler(fusionId,cDataHandler.dhConstants.DB.FUSION,undefined, ‘avengers’);   
  
  var result = fusionHandler.remove ();
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  var result = fusionHandler.save (importioData.data);
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  // write to a property
  var propertiesHandler = new cDataHandler.DataHandler(‘avengers’,cDataHandler.dhConstants.DB.PROPERTIES,undefined, ‘myproperties’,UserProperties);  
  
  var result = propertiesHandler.remove ();
  if (result.handleCode <0) throw (JSON.stringify(result));
  
  var result = propertiesHandler.save (importioData.data);
  if (result.handleCode <0) throw (JSON.stringify(result));
 }
 

Cleaning up column headings

Another quirk about this (and spreadsheets) is that you can find strange column headings that don’t work for databases. Importio.query() will clean these up and return headings that work with all databases.

The Code

It is on GitHub

cDriverImportio created by GitJsonApi
https://github.com/brucemcpherson/cDriverImportio
0 forks.
1 stars.
0 open issues.
Recent commits:

 

For help and more information join our forum, follow the blog or follow me on twitter .