Driver Import.io - Desktop Liberation

This is the the driver for DB.IMPORTIO described in Database abstraction with google apps script

The library reference is

MJWmIWvhOOsv401piLKzeNai_d-phDA33

Page Content hide

Reference

This driver came about following this post from +Martin Hawksey about import.io web scraping, and it uses his library to access importio. This is a little different from the others – it only has a query() method. Import.io allows you do set up some web scraping, and this driver will pick up the data associated with a particular scraping instance. Of course caching is built in just as with the other drivers. The siloId is the importio data set reference, and in this example i’ve stored my importio restAPIkey and my user id in my user properties like this, so you should head over to importio and get your own. {“restAPIKey”:”3N4k1+hJgy0xxxxxxxxxxBd5lgoEYy/7K60V1899gK32ljKs6Ojhu8YQ==”,”userId”:”3be1fccf-xxxxx19d1079″} Here’s how to get a handler. var importioHandler = new cDataHandler.DataHandler ('caff10dc-3bf8-402e-b1b8-c799a77c3e8c',cDataHandler.dhConstants.DB.IMPORTIO, undefined,'myimportio',JSON.parse(UserProperties.getProperty("importioKeys"))); The interesting thing about this of course is that, used in conjunction with the other databases it makes it easy to copy from impoty.io to a range of back ends in a few lines. Here’s a session that populates each of parse.com, fusion tables, scriptdb, a json file on google drive, a spreadsheet, properties service and orchestrate.io from data taken from an import.io scraper query. function driveTest4() { // this takes some data from import.io scraper and replicates it to all known back end databases var importioHandler = new cDataHandler.DataHandler (‘caff10dc-3bf8-402e-b1b8-c799a77c3e8c’,cDataHandler.dhConstants.DB.IMPORTIO, undefined,’myimportio’,JSON.parse(UserProperties.getProperty(“importioKeys”))); var importioData = importioHandler.query({searchterm: “avengers 2”},undefined,1); if (importio.handleCode <0) throw (JSON.stringify(importioData)); // write to a sheet after deleting all var sheetHandler = new cDataHandler.DataHandler(‘avengers’,cDataHandler.dhConstants.DB.SHEET,undefined, sheetId); var result = sheetHandler.remove (); if (result.handleCode <0) throw (JSON.stringify(result)); var result = sheetHandler.save (importioData.data); if (result.handleCode <0) throw (JSON.stringify(result)); // write to a json file ON google drive var driveHandler = new cDataHandler.DataHandler(‘avengers.json’,cDataHandler.dhConstants.DB.DRIVE,undefined,’/datahandler/driverdrive’); var result = driveHandler.remove (); if (result.handleCode <0) throw (JSON.stringify(result)); var result = driveHandler.save (importioData.data); if (result.handleCode <0) throw (JSON.stringify(result)); // write to a parse.com file var parseHandler = new cDataHandler.DataHandler (‘avengers’,cDataHandler.dhConstants.DB.PARSE,undefined,’myparse’,JSON.parse(UserProperties.getProperty(“parseKeys”))); var result = parseHandler.save (importioData.data); if (result.handleCode <0) throw (JSON.stringify(result)); // write to a orchestrate.com file var orchestrateHandler = new cDataHandler.DataHandler (‘avengers’,cDataHandler.dhConstants.DB.ORCHESTRATE,undefined,’myorchestrate’,JSON.parse(UserProperties.getProperty(“orchestrateKeys”))); var result = orchestrateHandler.remove (); if (result.handleCode <0) throw (JSON.stringify(result)); var result = orchestrateHandler.save (importioData.data); if (result.handleCode <0) throw (JSON.stringify(result)); // write to a scriptdb database var dbHandler = new cDataHandler.DataHandler (‘avengers’,cDataHandler.dhConstants.DB.SCRIPTDB,undefined,’mydb’,ScriptDb.getMyDb()); var result = dbHandler.save (importioData.data); if (result.handleCode <0) throw (JSON.stringify(result)); // write to a fusion table //—create a new fusion table .. only need to do this once var fusionHandler = new cDataHandler.DataHandler(undefined,cDataHandler.dhConstants.DB.FUSION,undefined, ‘avengers’); if(!fusionHandler.isHappy()) throw (‘failed to create fusion table’); // pick up the name assigned var fusionId = fusionHandler.getTableName(); Logger.log(fusionId); // ifwe know the id.. //var fusionId = “1hNZzMOjLcy3etuFXqkul2HIKGfEMCtJ47GaZrDm0”; //var fusionHandler = new cDataHandler.DataHandler(fusionId,cDataHandler.dhConstants.DB.FUSION,undefined, ‘avengers’); var result = fusionHandler.remove (); if (result.handleCode <0) throw (JSON.stringify(result)); var result = fusionHandler.save (importioData.data); if (result.handleCode <0) throw (JSON.stringify(result)); // write to a property var propertiesHandler = new cDataHandler.DataHandler(‘avengers’,cDataHandler.dhConstants.DB.PROPERTIES,undefined, ‘myproperties’,UserProperties); var result = propertiesHandler.remove (); if (result.handleCode <0) throw (JSON.stringify(result)); var result = propertiesHandler.save (importioData.data); if (result.handleCode <0) throw (JSON.stringify(result)); }

Cleaning up column headings

Another quirk about this (and spreadsheets) is that you can find strange column headings that don’t work for databases. Importio.query() will clean these up and return headings that work with all databases.