Web API opening a zip file?

Nate_Riordan

Hi All,

 

I'm working with a customer getting data from a web API that happens to send the data down to the user in a zip format.  I've managed to figure out how to change the default from XML in the archive to CSV, but I'm still running into the challenge of getting the data pulled into JMP instead of the downloads folder.

Data is coming from:

• CAISO
• EIA

 

 

The web command will get the data downloaded nicely.  But it would only be truly useful to the customer if the data goes from Web API into JMP. @Craige_Hales had some great posts to point me further down the Run Program() and cURL path:

• https://community.jmp.com/t5/Discussions/How-to-unzip-using-JMP-Zip-Archive-Commands/td-p/19589
• https://community.jmp.com/t5/Uncharted/Web-site-wants-a-password/ba-p/29267

However I'm still getting stuck.  Perhaps I need to learn more about the Run Program read statement, or understand cURL options more.

 

 

/* generating a start time string for query injection */
starttime = "20160418T07:00-0000";
startdatetime = "&startdatetime=" || starttime;
If( starttime == "",startdatetime = "");

/* generating a end time string for query injection */
endtime = "";
enddatetime = "&enddatetime=" || endtime;
If( endtime == "",enddatetime = "");

/* FILENAME */
FILENAME = "CURR_LMP_GRP";

// CAISO (california energy pricing API)
oasis = "http://oasis.caiso.com/oasisapi/";
query = "GroupZip?groupid="|| FILENAME || startdatetime || "&version=1&resultformat=6";
url = oasis || query;

// download the data
web(url);

// the goal is to directly pull the item into JMP
/* RunProgram to pull the table into JMP (taken from a Craig Hales Post) */

If( HostIs("Windows"),
	data = RunProgram(
	executable( Convert File Path( "$desktop", windows ) || "curl.exe" ),
	options( "-k -s --basic " || url ),
	readfunction( "blob" )
	);
, // else mac version
	data = RunProgram(
	executable( "/usr/bin/curl" ),
	options( {"-k", "-s", "--basic", url} )
	, readfunction( "blob" )
	);
);

try( dt = open(data, "text"); error="", error=exception_msg; dt=empty() );
show(dt,error);

 

There is some potential extensibility of this problem for us as well.  Think of the demo power of being able to scrape amazon's web api for customer reviews.  Run Program() and a python script could do the trick.

 

--Nate

Craige_Hales

try something like this:

 

za = open( "http://someurl.that.downloads.a.zip", "zip"); // 2nd zip important!

show(za<<members);

txt = za<<read( "member1" ); // name from list above

 

za is the ZipArchive object in the scripting index. the url might not end in .zip but must download a zip.