http請求的批次你好,
我嘗試編寫一個腳本,透過組合端點來發送 http 請求。
將請求本身放在一起不是問題,但在某些情況下,我需要一次請求 1000 多個專案 (test_article) 的一些資訊。
我受到 http 請求最大長度的限制,一次只能要求大約 1000 個。
不幸的是,我無法弄清楚如何組合一個循環來發送批量 1000 個 test_article 的請求,並在收到這些請求時將其連接起來。
任何見解將不勝感激
進階統計建模自動化和腳本編寫消費者與市場研究資料混合與清理品質和工藝工程回覆:http請求的批次嗨瑞安,
感謝你的回答。 我會看看是否可以根據我在下面列出的問題調整您的建議。
最好的,
賽巴斯蒂安回覆:http請求的批次我意識到我遺漏了我最初詢問時開始的腳本。 為了清楚起見,這是起點腳本,我想找到一種在專案數量超過 1000 時分解為批次的方法。
最好的
Names Default To Here( 1 );
dt = Data Table( "test_article" ); // This is a summary data table only contains 1 column of interest 'test_article' - I want to retrieve some info for each of the unique items in that column
//If the number of items exceeds 1000, I'll need to break down the query in batches of 1000 or less to construct the http request
//Right if the list is 1000 or less, I'm golden, if it's greater, the request will fail
dt:test_article << set data type( "Numeric" );
dt << Transpose( columns( :test_article ), Output Table( "Transpose of test_article" ) );
dt = Data Table( "Transpose of test_article" );
Col_List = dt << Get Column Names( "String" );
For( i = 1, i <= N Items( Col_List ), i++,
If( Contains( Col_List[i], "Row" ),
Column( Col_List[i] ) << Set Selected( 1 )
)
);
dt << Combine Columns(
delimiter( "," ),
selected Columns only( 1 ),
Selected Columns are Indicator Columns( 0 ),
Column Name( "test_article" )
);
nc = N Cols( Data Table( "Transpose of test_article" ) );
Show( nc );
rowNum = 1;
col = Column( Data Table( "Transpose of test_article" ), "test_article" );
tableVal = col[rowNum];
adr0 = "http:/xyz:1234/oligos/more-oligo-info-intended?ids=";
address = adr0 || tableval;
request = New HTTP Request( URL( address ), Method( "GET" ), Timeout( 240 ) );
json1 = request << Send;
dt = JSON To Data Table( json1 );
Current Data Table() << set name( "test_article_info" );
Data Table( "test_article" ) << close window;
Data Table( "Transpose of test_article" ) << close window;
// I need to build a for loop that will send 1 request after the next and progressively concatenate those file together
回覆:http請求的批次多謝! 我會嘗試根據我的問題調整您的方法。回覆:http請求的批次這是我使用的一種方法。 它使用 JMP 的單一實例。 程式碼將 HTTP 結果組合成單一 JSON 對象,隨後將其轉換為資料表。
list = {};
For( i = 1, i <= number_of_iterations, i++,
request = HTTP Request(
Url( url ),
Method( "GET" ),
Headers( request_headers ),
QueryString( query ),
Password( access_token ),
Timeout( 360 )
);
data = request << Send;
If( request << IsSuccess,
response = Parse JSON( data );
If( AndMZ( Is Associative Array( response ), Contains( response, "records" ), N Items( response["records"] ) > 0 ),
list ||= response["records"]
);
,
Wait( 0.25 )
);
);回覆:http請求的批次我有一個關於後台 http 請求的問題@Craige_Hales回答道。
HTTP後台請求
https://community.jmp.com/t5/Uncharted/Can-JMP-run-JMP/ba-p/283748#U283748
然後我獲取了他的程式碼並對其進行了修改以執行您正在尋找的操作。就我而言,我正在查詢不同的日期,但您可以修改程式碼以循環您想要的任何內容。您可能需要根據您的特定用例進行一些調整。
希望有幫助。
url = "www....";
//Using this to generate different queries
query_keys = Associative Array();
query_keys["key1"] ="Bob"; //Or whatever your query parameters are
//I am looping over different dates
days_back = 20;
day_inc = 2;
start = Date Increment(today(),"day",-days_back);
end = Today();
cur_start = Date Increment(today(),"day",-day_inc);
cur_end = end;
index = 1;
//Here I am making files that contain the url and query key for each http request. I will use these later. This is only really needed if you intent to parallelize the http requset among multiple jmp instances.<BR />//If you don't want to parallelize the queries you can skip the writing to disk and just make a list of your query keys and/or urls.
while(cur_end>start,
query_key["startdate"] = char(Format(cur_start,"yyyy-mm-dd"));
query_key["enddate"] = char(Format(cur_end,"yyyy-mm-dd"));
Save Text File("$temp/url_file"||char(index)||".txt",url);
Save Text File("$temp/query_keys_file"||char(index)||".txt",As JSON Expr(query_keys));
index++;
if(Date Increment(cur_start,"day",-day_inc)>start,
cur_start = Date Increment(cur_start,"day",-day_inc),
cur_start = start;
);
cur_end = Date Increment(cur_end,"day",-day_inc);
);
index--;
nscripts = index;
njmp = 1; //You can parallelize this as much as you want by launching more jmp instances. Or if you want to
// a place to keep up with running instances
rp = [=> ];
rpIndex=0;
For( i = 1, i <= njmp, i += 1,
// build a custom JSL for each JMP to execute
workerFileName = Save Text File(
"$temp\deleteMe_RunsWhenLaunchedBeCareful" || Char( i ) || ".jsl",
"\[//!
deletefile("$temp/sentinel.txt");
for(i=1,i<=]\"||char(nscripts)||"\[,i++,
if(Is File("$temp/url_file"||char(i)||".txt"),
url = Load Text File( "$temp/url_file"||char(i)||".txt" ) ;
query_key = Parse JSON( Load Text File( "$temp/query_keys_file"||char(i)||".txt" ) );
Delete File("$temp/url_file"||char(i)||".txt");
Delete File("$temp/query_keys_file"||char(i)||".txt");<BR />//Using try so if one request fails I can keep moving through the requests
Try(
dig_http = New HTTP Request(URL(url),Method("Get"),Query String(query_key),Timeout(60*60*2))<<SEND></SEND>//My data is coming in as csv so I convert that to jmp.
file_name = "$temp/data"||char(i)||".csv";
Save Text File(file_name,dig_http);
dt = Open(file_name);
dt << Save("$temp/data"||char(i)||".jmp");
Delete File("$temp/data"||char(i)||".csv");
,<BR />//Dummy file if try fails
dt = New Table();
dt << Save("$temp/data"||char(i)||".jmp");
);
);
);
exit();
]\"
);
// don't launch another copy until the previous copy is running.
// JMP will become unhappy if the preferences file is busy in another
// copy of JMP. The sentinel will be deleted when JMP starts
savetextfile("$temp/sentinel.txt","");
rp[rpIndex+=1] = Run Program( executable( "jmp" ), options( {workerFileName} ) );
write("\!n",i," started...");
// wait for the sentinel to vanish
while(fileexists("$temp/sentinel.txt"),
wait(1);// don't burn the CPU while waiting
);
write("ok");
);
// many queries are probably still queued, wait for them to finish
scripts_left = nscripts;
While( scripts_left,
scripts_left = 0;
For( i = 1, i <= nscripts, i++,
If(Is File("$temp/url_file"||char(i)||".txt"),
scripts_left += 1; // count the ones still alive
)
);
write("\!n",scripts_left," http requests left");
wait(1); // don't burn the CPU while waiting
);
Write( "\!ngathering results..." );
dt = New table("Final Table");
For( i = 1, i <= nscripts, i += 1,
filename="$temp/data"||char(i)||".jmp";
// do something with the results
// clean up the temp files
tmp_dt = open(filename);
dt << Concatenate(tmp_dt,Append to First Table);
Close(tmp_dt);
deletefile(filename);
deletefile("$temp\deleteMe_RunsWhenLaunchedBeCareful" || Char( i ) || ".jsl");
);
Write( "\!ndone" );