如何使用讀取功能處理大量運行程序我正在嘗試運行一個將大量逗號分隔數據打印到控制台的外部應用程序。 輸出本質上是一個大型數據表,來自 .NET 應用程序。 “行”以逗號分隔,每行後跟一個換行符。 第一行是標題,需要用這些更新數據表。 我遵循了讀取控制台的示例代碼,但我遇到了一些問題,其中一些行部分完成(未在換行符處分隔)。 我試圖然後剝離部分行並將其附加到下一行的開頭,但我仍然缺少幾行(約 35 出 4338)。 如果我寫入一個文本變量並在程序完成後進行處理,它可以工作,但速度要慢得多(大概是因為字符串很長並且它與外部應用程序不並發,但這只是我的猜測)。
有什麼方法可以控制何時調用讀取函數,以便我可以一次捕獲整行? 是否有任何已知的方法來處理這個問題(數據是 4000 行 x 約 200 列,可能填充了大約 50% 的單元格,並且每個單元格填充了大約 10 個字符的字符串,以防相關)。
我確實可以訪問 .NET 源代碼,並且可以修改輸出以使其對 JSL 友好,以備不時之需。
我正在粘貼我的代碼,用“IDx”替換敏感的列名。 提前致謝!
results = Run Program(
Executable(myExecutable),
Options(options),
Read Function(Function({this},
fullValue = this << Read;
lines = Words( fullValue,"\!n" );
lineIndex = 1;
totalLines = totalLines + N Items(lines);
callCount++;
words = Words(lines[1], ",");
if (N Items(words) > 1,
if (extraContent != "", lines[1] = appendExtraContent(lines[1],extraContent),);
newExtraContent = getExtraContent(fullValue);
if (N Items(lines) == 1,
extraContent = extraContent || newExtraContent,
extraContent = newExtraContent
);
if (extraContent != "",
print(lines[NItems(lines)]);
Remove From(lines, N Items(lines));
extraLines++;,
);
if (N Items(lines) >= 1,
if (first == 1,
headers = Words(lines[1],",");
For (i = 14, i <= N Items(headers), i++,
dt << New Column(headers[i], Character);
);
headerCount++;
first = 0;
lineIndex++;
cols = dt << Get Column Names();,);
For( i = lineIndex, i <= N Items(lines), i++,
//print(lines[i]);
count++;
word = break_string_by_sep(lines[i], ",");
if ( N Items(word) > maxHeaders, maxHeaders = N Items(word),);
if (N Items(word) >= 14,
dt << Add Row({
:ID1 = word[1];
:ID2 = word[2];
:ID3 = Num(word[3]);
:ID4 = Num(word[4]);
:ID5 = word[5];
:ID6 = Num(word[6]);
:ID7 = word[7];
:ID8 = Num(word[8]);
:ID9 = word[9];
:ID10 = word[10];
:ID11 = Num(word[11]);
:ID12 = Num(word[12]);
:ID13 = Num(word[13]);
For (k = 14, k <= N Items(cols) , k++,
if (N Items(word) >= k,
AsColumn(dt,cols[k]) = word[k],
);
);
}),
);
);
,);
,);
);
)
);
While (!(results << Is Read EOF), Wait(1));
dt << ShowWindow(1);
);
appendExtraContent = Function({partialValue, extra}, {default local},
fullLine = extra || partialValue;
fullLine;
);
getExtraContent = Function({allLines}, {default local},
lines = Words(allLines, "\!n");
lastValue = N Items(lines);
startValue = Contains(allLines, lines[lastValue]);
last7 = Substr(allLines, -1);
if (last7 == "\!n",
extra = "",
extra = Substr(allLines, startValue);
);
extra;
);
Re: 如何使用讀取功能處理大量運行程序嘗試這樣的事情(為 windows .bat 文件編寫以模擬您的外部數據源):
dt = New Table( "fetched", New Column( "a" ), New Column( "b" ), New Column( "c" ), New Column( "d" ), New Column( "bufsize" ) );
// https://stackoverflow.com/questions/2591758/batch-script-loop
bat = Save Text File( "$temp/data.bat", "@echo off\!nfor /l %%x in (1, 1, 100000) do echo %%x, 444, 555, 666" );
buffer = "";
runprog = RunProgram(
executable( "CMD.EXE" ),
options( {"/C", bat} ),
readfunction(
Function( {this},
{data},
While( this << canread,
buffer = buffer || (this << read);
While( Pat Match( buffer, Pat Break( "\!n" ) >> data + "\!n", "" ),
dt << addrow( 1 );
{dt:a, dt:b, dt:c, dt:d} = Words( data, "," );
dt:bufsize = Length( buffer );
If( Mod( N Rows( dt ), 1000 ) == 0,
Wait( 0 )
); // show the updates
);
)
)
)
);
100,000 行表格更新需要一些時間(可能需要 15 秒)。 它以這種方式運行得更快(約 4 秒):
dt = New Table( "fetched", New Column( "a" ), New Column( "b" ), New Column( "c" ), New Column( "d" ), New Column( "bufsize" ) );
// https://stackoverflow.com/questions/2591758/batch-script-loop
bat = Save Text File( "$temp/data.bat", "@echo off\!nfor /l %%x in (1, 1, 100000) do echo %%x, 444, 555, 666" );
buffer = "";
dt << begin Data Update;
runprog = RunProgram(
executable( "CMD.EXE" ),
options( {"/C", bat} ),
readfunction(
Function( {this},
{data},
While( this << canread,
buffer = buffer || (this << read);
While( Pat Match( buffer, Pat Break( "\!n" ) >> data + "\!n", "" ),
dt << addrow( 1 );
{dt:a, dt:b, dt:c, dt:d} = Words( data, "," );
dt:bufsize = Length( buffer );
);
)
)
)
);
Write( "\!nWaiting\!n" );
While( !(runprog << isreadeof), Wait( .01 ) );
dt << end Data Update;
Write( "\!nDone" );
<<開始數據更新可防止表格格式化大量數字。 最後的 while 等待運行程序對象完成讀取,然後再次打開數據更新。 CMD.EXE 窗口需要大約 4 秒來滾動 .bat 文件的輸出。