基於 FOR() 迴圈的(慢速)JSL 程式碼的最佳化你好,
我需要編寫一個用於可靠性資料分析的程式碼。
在實驗開始 (REL_T0) 之前測量多個指標(metric1、metric2、...metricN),然後在每次丟棄事件(DROP1、DROP2、...DROPN...)後再次測量。
我的目標是計算每個序號和每個指標的掉落事件後 REL_T0 與每個測量值之間的漂移值。
我的初始程式碼是基於 FOR() 迴圈 - 我知道這絕對不是最佳的 - 並且適用於小型資料集。
儘管如此,我需要優化我的程式碼,使其在更大的資料集上運行得更快。
目前,如果有 10000 行和 50 個指標,該過程大約需要 20 分鐘才能完成,這太長了。
我嘗試以不同的方式組織 FOR() 循環,但沒有產生重大影響。
我確信有一些方法可以使用 JSL 專有命令以更快的方式進行這些簡單的計算。
你們有什麼感想?
下面的程式碼建立一個虛擬表,然後執行我的初始程式碼,最後得到一個有計算漂移的新欄位「Δ_metricN」。
我想使用 JSL 命令而不是 FOR() 循環來做同樣的事情,這將使我的程式碼快如閃電!
預先非常感謝。
牧師。
//Create dummy table
RawData = New Table( "RawData",
Add Rows( 20 ),
New Column( "serial_number", Character, set values( {"A","A","A","A","A","A","A","B","B","B","B","B","B","C","C","C","C","C","C","C"})),
New Column( "rel_event", Character, set values( {
"REL_T0","DROP1","DROP2","DROP3","DROP4","DROP5","DROP6",
"REL_T0","DROP1","DROP2","DROP3","DROP4","DROP5",
"REL_T0","DROP1","DROP2","DROP3","DROP4","DROP5", "DROP6"})),
New Column("metric1", Continuous, set values({1,2,3,5,7,10,13,2,4,5,6,8,11,1,2,4,5,5,9,12}) ),
New Column("metric2", Continuous, set values({2,3,4,7,9,14,18,2,3,4,7,9,14,2,5,7,8,9,11,13}) );
);
RawData << save( "$Desktop\RawData.jmp" );
Column(RawData, "rel_event") << Set Property( "Value ordering", {"REL_T0","DROP1","DROP2","DROP3","DROP4","DROP5","DROP6"} );
//Create handle for data table to be analyzed
RawData = Current Data Table();
ChosenDataCol = {"metric1", "metric2"};
NumDataCol = N Items( ChosenDataCol );
NumRow = N Row( RawData );
RawDataCol = RawData << get column names;
Show( NumDataCol );
Show( ChosenDataCol );
Show( RawData );
Show( NumRow );
Show( RawDataCol );
//Scan each selected data row for which Deltas must be computed
For( cpt = 1, cpt <= NumDataCol, cpt++,
//Create column to receive delta data
DeltaCol = RawData << New Column( "∆_" || Char( ChosenDataCol[cpt] ), Numeric, "Continuous" );
TargetCol = Column( RawData, ChosenDataCol[cpt] );
serial_numberlist = {}; //List of unique serial_number's within the data table of interest
Show( cpt );
Show( ChosenDataCol[cpt] );
For( i = 1, i <= NumRow, i++,
Currentserial_number = :serial_number[i];
If( Contains( serial_numberlist, Currentserial_number ) == 0,
Insert Into( serial_numberlist, Currentserial_number )
);
);//For i
Show( serial_numberList );
Numserial_number = N Items( serial_numberlist );
Show( Numserial_number );
//Scan data table for each unique serial_number
For( i = 1, i <= Numserial_number, i++,
Currentserial_number = serial_numberList[i];
Currentserial_numberrows = RawData << GetRowsWhere( :serial_number == Currentserial_number ); //matrix of rows containing current serial_number
Show( Currentserial_numberrows );
NumCurrentserial_numberrows = N Row( Currentserial_numberrows );
Show( NumCurrentserial_numberrows );
show(i);
TzeroExist = 0;
TzeroVal = 0;
//identify Time Zero value for each serial_number under consideration
For( j = 1, j <= NumCurrentserial_numberrows, j++,
CurrentRead = :rel_event[Currentserial_numberrows[j]];
Show( CurrentRead );
If( CurrentRead == "REL_T0",
TzeroVal = TargetCol[Currentserial_numberrows[j]];
Show( TzeroVal );
TzeroExist = 1;
);
); //For j
//Calculate Drift
For( j = 1, j <= NumCurrentserial_numberrows, j++,
If( TzeroExist == 1,
DeltaCol[Currentserial_numberrows[j]] = TargetCol[Currentserial_numberrows[j]] - TzeroVal
)
); //For j
);//For i
);//For cpt 進階統計建模自動化和腳本編寫回覆:基於 FOR() 迴圈的(慢)JSL 程式碼的最佳化感謝賈莫的回覆。
我喜歡使用 For Each Row() 的想法,它簡化了整個腳本。
但為了添加更多上下文並使腳本更加健壯,REL_T0 可能並不總是每個序號的第一行(資料庫並不總是按 JMP 表中的正確順序查詢資料),而且漂移也可能是正值或負數,所以我不知道Col Min() 是否適合這裡(也許我沒讀對)。
是否可以直接將start_row錨定到每個序號的 REL_T0 值?
回覆:基於 FOR() 迴圈的(慢)JSL 程式碼的最佳化是的,可以直接對該腳本進行少量更改。我添加了調試列,您可以刪除該列以顯示它與正確的行匹配
Names Default To Here(1);
dt = New Table("RawData",
Add Rows(27),
Compress File When Saved(1),
New Column("serial_number",
Character,
"Nominal",
Set Values(
{"A", "A", "A", "A", "A", "A", "A", "B", "B", "B", "B", "B", "B", "C", "C", "C", "C", "C", "C", "C", "D", "D", "D", "D", "D", "D", "D"}
)
),
New Column("rel_event",
Character,
"Nominal",
Set Values(
{"REL_T0", "DROP1", "DROP2", "DROP3", "DROP4", "DROP5", "DROP6", "REL_T0", "DROP1", "DROP2", "DROP3", "DROP4", "DROP5", "REL_T0", "DROP1",
"DROP2", "DROP3", "DROP4", "DROP5", "DROP6", "DROP1", "DROP2", "DROP3", "REL_T0", "DROP4", "DROP5", "DROP6"}
)
),
New Column("metric1",
Numeric,
"Continuous",
Format("Best", 10),
Set Values([1, 2, 3, 5, 7, 10, 13, 2, 4, 5, 6, 8, 11, 1, 2, 4, 5, 5, 9, 12, 1, 2, 4, 5, 5, 9, 12])
),
New Column("metric2",
Numeric,
"Continuous",
Format("Best", 10),
Set Values([2, 3, 4, 7, 9, 14, 18, 2, 3, 4, 7, 9, 14, 2, 5, 7, 8, 9, 11, 13, 2, 5, 7, 8, 9, 11, 13])
)
);
debugcol = dt << New Column("REL_T0_ROW", Numeric, Continuous);
delta1col = dt << New Column("delta1", Numeric, Continuous);
delta2col = dt << New Column("delta2", Numeric, Continuous);
For Each Row(dt,
// find REL_T0 row for current serial_number
start_row = Col Min(If(:rel_event == "REL_T0", Row(), .), :serial_number);
debugcol[Row()] = start_row;
delta1col[Row()] = :metric1 - :metric1[start_row];
delta2col[Row()] = :metric2 - :metric2[start_row];
);

回覆:基於 FOR() 迴圈的(慢)JSL 程式碼的最佳化非常感謝賈莫!
我在一個包含 18000 行和 2 個指標的資料表中嘗試了這個新版本:
- 舊方法:~5 分鐘
- 你的方法:<2分鐘
已經是很大的進步了!
再次感謝!
牧師
回覆:基於 FOR() 迴圈的(慢)JSL 程式碼的最佳化你好,霍吉,
感謝您的建議,我檢查了您與規範化插件相關的帖子。我一定會在未來的專案中牢記這一點。
牧師
回覆:基於 FOR() 迴圈的(慢)JSL 程式碼的最佳化使用矩陣執行這些計算很可能可以輕鬆地進一步提高速度。還添加
dt << begin data update;
在循環之前和
dt << end data update;
很可能還會提供一些速度改進,特別是如果您的表格沒有以不可見/私密的方式打開,因為它會「延遲」表格中的視覺效果更新。
回覆:基於 FOR() 迴圈的(慢)JSL 程式碼的最佳化測量數據的標準化是數據分析中的常見任務。
標準化 GUI提供了許多選項來執行此操作。
在 GUI 中,使用者可以指定許多不同的設定 - 並且 AddIn 會產生對應的程式碼。
產生的公式是基於Col ...聚合,例如Col Min、Col Max、Col Median ...
其他選項:

對於此應用程式案例,您可以使用「first」(基於ColMin(row())邏輯)
...或透過「使用子集進行標準化」指定參考[對於參考值不是「第一個」值的情況]:

回覆:基於 FOR() 迴圈的(慢)JSL 程式碼的最佳化(通常)提高速度的一件事是從循環中刪除所有列印,但在這種情況下,在沒有這些循環的情況下重寫它會更快,並且在遍歷資料表時不要使用for 循環(對於每一行是更好的選擇)。
這是一個有望產生更快結果的選項(也可以將其轉換為公式,如果需要,它仍然應該相當快)。它假設資料以這樣的方式排序,即 REL_T0 始終是每個序號的第一行,如果不是這種情況,則需要進行一些小的更改
Names Default To Here(1);
//Create dummy table
RawData = New Table("RawData",
Add Rows(20),
New Column("serial_number",
Character,
set values({"A", "A", "A", "A", "A", "A", "A", "B", "B", "B", "B", "B", "B", "C", "C", "C", "C", "C", "C", "C"})
),
New Column("rel_event",
Character,
set values(
{"REL_T0", "DROP1", "DROP2", "DROP3", "DROP4", "DROP5", "DROP6", "REL_T0", "DROP1", "DROP2", "DROP3", "DROP4", "DROP5", "REL_T0", "DROP1",
"DROP2", "DROP3", "DROP4", "DROP5", "DROP6"}
)
),
New Column("metric1", Continuous, set values({1, 2, 3, 5, 7, 10, 13, 2, 4, 5, 6, 8, 11, 1, 2, 4, 5, 5, 9, 12})),
New Column("metric2", Continuous, set values({2, 3, 4, 7, 9, 14, 18, 2, 3, 4, 7, 9, 14, 2, 5, 7, 8, 9, 11, 13}))
);
delta1col = RawData << New Column("delta1", Numeric, Continuous);
delta2col = RawData << New Column("delta2", Numeric, Continuous);
For Each Row(RawData,
// find REL_T0 row for current serial_number
start_row = Col Min(Row(), :serial_number);
delta1col[Row()] = :metric1 - :metric1[start_row];
delta2col[Row()] = :metric2 - :metric2[start_row];
);
