基于 FOR() 循环的(慢速)JSL 代码优化你好,
我需要编写一个可靠性数据分析的代码。
在实验开始(REL_T0)之前测量几个指标(metric1、metric2、...metricN),然后在每次掉线事件之后再次测量(DROP1、DROP2、... DROPN...)。
我的目标是计算每个序列号和每个指标在丢失事件后 REL_T0 和每个测量值之间的漂移值。
我的初始代码基于 FOR() 循环 - 我知道这肯定不是最佳的 - 并且在小数据集上运行良好。
尽管如此,我还是需要优化我的代码,使其在更大的数据集上运行得更快。
目前,有 10,000 行和 50 个指标,该过程大约需要 20 分钟才能完成,这太多了。
我尝试以不同的方式组织 FOR() 循环,但没有显著的影响。
我确信有方法可以使用 JSL 专有命令以更快的方式完成这些简单的计算。
你们有什么感想?
下面的代码创建了一个虚拟表,然后运行我的初始代码,最终得到具有计算漂移的新列“∆_metricN”。
我希望使用 JSL 命令而不是 FOR() 循环来做同样的事情,这会让我的代码运行速度快如闪电!
提前致谢。
牧师。
//Create dummy table
RawData = New Table( "RawData",
Add Rows( 20 ),
New Column( "serial_number", Character, set values( {"A","A","A","A","A","A","A","B","B","B","B","B","B","C","C","C","C","C","C","C"})),
New Column( "rel_event", Character, set values( {
"REL_T0","DROP1","DROP2","DROP3","DROP4","DROP5","DROP6",
"REL_T0","DROP1","DROP2","DROP3","DROP4","DROP5",
"REL_T0","DROP1","DROP2","DROP3","DROP4","DROP5", "DROP6"})),
New Column("metric1", Continuous, set values({1,2,3,5,7,10,13,2,4,5,6,8,11,1,2,4,5,5,9,12}) ),
New Column("metric2", Continuous, set values({2,3,4,7,9,14,18,2,3,4,7,9,14,2,5,7,8,9,11,13}) );
);
RawData << save( "$Desktop\RawData.jmp" );
Column(RawData, "rel_event") << Set Property( "Value ordering", {"REL_T0","DROP1","DROP2","DROP3","DROP4","DROP5","DROP6"} );
//Create handle for data table to be analyzed
RawData = Current Data Table();
ChosenDataCol = {"metric1", "metric2"};
NumDataCol = N Items( ChosenDataCol );
NumRow = N Row( RawData );
RawDataCol = RawData << get column names;
Show( NumDataCol );
Show( ChosenDataCol );
Show( RawData );
Show( NumRow );
Show( RawDataCol );
//Scan each selected data row for which Deltas must be computed
For( cpt = 1, cpt <= NumDataCol, cpt++,
//Create column to receive delta data
DeltaCol = RawData << New Column( "∆_" || Char( ChosenDataCol[cpt] ), Numeric, "Continuous" );
TargetCol = Column( RawData, ChosenDataCol[cpt] );
serial_numberlist = {}; //List of unique serial_number's within the data table of interest
Show( cpt );
Show( ChosenDataCol[cpt] );
For( i = 1, i <= NumRow, i++,
Currentserial_number = :serial_number[i];
If( Contains( serial_numberlist, Currentserial_number ) == 0,
Insert Into( serial_numberlist, Currentserial_number )
);
);//For i
Show( serial_numberList );
Numserial_number = N Items( serial_numberlist );
Show( Numserial_number );
//Scan data table for each unique serial_number
For( i = 1, i <= Numserial_number, i++,
Currentserial_number = serial_numberList[i];
Currentserial_numberrows = RawData << GetRowsWhere( :serial_number == Currentserial_number ); //matrix of rows containing current serial_number
Show( Currentserial_numberrows );
NumCurrentserial_numberrows = N Row( Currentserial_numberrows );
Show( NumCurrentserial_numberrows );
show(i);
TzeroExist = 0;
TzeroVal = 0;
//identify Time Zero value for each serial_number under consideration
For( j = 1, j <= NumCurrentserial_numberrows, j++,
CurrentRead = :rel_event[Currentserial_numberrows[j]];
Show( CurrentRead );
If( CurrentRead == "REL_T0",
TzeroVal = TargetCol[Currentserial_numberrows[j]];
Show( TzeroVal );
TzeroExist = 1;
);
); //For j
//Calculate Drift
For( j = 1, j <= NumCurrentserial_numberrows, j++,
If( TzeroExist == 1,
DeltaCol[Currentserial_numberrows[j]] = TargetCol[Currentserial_numberrows[j]] - TzeroVal
)
); //For j
);//For i
);//For cpt 高级统计建模自动化和脚本回复:基于 FOR() 循环的(慢速)JSL 代码优化感谢 Jarmo 的回复。
我喜欢使用 For Each Row() 的想法,它简化了整个脚本。
但是为了添加更多上下文并使脚本更加健壮,REL_T0 可能并不总是每个序列号的第一行(数据库并不总是按照 JMP 表中的正确顺序查询数据),并且漂移可能是正的或负的,所以我不知道 Col Min() 是否适合这里(也许我读得不正确)。
是否可以直接将start_row锚定到每个序列号的 REL_T0 值?
回复:基于 FOR() 循环的(慢速)JSL 代码优化是的,只需对该脚本进行少量修改即可。我添加了调试列,您可以将其删除以显示它与正确的行匹配
Names Default To Here(1);
dt = New Table("RawData",
Add Rows(27),
Compress File When Saved(1),
New Column("serial_number",
Character,
"Nominal",
Set Values(
{"A", "A", "A", "A", "A", "A", "A", "B", "B", "B", "B", "B", "B", "C", "C", "C", "C", "C", "C", "C", "D", "D", "D", "D", "D", "D", "D"}
)
),
New Column("rel_event",
Character,
"Nominal",
Set Values(
{"REL_T0", "DROP1", "DROP2", "DROP3", "DROP4", "DROP5", "DROP6", "REL_T0", "DROP1", "DROP2", "DROP3", "DROP4", "DROP5", "REL_T0", "DROP1",
"DROP2", "DROP3", "DROP4", "DROP5", "DROP6", "DROP1", "DROP2", "DROP3", "REL_T0", "DROP4", "DROP5", "DROP6"}
)
),
New Column("metric1",
Numeric,
"Continuous",
Format("Best", 10),
Set Values([1, 2, 3, 5, 7, 10, 13, 2, 4, 5, 6, 8, 11, 1, 2, 4, 5, 5, 9, 12, 1, 2, 4, 5, 5, 9, 12])
),
New Column("metric2",
Numeric,
"Continuous",
Format("Best", 10),
Set Values([2, 3, 4, 7, 9, 14, 18, 2, 3, 4, 7, 9, 14, 2, 5, 7, 8, 9, 11, 13, 2, 5, 7, 8, 9, 11, 13])
)
);
debugcol = dt << New Column("REL_T0_ROW", Numeric, Continuous);
delta1col = dt << New Column("delta1", Numeric, Continuous);
delta2col = dt << New Column("delta2", Numeric, Continuous);
For Each Row(dt,
// find REL_T0 row for current serial_number
start_row = Col Min(If(:rel_event == "REL_T0", Row(), .), :serial_number);
debugcol[Row()] = start_row;
delta1col[Row()] = :metric1 - :metric1[start_row];
delta2col[Row()] = :metric2 - :metric2[start_row];
);

回复:基于 FOR() 循环的(慢速)JSL 代码优化非常感谢 Jarmo!
我将这个新版本尝试用于包含 18000 行和 2 个指标的数据表:
- 旧方法:~5分钟
- 你的方法:<2分钟
这已经是一个巨大的进步了!!!
再次感谢!!
牧师
回复:基于 FOR() 循环的(慢速)JSL 代码优化你好,霍吉,
感谢您的建议,我查看了您与标准化插件相关的帖子。我一定会在未来的项目中牢记这一点。
牧师
回复:基于 FOR() 循环的(慢速)JSL 代码优化使用矩阵来执行这些计算很可能可以很容易地进一步提高速度。此外,还添加
dt << begin data update;
循环之前
dt << end data update;
最有可能的还会提供一些速度改进,特别是如果您的表格没有以不可见/私人的方式打开,因为它会“延迟”,然后表格中的视觉效果就会更新。
回复:基于 FOR() 循环的(慢速)JSL 代码优化测量数据的标准化是数据分析中的一项常见任务。
规范化 GUI提供了许多选项来执行此操作。
在 GUI 中,用户可以指定许多不同的设置 - 并且 AddIn 会生成相应的代码。
生成的公式基于Col...聚合,如Col Min、Col Max、Col Median ...
其他选择:

对于此应用案例,您可以使用“first”(基于ColMin(row())逻辑)
...或者通过“使用子集进行规范化”指定参考[对于参考值不是“第一个”的情况]:

回复:基于 FOR() 循环的(慢速)JSL 代码优化可以(通常)加速的一件事是从循环中删除所有打印,但在这种情况下,不使用那些循环重写它会更快,并且在浏览数据表时不要使用 for 循环(对于每一行都是更好的选择)。
这是一个希望产生更快结果的选项(这也可以变成一个公式,如果需要的话应该还是相当快的)。它假设数据以这样的方式排序,即 REL_T0 始终是每个序列号的第一行,如果不是这样,则需要进行一些小的更改
Names Default To Here(1);
//Create dummy table
RawData = New Table("RawData",
Add Rows(20),
New Column("serial_number",
Character,
set values({"A", "A", "A", "A", "A", "A", "A", "B", "B", "B", "B", "B", "B", "C", "C", "C", "C", "C", "C", "C"})
),
New Column("rel_event",
Character,
set values(
{"REL_T0", "DROP1", "DROP2", "DROP3", "DROP4", "DROP5", "DROP6", "REL_T0", "DROP1", "DROP2", "DROP3", "DROP4", "DROP5", "REL_T0", "DROP1",
"DROP2", "DROP3", "DROP4", "DROP5", "DROP6"}
)
),
New Column("metric1", Continuous, set values({1, 2, 3, 5, 7, 10, 13, 2, 4, 5, 6, 8, 11, 1, 2, 4, 5, 5, 9, 12})),
New Column("metric2", Continuous, set values({2, 3, 4, 7, 9, 14, 18, 2, 3, 4, 7, 9, 14, 2, 5, 7, 8, 9, 11, 13}))
);
delta1col = RawData << New Column("delta1", Numeric, Continuous);
delta2col = RawData << New Column("delta2", Numeric, Continuous);
For Each Row(RawData,
// find REL_T0 row for current serial_number
start_row = Col Min(Row(), :serial_number);
delta1col[Row()] = :metric1 - :metric1[start_row];
delta2col[Row()] = :metric2 - :metric2[start_row];
);
