nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Atomic Operation

swanthog

Hello.

I am trying to measure the elapsed time of this filter (calculates 10 second moving average for column 'PowerRail'):

...
tStart = HP Time();
new_col << Formula( Summation( i = 1,10,Lag(dt:PowerRail,i))/10);
tStop = HP Time();
...

It appears that instead of progressing linearly, a new thread is invoked and the actual time to complete the operation is not realized.

As it stands, 200uS is measured which is not realistic. The CSV file with the 'PowerRail' column is 11GB in size. 200uS is more likely the time to invoke the new thread.

At scripts end, the JMP table opens and the moving average values are shown in the visible area of the table, but if I scroll down I see that the formula hasn't acutally completed processing the entire column.

Is it possible to run the formula in an atomic operation, i.e., not in a different thread, so that the actual time to complete is measured?

Thanks in advance.

Accepted answers

You're right that the script continues before the formula is actually evaluated; usually that is preferred otherwise your scripts might take forever to run. You can make the script wait using Run Formulas:

Names default to here( 1 );

//Open sample data and create a column to modify
dt = Open("$Sample_Data/iris.jmp");
new_col = dt << New Column("Test",Numeric,"Continuous",Formula(1));

//Set the column formula
tStart = HP Time();
new_col << Formula( for(i=1,i<=100000, i++, i+1) );
tMiddle = HP Time();
print(tMiddle-tStart);

//Wait for the formula to finish calculating
dt << Run Formulas();
tStop = HP Time();
print(tStop-tStart);

All comments

Names default to here( 1 );

//Open sample data and create a column to modify
dt = Open("$Sample_Data/iris.jmp");
new_col = dt << New Column("Test",Numeric,"Continuous",Formula(1));

//Set the column formula
tStart = HP Time();
new_col << Formula( for(i=1,i<=100000, i++, i+1) );
tMiddle = HP Time();
print(tMiddle-tStart);

//Wait for the formula to finish calculating
dt << Run Formulas();
tStop = HP Time();
print(tStop-tStart);

swanthog

ih,

Thank you for your quick response.

Using 'Run Formulas()', time to run the filter is actually 4 hours!

gzmorgan0

This is just an FYI is you are looking at performance interactively. Run the script with the JMP debugger and select the button that looks like a clock on top of a pound sign #. It returns the % of time at the step, or the number of "steps" to complete the calculations. Below is the script, the time differentials and the debugger performance. It appaers the Col Moving Average() function is the fsatest method.

Names default to here(1);

dt = New Table("Test", New Column("random", numeric), New Column("MA1", numeric),
	New Column("MA2", numeric),
	New Column("MA3", numeric),
);
dt << add rows(2000000);
xmat = J(2000000, 2, Random Integer(1,10));

dt:random << Set values(xmat[0,1]);
wait(5);

t1 = HP Time();
dt:MA1 << Formula( Summation( i = 1,10,Lag(dt:random,i))/10, <<Run Formula );
t2 = HP time();

xmat[1::9,2]=empty();
for(j=10, j<=nrow(xmat), j++, xmat[j,2]=Mean(xmat[j-9::j,1]) );
dt:MA2 << set values(xmat[0,2]);
t3 = HP Time();

dt:MA3 << Formula(Col Moving Average(:random, 9,0,1), << Run Formula);

t4 = HP Time();

SHow(t4-t3, t3-t2, t2-t1);

Here are the numeric results:

t4 - t3 = 166; //Using Col Moving Average
t3 - t2 = 2246576; //Using a For loop
t2 - t1 = 1038; //Using the formula of Summation divided by 10

Here is the screen shot of the debugger performance report