nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

How to record DoE generation time ?

Victor_G

Hi dear JMP Users,

I have a question regarding the recording of the time used for the generation of a DoE. My project is to evaluate the time complexity of different Space-Filling DoE algorithms on different configurations (dimensionality and sample size).
However, when creating a Space-Filling design (or any other Custom DoE), there is no indication of the time needed to compute the design. I tried to check the log but these steps are not captured, so there is no timestamp associated with this generation step.

Does anyone has an idea to record the design generation time ? With or without JSL scripting ?

Thanks in advance !

Accepted answers

Mark_Bailey

Consider using this function instead:

HP Time()

jthi

If you are going to do the DoE with JSL, something like this is what I would do. If you have to do them multiple times with different seeds/settings you could go slightly deeper with the JSL.

Names Default To Here(1);

record_doe = function({doe_expr}, {Default Local},
	wait(0);
	start = Tick Seconds();
	
	d = Eval(doe_expr);
	dt = d << Make Table;
	
	wait(0);
	end = Tick Seconds();
	d << Close Window;
	Close(dt, no save);
	
	return(end - start);
);

designs = Associative Array();
designs["Space Filling MaxPro"] = Expr(DOE(
	Space Filling Design,
	{Add Response( Maximize, "Y", ., ., . ),
	Add Factor( Continuous, -1, 1, "X1", 0 ),
	Add Factor( Continuous, -1, 1, "X2", 0 ),
	Add Factor( Continuous, -1, 1, "X3", 0 ),
	Add Factor( Continuous, -1, 1, "X4", 0 ),
	Add Factor( Continuous, -1, 1, "X5", 0 ),
	Add Factor( Continuous, -1, 1, "X6", 0 ),
	Add Factor( Continuous, -1, 1, "X7", 0 ),
	Add Factor( Continuous, -1, 1, "X8", 0 ),
	Add Factor( Continuous, -1, 1, "X9", 0 ), Set Random Seed( 830486233 ),
	FFF Optimality Criterion( MaxPro ),
	Space Filling Design Type( Fast Flexible Filling, 27 ), Simulate Responses( 0 ),
	Set Run Order( Randomize )}
));

designs["Space Filling Centroid"] = Expr(DOE(
	Space Filling Design,
	{Add Response( Maximize, "Y", ., ., . ),
	Add Factor( Continuous, -1, 1, "X1", 0 ),
	Add Factor( Continuous, -1, 1, "X2", 0 ),
	Add Factor( Continuous, -1, 1, "X3", 0 ),
	Add Factor( Continuous, -1, 1, "X4", 0 ),
	Add Factor( Continuous, -1, 1, "X5", 0 ),
	Add Factor( Continuous, -1, 1, "X6", 0 ),
	Add Factor( Continuous, -1, 1, "X7", 0 ),
	Add Factor( Continuous, -1, 1, "X8", 0 ),
	Add Factor( Continuous, -1, 1, "X9", 0 ), Set Random Seed( 1 ),
	FFF Optimality Criterion( Centroid ),
	Space Filling Design Type( Fast Flexible Filling, 20 ), Simulate Responses( 0 ),
	Set Run Order( Randomize )}
));



design_durations = Associative Array();
For Each({{design_name, design_expr}}, designs,
	design_dur = record_doe(design_expr);
	design_durations[design_name] = design_dur
);

Show(design_durations);

You could also of course save the designs

All comments

jthi

Are you running the design generations manually? If that is the case, you could maybe create simple window with JSL which would take the timestamp when you press a button, run design generation, press button again and maybe extract the DoE settings if necessary. You can also inject scripts to JMP platform buttons but I don't remember if those did happen before or after the button execution happens.

Victor_G

Hi @jthi,

Your suggestion is indeed what recommended me Ryan Lekivetz in our private discussion ; what he suggests was the introduction of Tick Seconds() function in the design script before and after design generation, to count design generation time
An example below (inspired from Use Matrices) :

t1 = Tick Seconds();

DOE(
	Space Filling Design,
	{Add Response( Maximize, "Y", ., ., . ),
	Add Factor( Continuous, -1, 1, "X1", 0 ),
	Add Factor( Continuous, -1, 1, "X2", 0 ),
	Add Factor( Continuous, -1, 1, "X3", 0 ),
	Add Factor( Continuous, -1, 1, "X4", 0 ),
	Add Factor( Continuous, -1, 1, "X5", 0 ),
	Add Factor( Continuous, -1, 1, "X6", 0 ),
	Add Factor( Continuous, -1, 1, "X7", 0 ),
	Add Factor( Continuous, -1, 1, "X8", 0 ),
	Add Factor( Continuous, -1, 1, "X9", 0 ), Set Random Seed( 830486233 ),
	FFF Optimality Criterion( MaxPro ),
	Space Filling Design Type( Fast Flexible Filling, 27 ), Simulate Responses( 0 ),
	Set Run Order( Randomize ), Make Table}
);

t2 = Tick Seconds();

Show( t2 - t1 );

And for this design generation, I can find the value in the log :

t2 - t1 = 2.31666666665114;

So I will use this method manually for my different designs (it shouldn't take too long), if you have any idea on how to automatize this process or make it more simple, you're welcome

Thanks for the interest and recommendation,

Mark_Bailey

Consider using this function instead:

HP Time()

jthi

If you are going to do the DoE with JSL, something like this is what I would do. If you have to do them multiple times with different seeds/settings you could go slightly deeper with the JSL.

Names Default To Here(1);

record_doe = function({doe_expr}, {Default Local},
	wait(0);
	start = Tick Seconds();
	
	d = Eval(doe_expr);
	dt = d << Make Table;
	
	wait(0);
	end = Tick Seconds();
	d << Close Window;
	Close(dt, no save);
	
	return(end - start);
);

designs = Associative Array();
designs["Space Filling MaxPro"] = Expr(DOE(
	Space Filling Design,
	{Add Response( Maximize, "Y", ., ., . ),
	Add Factor( Continuous, -1, 1, "X1", 0 ),
	Add Factor( Continuous, -1, 1, "X2", 0 ),
	Add Factor( Continuous, -1, 1, "X3", 0 ),
	Add Factor( Continuous, -1, 1, "X4", 0 ),
	Add Factor( Continuous, -1, 1, "X5", 0 ),
	Add Factor( Continuous, -1, 1, "X6", 0 ),
	Add Factor( Continuous, -1, 1, "X7", 0 ),
	Add Factor( Continuous, -1, 1, "X8", 0 ),
	Add Factor( Continuous, -1, 1, "X9", 0 ), Set Random Seed( 830486233 ),
	FFF Optimality Criterion( MaxPro ),
	Space Filling Design Type( Fast Flexible Filling, 27 ), Simulate Responses( 0 ),
	Set Run Order( Randomize )}
));

designs["Space Filling Centroid"] = Expr(DOE(
	Space Filling Design,
	{Add Response( Maximize, "Y", ., ., . ),
	Add Factor( Continuous, -1, 1, "X1", 0 ),
	Add Factor( Continuous, -1, 1, "X2", 0 ),
	Add Factor( Continuous, -1, 1, "X3", 0 ),
	Add Factor( Continuous, -1, 1, "X4", 0 ),
	Add Factor( Continuous, -1, 1, "X5", 0 ),
	Add Factor( Continuous, -1, 1, "X6", 0 ),
	Add Factor( Continuous, -1, 1, "X7", 0 ),
	Add Factor( Continuous, -1, 1, "X8", 0 ),
	Add Factor( Continuous, -1, 1, "X9", 0 ), Set Random Seed( 1 ),
	FFF Optimality Criterion( Centroid ),
	Space Filling Design Type( Fast Flexible Filling, 20 ), Simulate Responses( 0 ),
	Set Run Order( Randomize )}
));



design_durations = Associative Array();
For Each({{design_name, design_expr}}, designs,
	design_dur = record_doe(design_expr);
	design_durations[design_name] = design_dur
);

Show(design_durations);

You could also of course save the designs

Victor_G

Thanks a lot for the solution @jthi !
The script sounds good for automating this to few designs, but since I have to run it for 31 designs, it would be a little tedious to add each DOE generation script inside the array. I will continue with the "manual" version, with the upgrade mentioned by @Mark_Bailey to replace Tick Seconds() by HP Time(), as it provides a more precise measurement in microseconds.
The final script for one specific example looks like :

t1 = HP Time();

DOE(
	Space Filling Design,
	{Add Response( Maximize, "Y", ., ., . ),
	Add Factor( Continuous, -1, 1, "X1", 0 ),
	Add Factor( Continuous, -1, 1, "X2", 0 ),
	Add Factor( Continuous, -1, 1, "X3", 0 ),
	Add Factor( Continuous, -1, 1, "X4", 0 ),
	Add Factor( Continuous, -1, 1, "X5", 0 ),
	Add Factor( Continuous, -1, 1, "X6", 0 ),
	Add Factor( Continuous, -1, 1, "X7", 0 ),
	Add Factor( Continuous, -1, 1, "X8", 0 ),
	Add Factor( Continuous, -1, 1, "X9", 0 ), Set Random Seed( 830486233 ),
	FFF Optimality Criterion( MaxPro ),
	Space Filling Design Type( Fast Flexible Filling, 27 ), Simulate Responses( 0 ),
	Set Run Order( Randomize ), Make Table}
);

t2 = HP Time();

Show( t2 - t1 );

Thank you all for the valuable inputs !