Adding augmented data points to a data table

Question

Hello JMP community,

I currently am trying to write a script that opens an existing data table which Contains an "X1" column and a "Y1" column with 10 rows. The script then predicts Y1 in a new column, makes a residual column (Predicted Y1 - Y1), and then makes a predicted residual column. From there, I want the script to add a certain amount of "made up" X1/Y1 values to the existing data table, in hopes of getting the process to automate. The following code is what I believe is responsible for adding the data points to the data table. The problem is that the following script doesn't add an accurate list of X1 values (the X1 values is simply every integer between 1-10, and when I run the following code I usually only get integers 1 and 10).

d = DOE( Augment Design, X( :X1 ), Y( :Y1 )/*, {Make Design, Make Table}*/ );
d << Make Design;
dt3 = d << Make Table;
dt2 << concatenate( dt3, Append to first table( 1 ) );

Here is my entire script if it helps any:

// 1. Creates X1 column, and corresponding Y1
Close All( Data Tables, No Save );
sourceDT = Open( "SH2.jmp" );
dt2 = sourceDT << Subset( Columns(), Rows(), Output Table Name( "Fixed.jmp" ) );

// 2. Creates predicted Ystat column, saves formulas into same data table (first iteration)
PredictedY1 = dt2 << Neural(
	Y( :Y1 ),
	X( :X1 ),
	Informative Missing( 0 ),
	Validation Method( "Holdback", 0.3333 ),
	Fit( NTanH( 3 ) )
);
PredictedY1 << Save Profile Formulas;

// 3. Adds residual column
dt2 << New Column( "Residual 1", Formula( Abs( :PredictedY1 - :Y1 ) ) );

// 4. Creates and predicts residual column
PredictedResidual1 = dt2 << Neural(
	Y( :Residual 1 ),
	X( :X1 ),
	Informative Missing( 0 ),
	Validation Method( "Holdback", 0.3333 ),
	Fit( NTanH( 3 ) )
);
PredictedResidual1 << Save Profile Formulas;

// Iteration 2
d = DOE( Augment Design, X( :X1 ), Y( :Y1 )/*, {Make Design, Make Table}*/ );
d << Make Design;
dt3 = d << Make Table;
dt2 << concatenate( dt3, Append to first table( 1 ) );

PredictedY2 = dt2 << Neural(
	Y( :Y1 ),
	X( :X1 ),
	Informative Missing( 0 ),
	Validation Method( "Holdback", 0.3333 ),
	Fit( NTanH( 3 ) )
);
PredictedY2 << Save Profile Formulas;

// 3. Adds residual column
dt2 << New Column( "Residual 2", Formula( Abs( :PredictedY1 2 - :Y1 ) ) );

// 4. Creates and predicts residual column
PredictedResidual2 = dt2 << Neural(
	Y( :Residual 2 ),
	X( :X1 ),
	Informative Missing( 0 ),
	Validation Method( "Holdback", 0.3333 ),
	Fit( NTanH( 3 ) )
);
PredictedResidual2 << Save Profile Formulas;

Column( dt2, "X1" ) << Suppress Formula Eval();
Column( dt2, "Y1" ) << Suppress Formula Eval();
dt2 << Select where( :Residual 1 <= 3 );
dt2 << delete rows;

Thanks,

Kyle

Mark_Bailey · Answer

You can still use either method I suggested: create a second data table and concatenate it or directly add rows to original data table. Sketch of idea: dt2 = New Table( "Augmentation Runs",
Add Rows( 10 ),
New Column( "X1", Numeric, Continuous, Values( J( 10, 1, Random Uniform() ) )
);

dt << Concatenate( dt2 ); Alternatively: For( I = 1, I < 11, I++, dt << Add Rows( { X1 = Random Uniform() } ); );

kcorder · Answer

I apologize for any confusion. You are correct that I want to create a second data table with the levels I determine and concatenate them to the existing rows. The problem is that the second data table creates a table with values explicitly either 1 or 10. I want the second data table to have values randomly distributed between 1 and 10 (which is the range of values that the original data table has). Augment Design may not be the best way to do this, but I am very new to JSL so i'm not sure of how to proceed

Thanks

Mark_Bailey · Answer

I am not quite sure what you want. My interpretation is that you want to append new rows to the previous rows. Augment Design creates new treatments and the Make Table message will output the new combined table. It creates the new rows that are optimal for the specified model and the given rows.

I think that you want to create a second data table with the levels you determine and concatenate them to the existing rows. Alternatively, you can iteratively send the << Add Rows() message to the first data table to append the new levels.