Iteration Questions: Data Filtering, and File Saving-Naming<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />Hello,
I am wondering how to setup a code that iteratively runs an analysis according to a specific data filter, and then repeat that analysis for a new data filter. The data filter is set to exclude rows based on the presence of a 1 or 0 in a column, and the columns are in sequential order. Here is an example script:
dt = Open("Hyperspectral_DataCleanedOrdered.jmp");
dt << Data Filter(
Mode(Include(1)),
Inverse,
Add Filter(
Columns(Rand001),
Where(:Rand001 >= 0.875 & :Rand001 <= 1)
),
);
doDiscriminant =
Expr(
da = dt << Discriminant(
X( :Site Type ),
Y(
:
:Name( "366" ),
:Name( "367" ),
:Name( "368" ),
:Name( "369" ),
:Name( "370" ),
:Name( "371" ),
:Name( "372" ),
:Name( "373" ),
:Name( "374" ),
:Name( "375" ),
:Name( "376" ),
:Name( "377" ),
:Name( "378" ),
:Name( "379" ),
:Name( "380" ),
:Name( "381" ),
:Name( "382" ),
)
);
for(i=1, i<=10, i++,
doDiscriminant;
da << StepwiseVariableSelection;
for(k=1, k<=i, k++, da << stepForward);
da << applyThisModel;
daRep = da << Report;
daRep["Score Summaries"]<<select;
daRep["Score Summaries"] << Journal("test1");
Current Journal() << Append(text Box("Test1"));
daRep << closeWindow;
);
Current Journal() << Save Journal ("Rand001") [Ideally, this name updates with each iteration]
The discriminant analysis is on loop for a certain amount of stepwise variable selection iterations (10): What I want is for each 1-10 iteration of the discriminant analysis to occur for each iteration of a data filter. The data filter is set to read a column (Rand001, Rand002,...Rand100), and than exclude specific rows based on the content of the column.
Also, I wasn't able to find a simple way to save journals/text files iteratively with a name that updates per-iteration, so that is another piece of useful advice here.
Thank you!
Re: Iteration Questions: Data Filtering, and File Saving-NamingNot sure what you aim at in the end, but this is one way to iteratively update the data filter:
dt = Open("$SAMPLE_DATA\big class.jmp");
dt << New Column("Rand001", Formula(Random Uniform(0, 1)));
dt << New Column("Rand002", Formula(Random Uniform(0, 1)));
dt << New Column("Rand003", Formula(Random Uniform(0, 1)));
// "Blank" data filter object
df = dt << Data Filter(Mode(Include(1)), Inverse);
For(i = 1, i <= 3, i++,
col = Column("Rand00" || Char(i)); // Dynamic col variable
df << delete all; //Remove old columns from df
df << add filter(Columns(col), Where(col > 0.875));// Add current column
Wait(2); //Delay just for demo purposes
);
But the approach of Vince above, to include a where clause in the analysis platform instead of using a data filter may make more sense here.
Re: Iteration Questions: Data Filtering, and File Saving-NamingVince,
Thank you for this information, I understand that my current method likely isn't the most efficient. If the Where segment of your suggested code will indeed exclude rows with cell-values of 1 for the respected Rand column, what is the syntax? This will solve my problem!
Re: Iteration Questions: Data Filtering, and File Saving-NamingI guess then what's the purpose of excluding? If you're just excluding so you get a different graph every time, you can do that with the Where.
other than that, you shouldn't column scope col.
:col == {1} // won't work
col == 1 // will work
Re: Iteration Questions: Data Filtering, and File Saving-NamingThank you for that help, it has given me some insight on my next steps. Though, what I was looking for was to iteratively apply a data filter, not iteratively run the DA. Example, here is a sample code:
dt = Current Data Table();
col= Column("Rand00"||char(i));
dt = open("$SAMPLE_DATA\big class.jmp");
dt<<New Column("Rand001", Formula(Random Integer(0,1)));
dt<<New Column("Rand002", Formula(Random Integer(0,1)));
dt<<New Column("Rand003", Formula(Random Integer(0,1)));
doDataFilter =
col= Column("Rand00"||char(i));
dt << Data Filter(
Mode(Include(1)),
Inverse,
Add Filter(
Columns("Rand00"||char(i)),
Where(:col == {1})
//Where(:"Rand00"||char(i) > 0.875 & :"Rand00"||char(i) =1)
),
);
for(i=1, i<=3, i++, doDataFilter;
col= Column("Rand00"||char(i)););
This code reads the error:
The WHERE clause has not been correctly specified. in access or evaluation of 'Equal' , :col == /*###*/{1} /*###*/
How can I write the WHERE clause to include the 'i' character, so that the filter moves along the designated rows with each iteration?
Re: Iteration Questions: Data Filtering, and File Saving-NamingI don't think I answered all of your questions but this might put you on the right track.
dt = open("$SAMPLE_DATA\big class.jmp");
dt<<New Column("Rand001", Formula(Random Integer(0,1)));
dt<<New Column("Rand002", Formula(Random Integer(0,1)));
dt<<New Column("Rand003", Formula(Random Integer(0,1)));
for(i = 1, i<=3, i++,
col = Column("Rand00"||char(i));
da = dt << Discriminant(
X( :sex ),
Y( :height ),
Where(col[row()] == 1),
Show Classification Counts( 1 ),
Biplot Ray Position( [0, 15.5914608488282, 1.5] )
);
da << Title("Rand00"||char(i));
da << Redraw;
da << Journal;
da << Close Window();
)
I would use either the where statement or the local data filter.
hope it's at least somewhat useful.
