building multiple neural network modelsI am using the neural platform to make predictions, and, being new to neural networking, I am only slightly familiar with all the input parameters: number of hidden layers, number of nodes/hidden layer, boosting models, learning rate, and tours. What I want to do is try to minimize RMSE and the validation model misclassification rate. What I've been doing is iteratively changing each parameter one by one, saving the model performance parameters, and pasting them into a new JMP table, but this is going to take days since there are so many combinations of layers, nodes, tours, etc. Would it be possible to write a script to where JMP Pro builds, say, 1,000 models and dumps the data into a table so that I don't have to manually change each model input parameter?
#Hidden layers: 1 or 2
#Sigmoidal nodes: 0, 1, 2, 3, or 4
#Linear nodes: 0, 1, 2, 3, or 4
#Radial nodes: 0, 1, 2, 3, or 4
#boosting models: no idea, 1-5 maybe?
#learning rate: 0-0.5 in 0.1 increments
#tours: 1, 5, 10, 20, 100
So that would be 2 x 5 x 5 x 5 x 5 x 5 x 5=~30,000.
I guess I could widdle a few of these down once I am more familiar with the dataset. Of course, having many nodes and a high number of boosting models doesn't make sense (nor do certain other combinations), but we're still talking about potentially hundreds of models worth testing. Surely this sort of model screening/comparing could be scripted, right?
AnalyticsScriptingRe: building multiple neural network modelsYou could design an experiment with fewer levels to fit a second-order model and then use the model to optimize the settings. Also, you can make each model and then extract all the results at once by right-clicking on one of the results table and selecting Make Into Combined Table.
Re: building multiple neural network modelsMark, Thanks for your suggestions. The second one I've tried and failed. I know the "make into combined table" option from other platforms, but maybe it was left out of the neural platform for some reason. Strangely, the "option" trick, whereby you hold down the option key, make a selection, and it applies to all parts of the report, does not work either in the neural platform, or else that could speed things up.
That would be cool if I could have my neural network parameters (# nodes, #tours, etc.) as X's in a DOE, with the goal of minimizing RSME, but it seems to not be possible because of the way you must enter the parameters in the particular boxes in the neural model launch. You'd have to do some really advanced scripting, and even then I'm not sure if DOE and Neural could communicate properly. But if anyone out there knows of a way in which I could get the neural platform to basically run the models reflected in this screenshot (being a sub-selection of the grand total), I would greatly appreciate it. In other words, the values in the first screenshot table would need to go to the appropriate input boxes in the second screenshot (i.e., the neural model launch).
Re: building multiple neural network modelsBTW, the "option" trick to get all relevant features to show the same information as the feature you are actively selecting is actually the "command" trick on a Mac (hence why that didn't previously work).
Re: building multiple neural network modelsHi @abmayfield ,
This was something that I've been wanting to script up for a while and was really fun, and your post gave the fire to get it going, thanks! I've been wanting to have a way to tune the NN platform for a while as it is very sensitive to the initial conditions, and it's rather slow to do it manually.
So, I adapted some scripts that I have made for other model platforms to automate their tuning processes, for example, the boosted tree, bootstrap forest, and XGBoost platforms. The scripts are meant to be saved into your data table and run with the green hot button.
Hopefully you know some about scripting and will know where to adapt these scripts to your specific situations/data structure/column names, etc. I have entered in some comments to try and help explain what I'm doing in each larger group. I'm attaching two scripts -- one for continuous Y's and one for Nominal/Ordinal Y's. I did have to include a test to see if the output Y is nominal or ordinal because the Number Col Box() value changes depending on this. I used the Big Class Families.jmp data table in the sample data as a test data set. I'm also including two output data tables I generate using either the numerical NN tuning or nominal/ordinal NN tuning. I did nominal on :sex using :age, :weight:, :height, and then I changed :age from Ordinal to Continuous and modeled age with :weight and :sex for the continuous NN tuning. The fits and stats are horrible, but it's intended only for the purpose of seeing if the JSL works or not. I also added a :Validation column to the data table by stratifying on :age, just to check that it all worked correctly.
I'm sure someone else could script it up more elegantly to test the Y() input column to be modeling type Continuous, Nominal, or Ordinal and then have the appropriate Number Col Box() value inserted to extract the data appropriately using only a single JSL code.
You will need to edit the Y() and X() inputs to the dt << Nerual() section as well as the part where you get the name of the report window, as it's named after the variable you're modeling.
The NN tuning tables could be treating like a space filling DOE in principle where the lower and upper settings are whatever you want to set it to, except for some of them, like N_Layers can only be 1 or 2 because JMP only allows 1 or 2 layers. Penalty_Method is only the options that are valid in the NN platform, and the Transform Covariate and Robust Fit only take on values of 0 or 1 (off or on). I'm including the NN tuning table as well.
If you have any questions, let me know.
Here's my code for the Nominal/Ordinal modeling type:
Names Default To Here( 1 );
dt = Data Table( "Big Class Families" );//enter the name of the current data table
dt_parms = Data Table( "NN Tuning" );//enter name of parameter data table for tuning the NN
dt_results = dt_parms << Subset( All rows, Selected Columns Only( 0 ) );//this copies the tuning table with all the different rows in it
//these commands create columns to record the fit results
dt_results << New Column( "Generalized R² Training" );
dt_results << New Column( "Generalized R² Validation" );
dt_results << New Column( "Entropy R² Training" );
dt_results << New Column( "Entropy R² Validation" );
dt_results << New Column( "RMSE Training" );
dt_results << New Column( "RMSE Validation" );
dt_results << New Column( "Mean Abs Dev Training" );
dt_results << New Column( "Mean Abs Dev Validation" );
dt_results << New Column( "Misclassification Rate Training" );
dt_results << New Column( "Misclassification Rate Validation" );
dt_results << New Column( "-LogLiklihood Training" );
dt_results << New Column( "-LogLiklihood Validation" );
dt_results << New Column( "Sum Freq Training" );
dt_results << New Column( "Sum Freq Validation" );
//This part just creates a separate modelling progress, delete if you don't want it.
i = 1;
imax = N Row( dt_parms );
dlgStatus = New Window( "Overall Modeling Progress",
V List Box(
Text Box( " Overall Modeling Progress ", <<SET font="" size=""></SET>
One thing that is strange that I don't understand is that I get this warning about the log window I create when I run it for the NN platform, but I've never gotten it from my other platforms. I can't follow where that warning comes from, but it still runs correctly. Maybe someone can fix that part of my code?
Unexpected ",". Perhaps there is a missing ")".
Trying to parse operand for "<<" operator.
Line 23 Column 10: ))►,
As a last note: be careful about the settings as it can take a VERY long time to run the modeling if you have huge numbers of boosts or tours or transfer functions. Just be cautious not to go overboard.
Hope this helps!,
DS
Re: building multiple neural network modelsYes! This is exactly what I want: a table with all the NN input parameters (e.g., number of TanH nodes, number of layers, etc.) that syncs with the NN platform. I have not tried it yet, and I do imagine it will take a while, but I am using a small 20 sample x 90 predictor "practice" dataset, so it might be doable. In reality, I've really only found that the type of activation and the number of nodes (as well as number of tours) actually lower my RMSE, with boosting not helping very much. But even then, this will be much faster than doing all ~500 models I planned to run (not nearly as many as I had intended to run yesterday thankfully since I want to stay under 4 nodes/activation type). I will try it out tomorrow and let everyone know how it goes.
Re: building multiple neural network modelsWell, I failed, but I think it's probably because of some tiny scripting mistake. I used your NN tuning table with my data (attached) and made all the necessary updates (I used three training+validation+test samples). If you run the last script ("neural network comparison"), it will build the new table, but no analyses are run. I just included a few X terms, but it will actually be all 86 of the columns in the "standardized data" group. When I use the debugger, it says something is going wrong with the
"For( i = 1, i <= imax, i++," area. I am hoping/guessing a more competent script writer can probably find the issue in seconds!
Re: building multiple neural network modelsHi @abmayfield ,
Thanks for sharing the data table, it helped to debug the changes you made in the script.
A few thing I noticed:
- You needed to add additional columns for the "Test" data set statistics you want to record.
- You accidentally removed the definition for imax, which is the number of rows of the parameter table
- You also accidentally changed the call to the NN platform where in the script it says dt << Neural(), this is where it sends the command to perform the neural net modeling on the data table "dt". You had changed it to something else.
- The definition of the window "w" needs to have the full name of the NN window, and if you're only running a single X, then apparently JMP puts that in as part of the window name, so it needs to be in both the GetName command and the IF statement below it. But that's only true when modeling with a single X. If using more than one, then you don't need to have the "by......" in the window name or IF statement.
- The last part is to make sure that the stats for the Training, Validation, and Test data are correctly called in the Number Col Box() command.
The below code works on your data table (also attached with modified script). It works with the NN Tuning table I gave earlier.
If you are really going to use all 86 columns, you need to put them in manually in the X(col1, col2,...) part of the script. You should then also edit the GetName part because it won't include all 86 names, it'll just be the window name with the response column.
Names Default To Here( 1 );
dt = Current Data Table();
dt_parms = Data Table( "NN Tuning" );
dt_results = dt_parms << Subset( All rows, Selected Columns Only( 0 ) );
dt_results << New Column( "Generalized R² Training" );
dt_results << New Column( "Generalized R² Validation" );
dt_results << New Column( "Generalized R² Test" );
dt_results << New Column( "Entropy R² Training" );
dt_results << New Column( "Entropy R² Validation" );
dt_results << New Column( "Entropy R² Test" );
dt_results << New Column( "RMSE Training" );
dt_results << New Column( "RMSE Validation" );
dt_results << New Column( "RMSE Test" );
dt_results << New Column( "Mean Abs Dev Training" );
dt_results << New Column( "Mean Abs Dev Validation" );
dt_results << New Column( "Mean Abs Dev Test" );
dt_results << New Column( "Misclassification Rate Training" );
dt_results << New Column( "Misclassification Rate Validation" );
dt_results << New Column( "Misclassification Rate Test" );
dt_results << New Column( "-LogLiklihood Training" );
dt_results << New Column( "-LogLiklihood Validation" );
dt_results << New Column( "-LogLiklihood Test" );
dt_results << New Column( "Sum Freq Training" );
dt_results << New Column( "Sum Freq Validation" );
dt_results << New Column( "Sum Freq Test" );
imax = N Row( dt_parms );
For( i = 1, i <= imax, i++,
Nlayer = dt_parms:N_Layers[i];
If( dt_results:N_Layers[i] == 1,
dt_results:N_TanH_2[i] = 0;
dt_results:N_Linear_2[i] = 0;
dt_results:N_Gauss_2[i] = 0;
);
If( dt_results:N_Layers[i] == 2,
dt_results:N_Boosts[i] = 0;
dt_results:Learn_Rate[i] = 0.1;
);
NTH1 = dt_results:N_TanH_1[i];
NTH2 = dt_results:N_TanH_2[i];
NL1 = dt_results:N_Linear_1[i];
NL2 = dt_results:N_Linear_2[i];
NG1 = dt_results:N_Gauss_1[i];
NG2 = dt_results:N_Gauss_2[i];
Nboosts = dt_results:N_Boosts[i];
LR = dt_results:Learn_Rate[i];
TCov = dt_results:T_Cov[i];
RFit = dt_results:Robust_Fit[i];
PMethod = dt_results:Penalty_Method[i];
Ntours = dt_results:N_Tours[i];
str = Eval Insert(
"report = (dt << Neural(
Y( :health designation ),
X(
:OFA...22_c0_g2_i1.p1
),
Validation ( :Validation with test ),
Informative Missing(0),
Transform Covariates(^TCov^),
Fit(
NTanH(^NTH1^),
NLinear(^NL1^),
NGaussian(^NG1^),
NTanH2(^NTH2^),
NLinear2(^NL2^),
NGaussian2(^NG2^),
Transform Covariates(^TCov^),
Penalty Method(\!"PMethod\!"),
Number of Tours(^Ntours^),
N Boost(^Nboosts^),
Learning Rate(^LR^)
)),
Go,
invisible
) << Report;"
);
Eval( Parse( str ) );
w = Window( dt << GetName || " - " || "Neural of health designation by OFA...22_c0_g2_i1.p1" );
If( !Is Missing( Regex( w << Get Window Title, "health designation by OFA...22_c0_g2_i1.p1" ) ) == 1,
ncol_box_v = 10;
ncol_box_test =19;
);
T_stats = w[Outline Box( 3 ), Number Col Box( 1 )] << Get;//You might need to adjust the Outline Box () and Number Col Box() values depending on your tree structure
V_stats = w[Outline Box( 3 ), Number Col Box( ncol_box_v )] << Get;//You might need to adjust the Outline Box () and Number Col Box() values depending on your tree structure
Test_stats = w[Outline Box( 3 ), Number Col Box( ncol_box_test )] << Get;
report << Close Window;
dt_results:Generalized R² Training[i] = T_stats[1];
dt_results:Entropy R² Training[i] = T_stats[2];
dt_results:RMSE Training[i] = T_stats[3];
dt_results:Mean Abs Dev Training[i] = T_stats[4];
dt_results:Misclassification Rate Training[i] = T_stats[5];
dt_results:Name( "-LogLiklihood Training" )[i] = T_stats[6];
dt_results:Sum Freq Training[i] = T_stats[7];
dt_results:Generalized R² Validation[i] = V_stats[1];
dt_results:Entropy R² Validation[i] = V_stats[2];
dt_results:RMSE Validation[i] = V_stats[3];
dt_results:Mean Abs Dev Validation[i] = V_stats[4];
dt_results:Misclassification Rate Validation[i] = V_stats[5];
dt_results:Name( "-LogLiklihood Validation" )[i] = V_stats[6];
dt_results:Sum Freq Validation[i] = V_stats[7];
dt_results:Generalized R² Test[i] = Test_stats[1];
dt_results:Entropy R² Test[i] = Test_stats[2];
dt_results:RMSE Test[i] = Test_stats[3];
dt_results:Mean Abs Dev Test[i] = Test_stats[4];
dt_results:Misclassification Rate Test[i] = Test_stats[5];
dt_results:Name( "-LogLiklihood Test" )[i] = Test_stats[6];
dt_results:Sum Freq Test[i] = Test_stats[7];
);
Hope this helps!,
DS
Re: building multiple neural network modelsAha! Thanks so much for re-doing it for me. I clearly have no knowledge of scripting, so I just changed "Neural" to the name of my data table.....which obviously wouldn't work. Once I changed the name of my data table to "Neural," added the Test columns, and made the other changes you recommended, it worked.
Now what I might do is use DOE to make a conditionally factorial design (since, as you noted, you can't boost with multiple hidden layers) and run a few hundred models on my work computer (so as not to overly tax my 8-GB RAM laptop).
I feel like your script needs to become a GUI or add-in (any takers?). JMP Pro 16, which I am beta-testing, has a really great "model screening" tool, but I think it tests across modeling types (e.g., PLS vs. Gen-Reg), not WITHIN the potentially thousands of models that could be built within each modeling type (or else it may never stop running). I have heard of the "simulator," but I think that is doing something different (not generating tons of simulated models), though I could be wrong. Again, thanks so much for your help. This will surely save me DAYS of time!
Re: building multiple neural network modelsHi @abmayfield ,
Glad that it fixed your problem.
With the tree-based modeling (XGBoost, boosted trees, and bootstrap forest), you can easily use the tuning table GUI within JMP to run many models. But, one downside is that it chews up a lot of RAM. The nice thing about running it this way is that you can run 10's of thousands of different parameter settings and it doesn't eat up much ram, just CPU consumption. I've run up to 40k different settings with bootstrap forest (which is slow) and it can take a few days on my work laptop (32 GB RAM and 2GHz CPU).
I am also an early adopter for JMP 16, and the model screening is nice, but as you point out, it's not tuning any of the models, which can be a bit misleading. It just uses the default settings, which aren't always the best settings to create a predictive model. It could at least help you to determine which platforms are worth the time effort to tune further and which aren't worth looking into.
You might want to also consider the SVM platform to model your nominal data as this is supposed to also be a good method for that kind of data.
As for the simulator, that is something different and is meant as a way to test the estimates with different initial conditions (randomization). This gives a distribution of values for the estimate and gives you a confidence interval for the original estimate and you can then decide on whether that model with the given estimates is good or not.
As for the DOE, you could do a factorial or even run as a space filling design with the factors set appropriately, and I think that platform also allows for conditional constraints. Also, you might consider doing a predictor screening or bootstrapping on the predictors to see if you actually need all 89 of them or if you can really only use a smaller subset.
As a last note, since you have a short and wide data table, you might want to consider instead of using a validation column, do a Leave one Out approach for the validation. If you do that, you'll have to edit the script for that.
Good luck!,
DS
Re: building multiple neural network modelsMore excellent suggestions, especially regarding the type of validation. I see that Neural supports holdback, Kfold, or row exclusion (whereby I'm guessing I'd just randomly choose a few rows to leave out). Were I to want to change the script to change the type of validation, would I just change the "Validation ( :Validation with test )," line? And would the held back samples be equivalent in the eyes of JMP as validation samples? If not, I'd need to remove the validation columns (or rename them), and the test columns would have to go, as well.
Re: building multiple neural network modelsHi @abmayfield ,
Kfold is also another great way to validate the short data set. Yes, the row exclusion you would set as a row state, which you could do by selecting the "test" data set and set the row state to "excluded". If you use Kfold or Holdback, JMP uses those for the validation of the data, not as testing of the data. Hopefully you also have a separate data set you can use as the test data and not have it in the same data table as the training and validation data.
I would just use Kfold for validation and then comment out the parts that you don't need to run the script. Below is the edited version with a 5-Kfold validation setting for the NN platform. Should work, it did for me.
Names Default To Here( 1 );
dt = Current Data Table();
dt_parms = Data Table( "NN Tuning" );
dt_results = dt_parms << Subset( All rows, Selected Columns Only( 0 ) );
dt_results << New Column( "Generalized R² Training" );
dt_results << New Column( "Generalized R² Validation" );
//dt_results << New Column( "Generalized R² Test" );
dt_results << New Column( "Entropy R² Training" );
dt_results << New Column( "Entropy R² Validation" );
//dt_results << New Column( "Entropy R² Test" );
dt_results << New Column( "RMSE Training" );
dt_results << New Column( "RMSE Validation" );
//dt_results << New Column( "RMSE Test" );
dt_results << New Column( "Mean Abs Dev Training" );
dt_results << New Column( "Mean Abs Dev Validation" );
//dt_results << New Column( "Mean Abs Dev Test" );
dt_results << New Column( "Misclassification Rate Training" );
dt_results << New Column( "Misclassification Rate Validation" );
//dt_results << New Column( "Misclassification Rate Test" );
dt_results << New Column( "-LogLiklihood Training" );
dt_results << New Column( "-LogLiklihood Validation" );
//dt_results << New Column( "-LogLiklihood Test" );
dt_results << New Column( "Sum Freq Training" );
dt_results << New Column( "Sum Freq Validation" );
//dt_results << New Column( "Sum Freq Test" );
imax = N Row( dt_parms );
For( i = 1, i <= imax, i++,
Nlayer = dt_parms:N_Layers[i];
If( dt_results:N_Layers[i] == 1,
dt_results:N_TanH_2[i] = 0;
dt_results:N_Linear_2[i] = 0;
dt_results:N_Gauss_2[i] = 0;
);
If( dt_results:N_Layers[i] == 2,
dt_results:N_Boosts[i] = 0;
dt_results:Learn_Rate[i] = 0.1;
);
NTH1 = dt_results:N_TanH_1[i];
NTH2 = dt_results:N_TanH_2[i];
NL1 = dt_results:N_Linear_1[i];
NL2 = dt_results:N_Linear_2[i];
NG1 = dt_results:N_Gauss_1[i];
NG2 = dt_results:N_Gauss_2[i];
Nboosts = dt_results:N_Boosts[i];
LR = dt_results:Learn_Rate[i];
TCov = dt_results:T_Cov[i];
RFit = dt_results:Robust_Fit[i];
PMethod = dt_results:Penalty_Method[i];
Ntours = dt_results:N_Tours[i];
str = Eval Insert(
"report = (dt << Neural(
Y( :health designation ),
X(:OFA...22_c0_g2_i1.p1 ),
Validation ( \!"KFold\!", 5 ),
Informative Missing(0),
Transform Covariates(^TCov^),
Fit(
NTanH(^NTH1^),
NLinear(^NL1^),
NGaussian(^NG1^),
NTanH2(^NTH2^),
NLinear2(^NL2^),
NGaussian2(^NG2^),
Transform Covariates(^TCov^),
Penalty Method(\!"PMethod\!"),
Number of Tours(^Ntours^),
N Boost(^Nboosts^),
Learning Rate(^LR^)
)),
Go,
invisible
) << Report;"
);
Eval( Parse( str ) );
w = Window( dt << GetName || " - " || "Neural of health designation by OFA...22_c0_g2_i1.p1" );
If( !Is Missing( Regex( w << Get Window Title, "health designation by OFA...22_c0_g2_i1.p1" ) ) == 1,
ncol_box_v = 10;
//ncol_box_test =19;
);
T_stats = w[Outline Box( 3 ), Number Col Box( 1 )] << Get;//You might need to adjust the Outline Box () and Number Col Box() values depending on your tree structure
V_stats = w[Outline Box( 3 ), Number Col Box( ncol_box_v )] << Get;//You might need to adjust the Outline Box () and Number Col Box() values depending on your tree structure
//Test_stats = w[Outline Box( 3 ), Number Col Box( ncol_box_test )] << Get;
report << Close Window;
dt_results:Generalized R² Training[i] = T_stats[1];
dt_results:Entropy R² Training[i] = T_stats[2];
dt_results:RMSE Training[i] = T_stats[3];
dt_results:Mean Abs Dev Training[i] = T_stats[4];
dt_results:Misclassification Rate Training[i] = T_stats[5];
dt_results:Name( "-LogLiklihood Training" )[i] = T_stats[6];
dt_results:Sum Freq Training[i] = T_stats[7];
dt_results:Generalized R² Validation[i] = V_stats[1];
dt_results:Entropy R² Validation[i] = V_stats[2];
dt_results:RMSE Validation[i] = V_stats[3];
dt_results:Mean Abs Dev Validation[i] = V_stats[4];
dt_results:Misclassification Rate Validation[i] = V_stats[5];
dt_results:Name( "-LogLiklihood Validation" )[i] = V_stats[6];
dt_results:Sum Freq Validation[i] = V_stats[7];
//dt_results:Generalized R² Test[i] = Test_stats[1];
//dt_results:Entropy R² Test[i] = Test_stats[2];
//dt_results:RMSE Test[i] = Test_stats[3];
//dt_results:Mean Abs Dev Test[i] = Test_stats[4];
//dt_results:Misclassification Rate Test[i] = Test_stats[5];
//dt_results:Name( "-LogLiklihood Test" )[i] = Test_stats[6];
//dt_results:Sum Freq Test[i] = Test_stats[7];
);
Good luck!,
DS
Re: building multiple neural network modelsWow, I can't believe that I've run 300 neural network models before lunch thanks to your scripting help. I hope this gets picked up and "kudo'd" as I imagine when more and more people start seeing the value of the NN platform, they will gravitate towards something like this, especially if it's a brand new dataset and you don't really know where to "dive in" with respect to the bewildering amount of NN model input parameters that can be slightly tweaked (sometimes resulting in huge differences even when just increasing, for instance, the learning rate from 0.05 to 0.1). I will play around with the different validation approaches next (validation column vs. holdback vs. KFold). Definitely need to be careful given my (currently) small sample size (hoping to use these preliminary data to get funds to add more "rows" i.e., corals).
Re: building multiple neural network modelsI have a (likely simple) scripting question. Say I want to change the input dt to another one. Shouldn't I just be able to copy and paste the script, and substitute the dt from "Neural" to something else?
str = Eval Insert(
"report = (dt << Neural (
Y( :health designation ),
X(
If I delete the original "Neural" file and substitute it with another table with the exact same information and a different name, the script won't work! Is it necessary to have the exact same data in two data tables, one of which is named "Neural?" Maybe I missed that part!
Re: building multiple neural network modelsHi @abmayfield ,
If you go back and review the script, the second line defines what the data table is to be used during the analysis. The JSL code dt = Current Data Table() defines a local variable "dt" that is then used in the rest of the script instead of having to call the data table by name. Similarly, dt_parms is the local variable that is assigned to the NN tuning table, "NN tuning". Those two tables are really the only things that you feed to the script. The rest is all internal.
If you copy/paste the JSL code into a new data table it should work automatically because you're running it locally in that data table.
You should not modify the call dt << Nerual(), except to define your Y()'s and X()'s, and maybe the Validation() method. The JSL code dt << Nerual() is a command that is telling JMP to perform NN modeling on the data table "dt". Hence, the definition of what dt is at the beginning of the code.
You will also need to modify the w = Window() call to reflect what the title of the NN report window will be, as well as the Regex() below it. Finally, depending on whether or not you actually will have the "test" data set as part of a validation column, then you'll either want to comment out (with the // command at the start of the line) or remove the comment out at the beginning of the code when it creates the Test result columns as well as the Test_stats definition and where the code writes the results to the new output data table.
Other than than that, you shouldn't have to edit much at all.
With regards to your last question, it kind of depends on how you're implementing the code. If you're running it from within the data table, i.e. that it's saved in the script portion of the data table and you're running it via the green hot button, then you would only need to change the Y() and X()'s that you're using as predictors and response to match the column names from the new data table. You will of course need to have the "NN tuning" table open. On the other hand, if you're running this as a separate script, then you might want to change the definition of dt from dt = Current Data Table() to dt = Data Table (" "), where you'd insert the name of the new data table between the quotes.
The "Neural" you're referring to is not a data table but the JMP command to run a Neural Net model on a specified data table. The "<<" command means to perform the action which is to the right of the << on the object to the left. Near the start of the code you can see that with the new column command. Where it says dt_results << New Column(), the code is telling JMP to create a new column for the object dt_results, which is the subset data generated from the dt_parms data table.
Hope this helps!,
DS
Re: building multiple neural network modelsOK, great. This answers a few additional questions I had. Now the only thing I'm struggling with is, when I copy, paste, and run the script in a new data table, I get an outline box error, which you allude to in the script comments. "expected character arg 1 in access or evaluation of outline box." What does that mean?? I've increased and decreased the numbers, but I get the same error message every time, and the script will not run. I don't even know what an outline box is. It's weird that the tree structure is identical. I just changed the file name. The file name is longer. Maybe that means I need a larger outline box?
