341284_en-US

liliconcom

using Validation data on boosted treeI developed a predictive model w boosted tree. I had 30,000 rows of training data. I have 10,000 tows of “validation” data. How do i run the validation data set through the model ( I have over 800 variables in my model, so getting a “formula” was not practical ). Please provide an interactive solution (not script).AnalyticsDOEOtherRe: using Validation data on boosted tree

With the formula all would be easy. Did you try? What happend?

I assume that you use JMP Pro.

Did you try to build a less complex model? Did you try Predictor Screening or other variable reduction techniques? Or other modeling techniques?

To have the best model, one should have compared different techniques/variable sets. 

Re: using Validation data on boosted tree

I don't understand your question.  If you have a validation column and put that in the Validation box, then the model results for the validation data are reported.  I do second the opinion that you might want a simpler model and that you should try different models.  I would add that you might want to set aside 10% of the data as test data, since the validation data is used in building the model and there is a danger of overfitting, particularly with 800 variables in your model.  So, I would suggest breaking your data into training, validation, and test data sets, running a variety of models, and using the Model Comparison feature to see how the models perform on the test data.  This comparison should be automated once you have saved the prediction formulas from all of your models.

Re: using Validation data on boosted treeI will try that.
However, because the model is so large the prediction formula didnt seem like it would even run at all- it seems to stall the program. Lastly, when i do reallt want to run the test data- how do i run it ?? Thank you so much - ksfire1
Re: using Validation data on boosted tree

If you create the validation column using the facility in the Predictive Modeling Platform, then JMP automatically runs your model on the training, validation, and test data.  So, you don't need to do anything further.  Now, your comments about problems getting your model to run are a different matter - it sounds like there are issues with the model,and I suspect that having 800 variables in your model reflects something that needs to be corrected.  I would look for ways to reduce the number of features.

Re: using Validation data on boosted tree

Predictor Screening is a good first step, to understand, which variables are important. See script from Scripting index, to understand how it works.

I first would do a sampling of the original table to reduce number of rows (main menu: Tables->subset …)

With this subset you can try several Things in short time, save the scripts for the promising ideas and apply them to the large dataset later.

 

Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/Boston Housing.jmp" );
obj = dt << Predictor Screening( Y( :chas ), X( :crim, :indus, :nox, :rooms, :age, :distance, :radial ) );