Does Predictor Screening include validation?

Question

Hi all,

I have been asked this by two customers recently and JMP Help suggests that there is no validation in predictor screening (in basic JMP).

Is that correct?

On top, one customer wanted to know if this platform works for timeseries data?

Thankful for any hints and comments!

Best,

Florian

Peter_Hersh · Accepted Answer

@Florian_Vogt ,

You need to have JMP Pro to do true validation.  You can use a workaround by excluding part of the data set, but I think this would be an opportunity to talk JMP Pro if they are interested in validation.  I think cross-validation is typically used for making predictive models and process screening is a tool for variable selection.  I guess I would get some clarification on the reason they want to use this validation column.  In the end, you are just identifying the most important variables with this technique, so I would not imagine cross-validation would be that useful.

As for time series with predictor screening you might consider created lag and multiple lag columns and use those as predictors for your current value.  This would at least give you an idea of which lag is most correlated to your current value.

Let me know if this makes sense or if you have any follow up questions.

Phil_Kay · Accepted Answer

Hi @Florian_Vogt ,

My understanding is that Predictor Screening does not use validation. Certainly not a validation that you have any control over.

So this is a reason for your customers to invest in JMP Pro!

They would also have more control over the model fitting and interpreting the model.

On time series, it depends what they mean by that. But one thing they can do is to create a series of lag variables for their original variable(s) and then use Predictor Screening (or ideally Bootstrap Forest with validation in Pro) to find the time lag that correlates most with the variable of interest. Creating the lag variables is very easy now with "Lag Mulitple..." from the Row functions in New Formula Column. I have a demo that uses this idea if you are interested.

Cheers,

Phil