189094_en-US

liliconcom

k-fold cross validation in time series with JMP / JMP PRO

I want to use Bootstrap or Boosted Trees but with a k-fold method as I use it in the partition trees or neural nets. However I only see the option of adding a fix validation column.

 

Since I am working with time series, random samples cannot be used well to fit my models. 

 

In addition to this, I am suspicious with the k-fold implemented in JMP as it always says random sample.

 

 

I am after a sequence like this one (at least):

 

And I think JMP does something like this (please confirm):

 

 

Here are all the options, the best for my case would be TimeSplit.

https://scikit-learn.org/stable/auto_examples/model_selection/plot_cv_indices.html#sphx-glr-auto-examples-model-selection-plot-cv-indices-py

 

Any ideas or workarounds?

 

Re: k-fold cross validation in time series with JMP / JMP PRO

When using Neural Networks and the validation column, the results are a simple extrapolation. Test rows are not used in this interface either.

 

Ideas on how to implement a proper k-fold for either trees or NN?

 

 

 

Re: k-fold cross validation in time series with JMP / JMP PRO

And if the cutpointvalidation method is used as a column to fit trees (recommended for time series in JMP):

 

 

Re: k-fold cross validation in time series with JMP / JMP PRO

Confirmed: In JMP 14, K-fold it is actually a random sampling done k times.