JMP does not allow for stopping rules.
Currently the platform is simple and interactive, but there are some other ways that we can help (stop the branching).
Use p-values for splits at partition nodes
Starting with JMP 5.1, p-values for splits at partition nodes were introduced.
When you select Maximize Significance, a new column called LogWorth is added to the candidate report.
* In JMP 9 and later, all branching is done this way.
More details about this method are provided in the white paper and related book excerpt: Monte Carlo Calibration of Distributions of Partition Statistics .
Use K-fold cross-validation
Use K-fold cross-validation.
This method randomly assigns all (non-excluded) rows to k groups and computes cross-validation statistics.
Use the extracted sample
Use the held out samples.
The training/validation-testing approach is described by Bishop (1995, p. 372): A training data set is used to build a tree.
The performance of this tree is measured using an independent validation data set, and the tree with the lowest error is selected. This tree is then validated by measuring its performance on a third independent data set called the test set.
C. M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 1995.
Supplement 1
There is no analysis menu in JMP that allows you to directly perform the above ``use extracted samples''.
To do this, select the rows you want to exclude in the data table and click Rows > Exclude/Do Not Exclude to exclude those rows. Selecting rows to exclude is typically done using Rows > Select Rows > Random Selection. This method allows you to specify the percentage or number of rows to randomly select.
*1: Starting with JMP 15, excluded rows are no longer treated as validation data, so this method is no longer available.
Supplement 2
When you use the k-fold cross-validation option or specify a validation data percentage in the launch window, a Run button is added to the partition report screen.
Clicking the Run button automatically branches the analysis until the cross-validation R-squared or the R-squared for the excluded rows no longer improves.
*2: This option can be specified in the JMP preferences.
FAQ # 2098
