Dear JMP Team,
following some ideas for potential improvements in the area of Boosted Trees and Random Forest.
1. General:
a) Enable the option of using out of the bag (oob) error estimates. These estimates allow to determine optimum model complexity, making a validation set obsolete (could be used in this case as true test set). For smaller data sets where splitting the data in training/validation/test sets is not possible oob also is an good options for getting a reasonable impression on expected model performance.
b) Enable plots showing the optimum number of trees, based on e.g. the average mulinomial distance or the average squared error (in fact I am not sure if it is fair to say the data in the graph below are true test data, because they have been used to determine model complexitx, but I like the idea behing the graph)
2. Boosted Trees
Although JMP Random Forest performenace very good in comparison to other sofware packages, JMP Boosted Trees usually is inferior in terms of predictive capability of the models.
a) Enable to define/control subsampling size (stochastic gradient boosting)
In boosting subsampling is from my point of view an essential part of the algorithm, unfortunately not available in JMP. Taking independently drawn subsamples introduces a some randomness into the analysis, which alleviates the risk of overfitting. Since each tree is build on a differnt set of samples these models usually gerneralize quite good (of course number of trees still has to be controlled). Further subsampling enables to use oob error estimates.
b) Enable multilevel categorical responses in boosted trees
Best Regards
Silvio
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />2. a) Control subsampling size (stochastic gradient boosting) has been added to JMP 13.
