How to generate a Learning Curve Plot for a machine learning method?

UserLightNew

Dear JMP users,

 

Is it possible to generate a plot of learning curve such as shown below? Both scripting answer or interactive answer would be greatly appreciated.

 

For example, for a machine learning method (e.g. linear regression), I would like to plot error rate in the Y-axis and training set size in the X-axis.

(figure source: http://blog.revolutionanalytics.com/2015/09/why-big-data-learning-curves.html) 

Also, is it possible to plot such learning curve for multiple models (e.g., different regression models) in the same plot?

 

 

Mark_Bailey

Yes, it is possible to generate this 'learning curve' with JMP. Scripting this simulation would probably be the best since the sample size is the simulation parameter. You might be able to use a formula with the simulation feature in JMP Pro.

Yes, you should be able to repeat this simulation with any modeling approach. You just capture a measure of the error for the training set and the validation set.

The blog post provides the R script, so you could use it, as is, if you have R installed on the same computer as JMP. A short JMP script is necessary to run R with this script.