Leave-one-out cross validation

wendytseng

Can someone help me understand the graph that is shown below from Gen Reg's platform for leave-one-out validation (or perhaps help me better understand leave-one-out validation)?  I used Diabetes and am predicting Y (continuous).  Why is one particular record being shown in the validation plot?  It was my understanding that the solution path Validation Plot plots average value across all leave-one-out's.

 

 Data set attached below with script.

Diabetes (wt edited).jmp

Phil_Kay

Hi Wendy,

I think if you look at the help documentation for k-fold it will help to explain this. Leave one out is just a special case of k-fold where k = number of rows of data.

"To obtain the final model, all k models derived for the optimal value of the tuning parameter are fit to the entire data set. Of these, the model that has the highest validation log-likelihood is selected as the final model. The training set used for that final model is designated as the Training set and the holdout fold for that model is the Validation set. These are the Training and Validation sets used in plots and in the reported results for the final solution."

So you are only seeing the plots for the one of the k models: the model with the highest validation log-likelihood.

I agree it is a bit weird to just see the actual vs predicted for a validation set with one data point. But hopefully this explains why now.

Cheers,

Phil