nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Model Comparison in Formula Depot doesn't show breakout of Training, Validation, and Test

JerryFish

I'm talking to a client in the morning, and need a bit of help. Using JMP Pro 13. Building several models with a Validation column (training, validation, test). I want to compare the models using Model Comparison in Formula Depot. When comparing models, I only see what appears to be a single weighted Rsqd value, rather than Rsqd broken out by Training, Validation, and Test (as in the Building Better Models book).

I must be missing a setting somewhere, but I can't find it. Suggestions?

Accepted answers

jiancao

I have also noticed this shortcoming. After you save the predictions formulas use Model Comparison in Predictive Modeling. It allows comparison by a Group column

All comments

jiancao

I have also noticed this shortcoming. After you save the predictions formulas use Model Comparison in Predictive Modeling. It allows comparison by a Group column

jerry_cooper

If you run Model Comparison from within the Formula Depot, then select the Local Data Filter from the red triangle menu for the Model Comparison outline box, you can select the Validation column and choose to filter the data to the Test rows only. The reported Rsqd's will then be for the Test group. Is that what you are looking for?

JerryFish

Thank you Jian! That works perfectly. I wonder why Publish Prediction Formula doesn't provide this option?

Anyway, I'm now ready for my meeting in the morning. Thank you very much!

Jerry

JerryFish

That works as well! Thanks Jerry!