nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Prediction Interval in Profiler-- Why not in GenReg?

JerryFish

Hi All-- It looks like Fit Model/Standard Least Squares profiler now offers Prediction Intervals in the red drop-down (in JMP 15). But if I'm looking at it right, Prediction Intervals are not available in the profiler for GenReg.

Anybody know why this is? Or am I missing something on how to turn them on in GenReg?

Accepted answers

Jason_Wiggins

Hey Jerry,

I had this same question a few years back and reached out to tech support. The gerneral response I got was that the assumptions behind prediction limits in regression are not appopriate in the case of generalized regression. I cannot recall who fielded the question but they were kind enough to share the paper I attached. There is a section showing the computation of prediction error. I found this to be insightful.

I too wonder if there is a gen reg appropriate version of prediction limits such as prediciton error in the paper that could be deployed in the platform. I like being able to see if a new observation falls within prediction limits. This is how I learned to validate a DOE. My quesiton arose because I had several DOE's that gen reg did a much better job at fitting than standard regression and wondered how to validate the model.

I have used a hold out or test set of data to demonstrate how well a gen reg model predicts. This has worked in some cases in my past life and is what i have used in demos. I don't think this is completely helpful if you are validating a model and can only afford a run or two. I am curious to see if anyone else has some insight that is helpful.

Jianqing Fan and Runze Li_2001.pdf

All comments

Jason_Wiggins

Hey Jerry,

Jianqing Fan and Runze Li_2001.pdf

JerryFish

Thanks Jason. I think we had the same DOE teacher in our pasts, as I learned the same way: Once you have a predictive model and find a sweet spot, do a confirmation run. If the confirmation run falls within prediction limits, the model is confirmed.

An option in GenReg for prediction limits would be great, though I defer to the smart statisticians and mathematicians for whether it can be done. From a completely intuitive perspective, it seems like there could be some kind of monte carlo estimate (at least) for these values. Hopefully a developer will take on the challenge!

JerryFish

A bit more info-- Jordan Hiller wasn't able to find Prediction Interval in the profiler, and when I went back I couldn't find it either. After some digging, I was able to find it (in JMP 14 and in 15, Least Squares Only). However, it is elusive, and I haven't figured out why.

If you open the sample data file InjectionMolding.jmp, then go to Analyze/Fit Model, a LogLinear Variance model is fit. Turn on the Profiler, go to the red hotspot, and there is the Prediction Interval option, in all its glory.

What I don't understand is why it hasn't shown up for me in other analyses. I'll keep looking.

martindemel

Hi,

I think one could use fractional weighted bootstrapping and simulation to get a fair statement of something close to prediction interval. This is a quite new apporach (Phil Ramsey and Chris G. are working on it, and I think Mike A. has done an add-in for generating the autovalidation column used in this approach). Another even manual approach could be to jsut bootstrap estimates and then simulate with the model formula and the most extreme estimates of each factor a kind of worst case scenario you could use as prediction interval (hopefully this is also statistically valid enough?). At least you should get some conservative boundaries.

Martin