325417_en-US

liliconcom

Fit Least Squares - Plot Residuals by Normal Quantiles Improvements

I don't have a lot of faith in the 'plot residuals by normal quantiles' for doing model diagnostics. I have typically saved the residuals from the model to a new column, then used the Distribution platform to create a normal quantile plot there.

 

The fit of the solid red line does not seem to match up well with where the residuals fall using Fit Least Squares. The slope of the solid red line is not a good fit. I looked at a few sample data sets and saw issues with the default model scripts in these data sets:

 

2x3x4 factorial

Bounce data

Wine data

 

I had this as a JMP ticket 2 years ago, but with no real resolution.

Statistics and AnalyticsRe: Fit Least Squares - Plot Residuals by Normal Quantiles Improvements

Great find, Kira!

It looks like Generalized Regression mirrors Fit Least Squares as well, but does its for standardized residuals and without the confidence limits. So that's another example where results seem to differ across modeling platforms.

Re: Fit Least Squares - Plot Residuals by Normal Quantiles Improvements

Hi There!
I'd also prefer seeing the standard deviation instead of the RMSE used in the normal quantile plot of the Fit Model output. 

The reason is that if you'd use the 'Mixed Model' personality instead of the 'standard least squares' personality the residual quantile plot would also use the standard deviation, not the RMSE for plotting the line (even if there are no random effects included in the model). So for consistency between platforms / personalities it would be favorable to use one of the options (preferably the standard deviation) only.

Best,
Kira 

Re: Fit Least Squares - Plot Residuals by Normal Quantiles Improvements - Status changed to: Open for Voting

Thanks, @jszarka. We're tracking the request internally but I don't have any update.

Re: Fit Least Squares - Plot Residuals by Normal Quantiles Improvements

Any updates for this thread?

Re: Fit Least Squares - Plot Residuals by Normal Quantiles Improvements

I would like an option to have the standard deviation of the residuals (stdev) be how the graph is created rather than the RMSE. I would bet that if I went through 10 JMP summary DOE data sets that are 'good', many of them would fail the fat pen test when using the RMSE as shown in my graphic above.

 

(Sorry for the delayed response - I was out on leave, not uninterested in this topic!)

Re: Fit Least Squares - Plot Residuals by Normal Quantiles Improvements - Status changed to: Needs Info

@jszarka, can you help clarify based on the question above?

Re: Fit Least Squares - Plot Residuals by Normal Quantiles Improvements

In the Distribution platform, you can replicate the diagonal reference line with the "Customize..." option and adding a new script:

 

Y Function( StdDev * (Normal Quantile( x )), x );

 

Where “StdDev” is the standard deviation of the residuals. In the Fit Model platform, the RMSE is used as an estimate for the standard deviation:

 

Y Function( RMSE * (Normal Quantile( x )), x );

 

Are you requesting that the Fit Model platform provide an option to use the calculated standard error of the residuals from the fitted model to construct the reference line?

Re: Fit Least Squares - Plot Residuals by Normal Quantiles Improvements

Here's an example from the 2x3x4 factorial data. You can see on the left that all dots are 'above' the solid red line, whereas on the right all dots are 'below' the solid red line.