How to interpret the result of random effects model in JMP?

Question

I fitted a random effects model with 2 factors (Part No. and Operator) by selecting the "Random Effects" option under "Attributes" on "Fit Model" window (with interaction Part*Operator).

I received the following output under REML Variance Components Estimates table.

Random Effect

Var Ratio

Var Component

Std Error

95% Lower

95% Upper

Wald p-Value

Pct of Total

Part

1.1555556

1.7333333

0.8656795

0.0366326

3.430034

0.0453*

53.608

Operator

-0.004115

-0.006173

0.0218

-0.0489

0.0365544

0.7771

0.000

Part*Operator

-0.199588

-0.299383

0.1464372

-0.586394

-0.012371

0.0409*

0.000

Residual

1.5

0.3354102

1.0110933

2.4556912

46.392

Total

3.2333333

0.9283863

1.9759148

6.2317667

100.000

-2 LogLikelihood = 203.88510393

How to interpret these results?

statman · Answer

This is not an easy discussion technically, but practically I have some thoughts:

1. We really don't care as much about the actual values of the variance components, but the relative values compared to the others in the study.

2. It is the magnitude of those components that is of most interest (not the sign).  Usually JMP will switch to Bayesian estimates when there are negative REML components.

3. Always be cautious of p-values.. they are a function of comparisons within the data set collected. Whether they have any association with reality (or are useful) is a function of how the data was collected.

4. In addition, unusual data points can have a huge effect on variance component estimates and subsequent quantitative tests.  ALWAYs plot the data and evaluate the data for unusual or special cause data points.

5. Plot the data compare to your predictions...does it make sense?  Can you explain what you see?  Don't turn off engineering or science.

gfirmstone · Answer

I stumbled on this discussion due to trying to understand how to interpret negative variance component estimates with a low Wald p-value (although in my case the negative variance component is for a main effect).  I was just assuming that a low Wald p-value should only be considered truly important if the variance component estimate was positive, if the estimate is negative it doesn't make sense to say it is statistically significant.  But your comment about the negative estimate for the interaction has me confused.  I would interpret the interaction variance component as the variation due to the interaction between parts and operators, so if the estimate is negative I was thinking it would be interpreted as not truly significant, regardless of the Wald p-value.  Thanks.

P_Bartell · Answer

I'll add just a bit to @Mark_Bailey 's explanation by making sure that you ALSO plotted the data using a variety of the graphical techniques available in the Gauge R & R and/or Measurement Systems Analysis platforms and subplatforms. These charts can often times be very informative by visualizing features/characteristics in the measurement system that are of interest and valuable over and above the static tabular results that are also presented. Here's a link to the online documentation for the MSA platform as a good place to start:

https://www.jmp.com/support/help/14-2/measurement-systems-analysis.shtml#

And here's a JMP On Demand Mastering JMP webinar that I hosted during my SAS tenure that you might also find valuable:

https://www.jmp.com/en_us/events/ondemand/mastering-jmp/evaluating-and-monitoring-your-process-using-msa-and-spc.html