Lower Control Limit of Test-Retest Plot in MSA

Question

Hello All,

I had a statistical interpretation question. I had run a few MSA studies and was interested in explaining or interpreting the results of the Test-Retest plot.

The definition that I find on the JMP website is

"Test-Retest Error

Indicates measurement variation or repeatability (also known as within error or pure error)."

In most cases, my data points are within the control limits and when I have a data point outside the upper control limit I believe I understand that that means this operator is measuring with higher variability (lower repeatability) compared to the rest.

However, the one that I am having a bit more trouble explaining to myself is if an operator's measurements are outside the lower limit. Is that a bad thing? Does this not signify that their overall repeatability is quite good, yet it is still marked red. Does it make sense to have a lower limit or am I missing something in the explanation?

Below is an example of my result. The control limits have different width as some of the group sizes are slightly different due to the nature of the experiment.

Any help is appreciated.

jthi · Answer

Those are not control limits (CL) but rather decision limits (DL) Analysis of Means Reports, Example of a Detailed Measurement Systems Analysis

Test-Retest Error Comparison Measurement Systems Analysis Report Options

Shows or hides an Analysis of Means for Variances or Analysis of Means Ranges chart for testing if any of the groups have different test-retest error levels. This chart shows if there are differences in the test-retest error between operators. The Analysis of Mean Ranges chart is displayed when ranges are used for variance components.

AdditiveRange10 · Answer

I guess I was mostly trying to get to a conceptual understanding of the meaning of the test-retest plot. The plot is constructed from the standard deviation of different measurements for each part grouped by any other variable used (in this case operator). Normally, we would expect lower standard deviation to be a desirable thing no? Thus, does having a lower control limit in this context make physical sense?

Or conversely, if a point is below the control limit (between operators) that means that one operator has a different measurement spread compared to the others but again lower standard deviation in measurements is a desirable thing right? In this case however, it is marked red when outside the limits whether on the lower or higher side

statman · Answer

From your graph, it appears you have an inconsistent number of data points within subgroup? It needs to understood whether the sampling is nested, systematic or crossed. It is always better if you can draw the sampling plan like this:

In any case, what was changing when the standard deviation was calculated?  Is data 3 measures of the same foiled sample by the same operator on the same day using the same technique? If so, then the repeatability precision of the measurement equipment is what is affecting the standard deviation.  While there is insufficient data to draw any conclusion, the data suggests there is evidence the repeatability of the equipment may be unstable.