nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Help with statistical analysis of repeated measurements

bbenny7

Let's say that I have to compare two equipments and check if they measure the same.

I know that I have some measurment error/uncertainty so I decide to measure the same sample several times at each equipment.

The data collection looks like the table Statistical analysis data collection (see attachment). My idea was to take an average for each sample, and then perform a matched-pairs analysis (see attached Statistical analysis matched-pairs). In example, I can see that the two equipments measure diffently.

Statistical analysis matched-pairs - Matched Pairs of A, B.png

However, if I compare the data in the table Statistical analysis data collection with Fit Y-by-X and run a non-parametric test for each sample, I can see that the differences are not statistical significant for all the samples.

Statistical analysis data collection - Fit Y by X of value by equipment.png

Waht is the correct way to analyse these data?

Statistical analysis matched-pairs.jmp

Statistical analysis data collection.jmp

Accepted answers

MRB3855

Hi @bbenny7 : I'd consider a mixed model. The Matched pairs analysis (paired t-test) you carried out is a special case of this mixed model.

In the Fit Model platform, Value is Y, and the effects are Equipment and Sample. Sample should be considered a random effect (via Attribute menu) so that Sample to sample variation is removed and comparisons are made within samples.

And you may want to ask yourself if the difference (statistically significant or not) is meaningful. i.e., the confidence interval may be very tight, and it may not include zero (indicating statistical significant different)...but is that confidence interval a practically meaningful difference? For example, in your matched pairs analysis, the conf int on the difference is -2.41 to -1.15; that is a very small difference. From a practical/scientific point of view, is that difference (somewhere between -2.41 and -1.15) relevant?

Edit: Also, looking at the plots, it appears the data are rounded to 1 dp; it is not advised to analyze rounded data. The rounded data may be perhaps what you report, but for data analysis always use the number of decimal places to the precision of the measurement.

All comments

Ressel

Not a statistician, but this looks like an application for variability analysis (Analyze > Quality and Process > Variability / Attribute Gauge Chart) or MSA (Analyze > Quality and Process > Measurement Systems Analysis)

bbenny7

@Ressel thanks for your answer.

I have tried the Variability/Attribute Gauge Chart, and the result is similar to the matched-pairs.

My concern is that, in this way, I am working with averages, but the limited number of samples and the large spread lead to a large uncertainty in the average estimation.

Is it still fine to work with averages?

Ressel

@bbenny7 I need to repeat my warning that I'm not a statistician, but looking at your more detailed table Statistical analysis data collection.jmp, I am uncertain what the denomination "Sample" means. Are you expecting actual differences between them, and if so do you expect these differences to be due to 1, 2 and 3 having different properties or are these differences perhaps a result of sampling variation?

Looking at the plot below, as a first approximation, would it be enough to assume samples 1, 2 and 3 are identical (or prove it with via ANOVA) and then compare the group means for equipment A and B against each other?

Also, your comment "the large spread lead to a large uncertainty in the average estimation" is already a conclusion that sounds potentially useful for communication to whoever may be interested in the results.

AlbertoJD2000

Good point on the ANOVA approach, if samples 1, 2, and 3 aren’t fundamentally different, pooling them could simplify things a lot. I’m also curious what the expected variation between the samples is meant to reflect.

MRB3855

Hi @bbenny7 : I'd consider a mixed model. The Matched pairs analysis (paired t-test) you carried out is a special case of this mixed model.

bbenny7

Hi @MRB3855 ,

thanks for your answer.

I have one more question about the parameter estimates: I cannot see equipment [B] and the intercept doesn't match the value for equipment [B]. How should I calculate the value for equipment [B]?

MRB3855

Hi @bbenny7 : It's the way the model is parameterized.

-Edit: Details here if interested: https://www.jmp.com/support/help/en/18.0/index.shtml#page/jmp/nominal-factors.shtml#

Intercept is interpreted as the average across both pieces of equipment.

A = intercept + 0.8888889

B = intercept - 0.8888889.

You can see this and comparisons etc if you open the "Effect Details" output.

Xinghua

You should to try "Equivalence Test".