Dear colleagues,
I used the deprecated SE mail list to ask this question:
a customer wants to compare two assay methods. They have different samples for the methods and every sample is measured three times. The question of interest is:
“Are the variances of the three repetitions larger for one method than for the other?”
So the goal is to test two distributions of variances. I didn’t find any platform in JMP that would do such a test and therefore I used Tabulate and the F distribution function to summarize the data and calculate a F-test. The attached journal has the data, scripts and an explanation of what I did.
Do I miss something in JMP?
Are my thoughts correct?
Did I do the proper steps and calculations?
I sent it along with the journal attached.
I got many answers that I hereby compile and comment. Thank you everybody for the contributions.
Variability chart / EMS and Mixed Model platforms
Both platforms give options to calculate the (average) variances that I’m interested in, but there is no option to run an F-test between the groups. For all platforms you use the original Assay Data set. In the Variability / Attribute Gauge Chart you set “Method” as X, Grouping, “ID” as Part, Sample ID and the model type set to “Nested”. The option “Heterogeneity of Variance Tests” performs the desired calculations. The squared standard deviations give exactly the F-statistics that I calculated using the Summary function. Also the EMS platform with a “nested” model looks at the global effects of ID and method and compares the variation of the measurements and not the variation of the variation. The mixed model with “Method” as fixed effect and as “Repeated”, “Unequal Variances” as structure, “ID” as random effect and “No intercept” gives identical results in the “Repeated Effects Covariance Parameter Estimates” outline. Again the (mean) variances can’t be tested against each other.
Fit Y-by-X
This platform puts all measurements per method into one bucket. So the equal variances test is based upon the squared deviations of all measurements within one method from their respective group mean whereas I’m interested in the squared deviations from the means of the individual tests (per ID). I could run ID as a grouping variable but then I wouldn’t get any information about differences between the two methods.
Method Comparison Addin
https://community.jmp.com/t5/JMP-Add-Ins/Method-Comparison/ta-p/21520
This addin combines some of the aspects that already have been discussed before and others that require data in a different structure than the one available.
Conclusion
The way the customer wants to analyze the data is not wrong from a methodological point of view. The way I did the calculations is correct as well. There remain some weaknesses in the analysis as such but that is another discussion.
