Multiplicity for Fit ModelHow does JMP handle statistical multiplicity when running Fit Model?
Advanced Statistical ModelingBasic Data Analysis and ModelingRe: Multiplicity for Fit ModelOK, I'll bite. What is "statistical multiplicity?" I've never heard the term before.
Re: Multiplicity for Fit ModelHi dlehman1:
So, statistical multiplicity is the essentially type I error or the probability of false positive. For my analysis used fit model to run odds ratios, but I need to know how it accounts for multiple comparisons/multiplicity because my dad is clinical trials and it has multiple subgroups. Below is the definition I found for multiplicity online via google:
"Multiplicity is a major consideration in the analysis of clinical trials. It occurs when multiple significance tests are carried out, increasing the family-wise error rate (FWER), the probability of a “false positive” statistically significant result or type 1 error."
Re: Multiplicity for Fit Model*dad* I meant to write data
Re: Multiplicity for Fit ModelHow did you get to the response mean tab from fit model output?
What I did was go to fit model under analyze, ran it and in the output selected wald test and odds ratio. On that screen is also the FDR logsworth, but I am curious to understand how you got to the screens you provided in the screen shot please.
Thank you,
Nisha
Re: Multiplicity for Fit ModelHi @NishaKumar In my example, my response was “Mean g/cat/day…”. Yours will be different.
If your response is not continuous, the options via top red triangle will be different. When I posted that I hadn’t realized your response is not continuous (I assume your response is not continuous since you are looking at odds ratios).
Re: Multiplicity for Fit ModelYes, my response is not continuous.
Re: Multiplicity for Fit ModelMultiple comparisons is a term I am familiar with, though I never do such adjustments and I'm not aware of built in capabilities for that in JMP (I certainly could be wrong about that and someone will correct me if I am). The most familiar correction is the Bonferroni correction which I'm not aware of in JMP. But it is easy enough to do by hand I think. However, I don't think it is a good idea to lower p values to account for the multiple comparisons - it does nothing to correct for dichotomous thinking that p values invite. How large is the effect size, what is the confidence interval, how many model assumptions have been made, etc. are all critical for interpreting the results of a study. When you just lower the p value, you just continue thinking that the result of a study is whether or not an effect is real, rather than the potential sizes and uncertainties about those effects.
To paraphrase Tufte's comment on pie charts (the only thing worse than one pie chart is more than one pie chart), the only thing worse than one p value is more than one p value.
Perhaps I've overstated things. But the real issue you are referring to (I believe) is subgroup analysis. Usually in RCTs, subgroup analysis is frowned upon unless it is part of the pre-registered study - in which case sampling issues have been thought of to begin with. I think a less mechanistic approach than Bonferroni is best - if you are thinking of p=.05 as denoting a strong enough effect to "matter" then if you do 20 subgroup analyses you should expect one of these to mislead you on average. In reality, you can probably expect more than one in 20 to mislead you.
