Equivalence test

Question

Is there an equivalence test procedure that can be used with categorical data (i.e., both independent variables and dependent variable are categorical)?

A nominal logistic fit for fertility as influenced by hormonal procedure, lactation, and age category shows no stat. sig. influence. I was under the impression that it would be inappropriate to conclude there is no difference between the means unless we employ an equivalence test comparison that statistically validates such claim(?).

Many thanks!

Peter_Bartell · Accepted Answer

Generally speaking it sounds like you may be confusing the two analytic concepts of equivalence testing and hypothesis testing. These are two very different types of decisions. Hypothesis testing never proves anything with certainty. All is allows one to do is either reject of fail to reject the null hypothesis under a certain prescribed set of conditions such as alpha risk, for a given sample size, etc. No where in either of those two conclusions is the notion of equivalence covered. Equivalence testing is a completely different analytics approach.

There are a few different types of equivalence testing out there that are used in different contexts. My colleague @Mark_Bailey for example has written a nice JMP script in this space:

https://community.jmp.com/t5/JMP-Scripts/Test-for-Parameter-Equivalence/ta-p/29375/jump-to/first-unread-message

In addition, often in a biotech setting, fitting nonlinear models to data is common and there are different types of 'equivalence testing' used in this context. These are supported in JMP's Fit Curve platform:

http://www.jmp.com/support/help/13-2/Equivalence_Test_2.shtml

Mark_Bailey · Accepted Answer

As it turns out, JMP provides equivalence testing (under multiple comparisons) in the Generalized Regression platform, if you have JMP Pro. I just want to emphasize Peter's point that hypothesis testing for a difference or for equivalence are very different. Your choice depends entirely on what you are testing for (alternative hypothesis). If your were using nominal logistic regression to test for associations and found none, then you are finished. You don't need to show statistically significant equivalence.

If you are surprised by this result then perhaps it is a matter of the power in the study.