nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

P-Value calculation formula in Equivalence Test

IkjuShin

Some customer asked two things about Equivalence test.

They are going to install Antomotive data correlation system linked JMP with VBA(Visual basic for application)

and hope to do equivalence test within two equipments.

Question 1

How is P-Value calculation formula in Equivalence Test in JMP ?

Question 2

Is 'median' available instead of 'mean' in p-value calculation of Equivalence Test ?

Accepted answers

brady_brady

For step 4, highlight the 2 columns of medians and right-click in the column header. Make the selections shown below to create a column of differences.

Create a column for difference in medians

For step 5, run the Distribution platform on the column you created in step 4, and examine the quantiles. For example, we can see that 95% of the bootstrap samples had a difference in medians of -5.18 and -4.89.

So, if we were testing the hypothesis of medians differing by more than 5, for example, we would fail to reject that hypothesis at the 0.05 level because our empirical confidence interval is not completely covered by the interval (-5, 5). That is, we have insufficient evidence, at the 0.05 level, to conclude that the medians of the distributions differ by less than 5.

(In fact, I simulated this data using two uniform distributions: U(0,1) and U(5,6). So, this result is what we would expect.)

Examine distribution of difference in medians

All comments

brady_brady

Hi Ikju,

1) The p-value for the TOST procedure is the higher of the p-values for each 1-sided test. For example, if the practical difference is 4, the 2 tests have these hypotheses:

H0(test 1): d >= 4

H0(test 2): d <= - 4

Suppose the p-value for test 1 is 0.00000003 and the p-value for test 2 is 0.98983. Then the p-value reported for the equivalence test is 0.98983.

Only when BOTH null hypotheses are rejected are the means declared practically equivalent.

More information is here: https://www.jmp.com/support/help/13-2/Example_of_an_Equivalence_Test.shtml

2) As far as I know JMP does not provide a median-based test for equivalence. In theory this should be possible, and I've seen an article on this, but the article cannot be read for free: http://onlinelibrary.wiley.com/doi/10.1002/pst.384/abstract

If the samples were large enough, I suppose you could construct a nonparametric equivalence test by taking bootstrap samples (sampling repeatedly with replacement) from the data, computing the difference in medians in each case, and using the simulation results to form a confidence interval for the difference in medians. If the confidence interval were completely contained by the interval (-k, k), where k is the difference considered practically significant, you could declare the samples practically equivalent.

Cheers,

Brady

brady_brady

I should add that if the user has JMP Pro, it is easy to perform the bootstrapping I've described:

1) Use Fit Y by X to cast the continuous variable into the Y role and the categorical variable into the X role, then run the analysis.

2) Use the red triangle menu to turn on the quantiles report.

3) In the quantiles report, right click on the median column and select Bootstrap.

4) When the bootstrap table appears, use a column transform to create a new column that is the difference in medians.

5) Use the distribution platform on this column to determine the (alpha/2) and (1-alpha/2) quantiles, which form an empirical 100(1-alpha)% confidence interval for the difference in medians.

IkjuShin

Hi Brady,

Thanks for your detailed explanation and it would be better if I can understand much more.

Could you explain a little more for 4) & 5)

I don't know how to manipluate JMP for 4) & 5)

4) When the bootstrap table appears, use a column transform to create a new column that is the difference in medians.

5) Use the distribution platform on this column to determine the (alpha/2) and (1-alpha/2) quantiles, which form an empirical 100(1-alpha)% confidence interval for the difference in medians

Equivalence test-median.pptx

Students.jmp

Students-2.jmp

brady_brady

For step 4, highlight the 2 columns of medians and right-click in the column header. Make the selections shown below to create a column of differences.

Create a column for difference in medians

(In fact, I simulated this data using two uniform distributions: U(0,1) and U(5,6). So, this result is what we would expect.)

Examine distribution of difference in medians