The other day, we held a web seminar (*) for customers in the manufacturing industry to explain statistical tests and usage examples using JMP. I explained the necessity and concept of the verification method using an example in quality control, but at the end, I explained " Test of equivalence Many customers were interested in this.
* An on-demand version (Japanese) of this seminar is available. Please refer to the end of this blog.
In a survey taken after the seminar, some respondents said that "in practice, equivalence tests are performed in most cases," confirming that there is a need in the manufacturing industry, where there are many cases where it is necessary to demonstrate equivalence. Ta.
Image of equivalence
Example on the left: The current process does not deviate from the target value (almost the same)
Example on the right: The weight of raw materials produced in two facilities can be considered equal.
The seminar featured an example of equivalence testing for the average of two samples, which shows that the characteristic values (outer diameter) of medical tubes manufactured by two manufacturing facilities are equivalent. In this blog series, we will focus on the following cases that we were not able to introduce at the seminar, and also as a follow-up to the equivalence test.
Part 1. Equivalence of one sample means (this blog post)
Part 2. Equivalence of two sample proportions (next blog post)
Example of one sample mean
Consider the following example:
- A chemical manufacturer previously delivered vinyl products (parts) from manufacturing company A, but due to cost considerations, they changed the manufacturing company to B.
- The average tensile strength of vinyl products manufactured by manufacturing company A was 100.0 (N/cm), and variations were suppressed.
⇒ We would like to show that the tensile strength of vinyl products manufactured by Manufacturer B is equivalent to that of Manufacturer A.
The results of measuring the tensile strength of 10 randomly selected samples of vinyl products from Manufacturer B are as follows.
99.6, 101.0, 102.3, 98.1, 100.9, 101.5, 101.4, 97.5, 100.7, 100.8
In tests of equivalence, analysts can determine in advance that if the difference is this much, they are equivalent. Tolerance range (margin) need to be decided. Here, the tolerance range is set to 1, and "anything within the range of 100±1 can be considered equivalent."
In reality, we calculate the average for the samples taken from manufacturing company B, and if the (two-sided) 90% confidence interval for the average falls within the tolerance range, we can consider them to be equivalent.
To perform a one-sample mean equality test in JMP, [Univariate distribution] Specify the continuous scale column in which you entered data as [Y] and display the report.
We can see from the report that the average of 10 vinyl products is 100.38. This is about 0.38 larger than the average of 100.0 for manufacturing company A, but we will examine whether it can be said that they are equivalent, taking into account the dispersion of the data.
Click the red triangle button to the left of the report "Tensile Strength (N/cm)" [Equivalence test] If you select , enter the target value as 100 and the difference (margin) that is considered to be effectively 0 as 1, the following report will be displayed.
Let's take a look at the "equivalence test" report. Various figures and statistics are displayed, but a conclusion is written at the end. In this example, we can conclude that the average is equal to 100. In other words, it was possible to say that ``the tensile strength of vinyl products manufactured by Manufacturer B can be considered to be equivalent to those manufactured by Manufacturer A.''
This chemical company is satisfied because it can reduce costs and produce equivalent vinyl products.
Equivalence testing method
In JMP, the equivalence testing method is Two One-Sided Test method (TOST method) is used. Two one-sided tests refer to tests for hypotheses ① and ② below.
① Null hypothesis: μ≦99, alternative hypothesis: μ>99
② Null hypothesis: μ≧101, alternative hypothesis: μ<101
If the null hypothesis is rejected in both tests (1) and (2) and the alternative hypothesis is supported, it can be said that the average is within the range of 99 to 101.
When we actually perform a one-sided test for these hypotheses, the p-value for ① is 0.0008 and the p-value for ② is 0.0380. Since there is a significant difference in both cases when the significance level is set to 0.05, both ① and ② support the alternative hypothesis, that is, they are equivalent (the average value is practically equal to the hypothesized value).
The first graph displayed is a clear illustration of the equivalence test. The range of equivalence (blue area) includes the 90% confidence interval for the mean. The key idea is that if the confidence interval for the average is included within the tolerance range, it can be considered equivalent.
Examples that cannot be said to be equivalent
In the example above, the conclusion was that "equivalence can be said," but let's also give an example where it cannot be said. In the previous example, the following tensile strength measurement results were obtained.
99.6, 101, 102.3, 98.1, 100.9, 101.5, 101.4, 97.5, 100.7, 100.8
As before, there are 10 samples, and the average value is 100.38 as before. However, the dispersion is larger than before, and the standard deviation is 1.53.
In this case, if you perform an equivalence test with the same hypothesized means and margins, you will find that the 90% confidence intervals for the means are not completely within the range of equivalence. The conclusion displayed at the end of the report also states that ``the population mean is not equal to 100'' cannot be rejected. In this case, they cannot be considered equivalent.
From this, Equivalence is determined by taking into account the variation in data. You'll see that.
How many samples are needed in terms of power?
Finally, I will briefly explain how to determine the required number of samples from the perspective of detection power.
In this example, we took 10 samples and tested the equivalence, but is 10 samples appropriate? Wouldn't it be better to have more? One way to answer this question is by determining the number of samples obtained from power calculations.
From the JMP menu bar Design of Experiments > Sample Size Explorer > Power > Power for Equality of One Sample Means You can calculate the number of samples needed for this example. Typically, you would use this feature to estimate how many samples to take before performing an equivalence test.
However, there are some assumptions that the analyst must make in advance.
Standard deviation (σ) : Standard deviation (estimated from previous similar exams, specific knowledge, etc.)
upper margin, lower margin : Assumed tolerance range of equivalence
detection power : Probability of correctly detecting equivalence. Usually, a detection power of 80% or higher is desirable.
For example, if you specify a standard deviation of 1, a margin of 1, and find the number of samples for which the power exceeds 80% for the first time (denoted as the "sample size"), it will be 11.
In this example, if we assume that the standard deviation is 1 in advance, we would need to take about 11 samples.
What if we assume a rather large standard deviation of 1.5? In this case, about 21 samples are required. Since there is a large variation in the data, if we are to say equivalence in that state, we would need about 20.
by Naohiro Masukawa (JMP Japan)
Reference: On-demand seminar
“Overview and usage examples of statistical testing methods using JMP for the manufacturing industry” on demand (Japanese)
