When conducting research, many researchers will think about a question: "Am I taking enough samples?" Because the number of samples will directly affect the rigor of the research and the reliability of the research conclusions. When the number of samples is too small, the research results will be unstable and the risk of obtaining "false negative" results will be high; when the number of samples is too large, the research cost and experimental difficulty will increase.
How to determine whether the sample size of the study is reasonable? Today JMP will introduce how to obtain the appropriate number of samples through Power analysis.
The relationship between sample number and power
Before the study begins, we will need to know how many samples to conduct the study.
When it comes to the number of samples, we have to mention an equally important concept: power, which is called statistical testing power, statistical testing power, and statistical performance analysis. This article refers to it as power. Simply put, power means: if there is indeed a difference between the two groups in practice, the ability to detect this difference with the current number of samples.
Under the condition that other parameters are known, the number of samples and power can be calculated from each other. To calculate the number of samples, power must be set; similarly, if the number of samples is known, the power under the current number of samples can also be calculated. The statistical result in the experiment is negative. If the power is too low at this time, it means that the number of samples is probably insufficient, so the originally different results cannot be obtained.
How to calculate the number of samples with JMP?
The calculation of the number of samples only requires two steps (as simple as putting an elephant in the refrigerator):
The first step is to consult historical documents to obtain the parameters required for sample number calculation, such as mean, standard deviation, rate, etc.
In the second step, enter these parameters into JMP and get the results.
JMP can estimate the number of samples for several common inter-group comparisons in clinical trials, such as estimation of mean comparisons, estimation of rate comparisons, and estimation of the number of samples for equivalence testing. It is simple to operate and highly interactive. The following will focus on how to use JMP for sample number and power calculation.
In JMP, select Experimental Design→Sample size explorers→power (Figure 1). You can see that there are multiple scenarios for calculating power and sample number under the power menu. You can choose the corresponding specific ones according to the research purpose. Menu.
Figure 1 Sample number and power calculation of two groups of independent sample means - menu selection
The two most common situations are introduced below, namely comparison of means of two groups and comparison of rates of two groups.
01 Estimating the number of samples for comparing the means of two groups
Preparation before estimating
To estimate the number of samples for comparing the means of two groups, the following parameters need to be determined in advance:
01 Whether the test type is single-tailed or double-tailed, it is usually double-tailed if there are no special requirements;
Type 02-the upper limit of the error α, usually takes the value 0.05 (two-tailed test) or 0.025 (single-tailed test);
03 type 2 error β or power1-β, it is generally recommended that the power is above 0.80;
04. The mean (or difference between the two groups) and standard deviation of each group. This data can be obtained by reviewing the literature or conducting pre-tests;
05 The ratio of the number of samples between each group is usually 1:1, and sometimes it can be set to 1:2 (depending on the purpose of the research and the actual situation);
06 If it is a follow-up study, the loss to follow-up rate also needs to be considered, which is usually 10% (depending on the actual situation);
07When writing a proposal report or research tender document, the above parameters need to be provided to demonstrate the feasibility and reliability of the research.
The estimated number of samples JMP accomplish
Click the power of the two independent sample means in Figure 1 to enter the sample number and power calculation interface (Figure 2).
Figure 2 Sample number and power calculation of two sets of independent sample means - operation interface
The left side of Figure 2 shows the various parameters involved in the calculation of the number of samples. From top to bottom, they are the test type, fixed parameters (alpha and standard deviation), test parameters (including the difference to be tested, the number of samples and power), and the right side is Sample number estimation curve.
![[TW] JMP community-CTA-downloadback to WB.png](https://us.v-cdn.net/6038925/uploads/images/41096i24F89E6C10D2C2FF/%5BTW%5D%20JMP%20community-CTA-downloadback%20to%20WB.png "[TW] JMP community-CTA-downloadback to WB.png")
In versions before JMP 16, if you want to calculate the number of samples, you need to enter the power; similarly, if you want to calculate the power, you need to enter the number of samples; in the JMP 16 version, the number of samples (or power) can be adjusted at any time, and the corresponding power (or sample number) will change accordingly. In addition, JMP 16 also provides a graph of the relationship between the number of samples and power, which can be adjusted as appropriate to observe the changes in the number of samples (or power) under different values of power (or number of samples).
Example 1: Calculate power by knowing the number of samples of two sets of means:
In one study, we got a negative result. At this time, we want to understand the power of the study to gain the credibility of the study. In this study, the standard deviations of the two groups were 2 and 3 respectively, the difference between the means was 2.5, and the sample number was 20 cases in each group. The operation of power calculation is shown in Figure 3. It can be seen that the corresponding power is 85.31%.
Figure 3 Power calculation animation of the mean of two groups of independent samples
Example 2: Calculate the number of samples for the mean of two groups with known power:
In a randomized controlled trial comparing two groups, we reviewed the literature and found that the standard deviations of the two groups were 1 and 1.5 respectively, the difference between the means was 1, the type I error alpha was 0.05, and the power was 80%. See Figure 4 for the sample number operation. It can be seen that each group requires 27 samples.
Figure 4 An animation of calculating the number of samples for the mean of two groups of independent samples
The ordinate of the sample number estimation curve on the right side of Figure 5 is power, and the abscissa is the total number of samples. The intersection of the two red lines is the currently displayed power and number of samples. This function helps researchers understand the relationship between power and sample number while calculating the number of samples, and helps researchers obtain a feasible and reliable number of samples based on actual clinical conditions.
In the previous two examples, we input power or sample number. In practice, we can also obtain the corresponding sample number or power by dragging Figure 5. If you drag power along the curve to 80%, you can see that the number of samples on the left shows that each group requires 27 samples (Figure 5).
Figure 5 Animated graph of the trend relationship between sample number and power
02 Sample size estimation for comparison of rates between two groups
Preparation before estimating
The sample number estimation for comparing the rates of two groups is basically the same as the parameters required for comparing the means of two groups, except that the parameter (04) is different. Here, the mean and standard deviation of each group are no longer needed, but the rate of each group needs to be obtained.
The estimated number of samples JMP accomplish
Click the power of the two independent sample ratios in Figure 1 to enter the corresponding operation interface (Figure 6).
Figure 6 Sample number and power calculation of two groups of independent sample proportions - operation interface
Example 3: Calculate the power of the number of samples with known two-group rates:
It is known that the rates of the two groups are 0.6 and 0.8 respectively, α is 0.05, and the number of samples is 50 in each group. What is its power? The operation is shown in Figure 7. It can be seen that the power is 60.88%.
This power is a bit low. If we want to achieve 80% power, how many samples are needed? You can enter 80 in power. As shown in Figure 8, the number of samples has changed, indicating that if you want the power to reach 80%, at least 79 samples are needed in each group.
Figure 7 Power calculation animation of two groups of independent sample rates
Figure 8 An animation of calculating the number of samples for two groups of independent sample rates
Click Save Settings in Figure 8 to store the current parameters and number of samples (Figure 9). Click on the prepared data collection table in Figure 2, and JMP will create a new data table (Figure 10) based on the current number of samples, making it easier for researchers to collect data directly.
Figure 9 Save settings output results
Figure 10 Output results of data collection table
After reading the above operations, are you can’t wait to start your experiment? Download JMP 16 and start your first analysis!
original: Experimental design series (1) | Calculation of sample size and power (power)
Further reading:
