When conducting research, many researchers will think about a question: "Do I have enough samples?" Because the number of samples will directly affect the rigor of the research and the reliability of the research conclusions. When the sample size is too small, the research results are unstable, and the risk of getting "false negative" results is high; when the sample size is too large, it will increase the research cost and experimental difficulty.
How to determine whether the sample size of the study is reasonable? Today, JMP will introduce how to obtain a suitable number of samples through Power analysis.
The relationship between the number of samples and power
Before the study begins, we will need to know how many samples to conduct the study.
When it comes to the number of samples, we have to mention an equally important concept: power, which is called statistical verification power, statistical verification power, and statistical power analysis, which is collectively referred to as power in this article. In simple terms, power means: if the two groups are actually different, the ability to find this difference with the current sample size.
Under the condition that other parameters are known, the number of samples and power can be calculated from each other. To calculate the number of samples, power must be set; similarly, if the number of samples is known, the power under the current number of samples can also be calculated. In the experiment, the statistical result is negative. If the power is too low at this time, it means that the number of samples is likely to be insufficient, so the original difference cannot be obtained.
How to count samples with JMP?
The calculation of the number of samples requires only two steps (as simple as putting an elephant in the refrigerator):
The first step is to consult historical documents to obtain the parameters required for the calculation of the number of samples, such as mean, standard deviation, rate, etc.
The second step is to enter these parameters into JMP and get the results.
JMP can realize the estimation of the number of samples for several common comparisons between groups in clinical trials, such as the estimation of mean comparison, the estimation of rate comparison, and the estimation of the number of samples for equivalence test, and the operation is simple and interactive. The following will focus on how to use JMP for sample count and power calculations.
In JMP, select Experimental Design → Sample size explorers → power (Figure 1). You can see that there are various scenarios for calculating power and sample size under the power menu. You can select the corresponding specific menu according to the research purpose. .
Figure 1 The number of samples and power calculation of the mean of two independent samples - menu selection
The two most common situations are described below, namely, the comparison of the means of the two groups and the comparison of the rates of the two groups.
01 Sample Size Estimates for Comparison of Two Group Means
Prepare for estimation
To estimate the sample size for the mean comparison of two groups, the following parameters need to be determined in advance:
01 Whether the test type is single-tailed or double-tailed, it is usually double-tailed unless there are special requirements;
The upper limit α of the 02 type-one error, usually 0.05 (two-tailed test) or 0.025 (one-tailed test);
Type 03 error β or power1-β, it is generally recommended that the power be above 0.80;
04 The mean (or the difference between the two groups) and standard deviation of each group, which can be obtained by consulting the literature or conducting preliminary experiments;
05 The ratio of the number of samples between groups, usually 1:1, and sometimes 1:2 (depending on the research purpose and actual situation);
06 If it is a follow-up study, the loss to follow-up rate should also be considered, which is usually 10% (depending on the actual situation);
07 When writing the opening report or research bid, the above parameters need to be provided to demonstrate the feasibility and reliability of the research.
estimated sample size JMP accomplish
Click the power of the mean of the two independent samples in Figure 1 to enter the sample number and power calculation interface (Figure 2).
Figure 2 The number of samples and power calculation of the mean of two independent samples - operation interface
The left side of Figure 2 is the parameters involved in the calculation of the number of samples, from top to bottom are the test type, fixed parameters (alpha and standard deviation), test parameters (including the difference to be tested, the number of samples and power), and the right side is the number of samples Estimation curve.
![[TW] JMP community-CTA-downloadback to WB.png](https://us.v-cdn.net/6038925/uploads/images/41096i24F89E6C10D2C2FF/%5BTW%5D%20JMP%20community-CTA-downloadback%20to%20WB.png "[TW] JMP community-CTA-downloadback to WB.png")
In versions prior to JMP 16, if you want to calculate the number of samples, you need to enter power; similarly, to calculate power, you need to enter the number of samples; in JMP 16 versions, the number of samples (or power) can be adjusted at any time, and the corresponding power (or sample) The number) changes accordingly. In addition, JMP 16 also provides a graph of the relationship between the number of samples and power, which can be adjusted arbitrarily according to the situation to observe the changes in the number of samples (or power) under different values of power (or number of samples).
Example 1: Calculate the power of the number of samples with the known mean of the two groups:
In a study, we got a negative result, at this time we want to know the power of the study to gain the credibility of the study. In this study, the standard deviations of the two groups were 2 and 3, the difference of the means was 2.5, and the sample size was 20 cases in each group. The operation of power calculation is shown in Figure 3, and it can be seen that the corresponding power is 85.31%.
Figure 3 Power calculation animation of the mean of two independent samples
Example 2: The number of samples for which power is known to calculate the mean of the two groups:
In a randomized controlled trial comparing two groups, we have reviewed the literature and found that the standard deviations of the two groups are 1 and 1.5, the difference between the means is 1, the first-class error alpha is 0.05, and the power is 80%. See Figure 4 for the sample number operation, and it can be seen that 27 samples are required for each group.
Figure 4 Motion diagram of sample number calculation of the mean of two independent samples
The ordinate of the sample number estimation curve on the right side of Figure 5 is power, and the abscissa is the total number of samples. The intersection of the two red lines is the current power and number of samples shown. This function helps researchers understand the relationship between power and the number of samples while calculating the number of samples, and helps researchers obtain a feasible and reliable number of samples according to the actual clinical situation.
In the first two examples, we input the power or the number of samples. In practice, we can also obtain the corresponding number of samples or power by dragging and dropping Figure 5. If the power is dragged along the curve to 80%, it can be seen that the number of samples on the left shows that each group needs 27 samples (Figure 5).
Figure 5 Dynamic graph of the trend relationship between the number of samples and power
02 Sample Size Estimates for Two Rate Comparisons
Prepare for estimation
The estimated number of samples for the comparison of the rates of the two groups is basically the same as the parameters required for the comparison of the mean of the two groups, except that the parameter (04) is different. Here, the mean and standard deviation of each group are no longer required, but the rate of each group needs to be obtained.
estimated sample size JMP accomplish
Click the power of the ratio of two independent samples in Figure 1 to enter the corresponding operation interface (Figure 6).
Figure 6 The number of samples and power calculation of the proportion of independent samples of two groups - operation interface
Example 3: Calculate the power of the sample number of known two groups of rates:
It is known that the rates of the two groups are 0.6 and 0.8, α is 0.05, and the number of samples is 50. What is the power? The operation is shown in Figure 7, and it can be seen that the power is 60.88%.
This power is a bit low. If we want to achieve 80% power, how many samples do we need? You can enter 80 in power. As shown in Figure 8, the number of samples has changed, indicating that if you want the power to reach 80%, at least 79 samples are required for each group.
Figure 7 Power calculation animation of two independent sample rates
Figure 8 Motion diagram of sample number calculation for two independent sample rates
Click Save Settings in Figure 8 to store the current parameters and number of samples (Figure 9). Click to create a data collection table in Figure 2, and JMP will create a new data table (Figure 10) based on the current number of samples, so that researchers can directly collect data.
Figure 9 Save the setting output result
Figure 10 The output result of making a data collection table
After reading the above operations, can't wait to start your experiment? Download JMP 16 and start your first analysis!
original: Experimental Design Series (1) | Calculation of sample size and power (power)
Further reading:
