Using the script below as an example, the F2 report shows the BCa confidence interval is [50.459, 61.602]. Saving the bootstrap samples to a data table and running distribution analysis on them produces different results, the BCa confidence interval is [50.4786, 61.6594] (see below). Should these results match? If not, what am I missing that explains why they are different?
dt = open("$Sample_Data/Dissolution DoE.jmp");
this_fc = dt << Fit Curve(
Data Format( Row ),
Y( :Dissolution 60, :Dissolution 120, :Dissolution 240, :Dissolution 360 ),
Group( :Set ),
F2 Analysis(
Alpha( 0.1 ),
Reference Level( "A" ),
Bootstrap Samples( 2500 ),
Random Seed( 123456 )
)
);
this_fc << (Fit[1] << Save Bootstrap Results);
Wait( 0 );
Current Data Table() << Distribution(
Arrange in Rows( 1 ),
Continuous Distribution(
Column( :R F2 ),
Horizontal Layout( 1 ),
Vertical( 0 ),
Process Capability( 0 )
)
);This article was written by Dr. Hyde to help me. I received the following question from a customer, and this question is the trigger for this post.
When my customer conducted f2 analysis, they encountered a significant difference between the results obtained using the BCa method (60) and the Percentile method (46.3), which could potentially lead to the rejection of equivalence. (refer to the below image, left side is BCa and right side is Percentile).
And after a while, I found the perfect answer for this from the paper (@Chandramouli_R Gave me).
I've linked the source of the paper below, and since it's quite a long paper, I've also summarized it here.
__________________________________________________________________________________
In conclusion, if the data has large variance, it is natural to obtain results where the lower limit of BCa is ‘significantly’ higher than the lower limit of Percentile due to bias correction and skewness distribution correction.
Since variance contributes to bias and skewness, if you wants to calculate CI by considering this aspect, I can recommend using the BCa method.
If the variance is not large and is all managed below 10-20%, the F2 distribution of the bootstrap generally follows a normal distribution, and there will be no significant difference in the conclusions of Percentile CI and BCa CI.
However, large variance causes skewness and bias of the distribution, and the Percentile CI method only obtains the lower limit of the 5% point regardless of the bias and distortion of the distribution, so it is not precise.
The BCa method obtains Z0, a variable that corrects bias, and A, which corrects skewness. Z0 is related to the proportion of bootstrap estimates that are smaller than the observed statistic, and a is proportional to the skewness of the bootstrap distribution.
Considering this, in the BCa CI method, if the distribution of the bootstrap is positively skewed, the CI is adjusted to the right, and if the distribution of the bootstrap is negatively skewed, the CI is adjusted to the left. In other words, BCa CI's method provides more accurate results and improved sensitivity by adjusting the confidence interval.
Because bias correction and acceleration approaches correct for skewness of the distribution in this way, the lower CI of BCa f2 may generally be higher than that estimated by the percentile CI method, thus increasing the probability of similarity.
In fact, there is a paper(Chandra gave me) that experiments to see how much the BCa Lower Limit and Percentile Lower Limit can differ as the variance increases.
Please refer to the table. The table below compares JMP, DD Solver, SAS, and PhEq.
What I need to see is compare the JMP calculated using the BCa method, with the PhEq calculated using the Percentile Method which I marked in a red box.
I can see that the difference in results is slight in the range of 10-20% variance, but as you go to 40-50% and 70-80%, you can see that the BCa value is higher than Percentile and the difference is significant.
*Source Paper: Dissolution Profiles Comparison Using Conventional and Bias Corrected and Accelerated f2 Bootstrap Approaches with Different Software’s: Impact of Variability, Sample Size and Number of Bootstraps | AAPS PharmSciTech (springer.com)
