Somebody told me we need to keep power >=80% when evaluating a DOE, but why the power is so low when using PB design? For example, the power is around 0.27 if I run 12 experiments with 11 factors. Can I trust this design with such low power? Thank you.
Hi @JMPLearner_2022,
Power is the ability to detect significant effect if they are effectively present.
This 80% is a pratical rule of thumb to evaluate if your screening design may be able to detect main effects efficiently, but let's keep in mind that power calculation depends not only on the design (and its sample size), but also on the significance level (often 0,05), difference size to be detected and expected signal-to-noise ratio (or RMSE) of the response(s).
So if your anticipated RMSE is lower (response(s) is/are measured more precisely and with lower variance) and/or the difference size you want to detect is higher, you might expect higher power for your effects.
Power might not be very interesting to consider in case of optimization design (mixture designs, space-filling designs...), because their primary focus will be on response(s) prediction precision, not estimates calculation precision. But power is of primary interest in case of screening design, like in your case.
Difficult to tell if we could trust your design with these info, since it depends not only on the design itself and its expectations (RMSE, significance level, difference size to detect, ...), but also on which factors you want to screen, which ones you expect to see significant, expected difference to detect... = domain expertise.
Maybe one solution for you would be to try computing different designs, in order to compare the strengths and weaknesses of the different screening designs.
Looking at the Custom Design, if I fix 12 experiments for 11 continuous factors (minimum number of runs required), I find also power for main effects close to 0,21 (so 21% of chances to see a significant effect if present). But if augmenting the number of runs to 16 (recommended by JMP, "Default" setting), power for main effects rise up to 0,84. If experimental cost is not too much, increasing the sample size may help you a lot to gain more confidence in your screening ability.
Hope this will help you,
