How to use sample size and power for factorial DoE to determine number of replicated treatmentsI'm trying to learn how to operate JMP's dialog boxes for sample size for use in experimental design. I'd like to understand how to determine the number of replicates necessary in a factorial experiment. I'd like to know how many times to run each treatment combination.
Assume there is no measurement error, no need for blocking or centerpoints, the response variable is continuous and everything is exactly as needed to meet all the assumptions for classic calculation of sample size (random stationary homoskedastic normally distributed errors, etc.).
Suppose an experiment will have two factors at two levels, full factorial. Thus, there are four treatment combinations. I want to estimate the two main effects and the one interaction. What are the steps to determine the number of replications of each treatment combination?
The same question applies for fractional factorial. If I have 15 factors each at two levels and I want to run a fractional factorial of 16 treatment combinations, how do I determine the number of replicates of each treatment?
For the sake of sample data, let alpha = 0.05, beta = 0.05, sigma = 1, delta = 1.
I've used "Two Sample Means" dialog. There the static text says "Sample size is the total sample size; per group would be n/2". If I use this calculator for factorial design, what's the number of replicates per treatment combination for full or fractional factorial? Do I just divide the total sample size by the number of treatment combinations?
Now suppose I want a centerpoint in the 2^2 experiment. That's five treatment combinations. Assume I'll replicate all five treatments. To design an experiment to test the significance of the centerpoint effect, I'll assume that effect is equal to delta. How do I use the JMP dialogs to get the number of replicates per treatment? Do I divide the "Two Sample Means" sample size by five?
If I want replication only at the centerpoint and no replication of the corner points, I'll assume the standard deviation that is revealed at the centerpoint is constant throughout the inference space. The goal is still to estimate the significance of the main effects, the interaction and the centerpoint. Assuming the same delta, how do I use the dialogs to get the number of replicates of the centerpoint combination? Do I divide the "Two Sample Means" sample size by one?
I'm aware of the custom design platform and its power analysis and know how to operate it. I would like to use DOE > Design Diagnostics > Sample Size and Power. After all, those calculators are in the "DOE" menu, I'm trying to figure out how to use them for DoE.
DOERe: How to use sample size and power for factorial DoE to determine number of replicated treatmentsThe sample calculators in the DOE menu are meant for simple cases with at most one factor. The power analysis is in included in the DOE platforms. the process is to generate the design, then evaluate the power. You can save the design in a data table and the compare the power, and other attribute, across more than one design.
See this section in JMP Help for explanations about how to perform a power analysis of a design. I see that you are familiar with these tools, though.
You can use Make Table from Full Factorial with the desired replication and number of center points, then select DOE > Design Diagnostics > Evaluate Design to get the Power Analysis.
Re: How to use sample size and power for factorial DoE to determine number of replicated treatmentsThe proposed approach is a trial-and-error way of designing an experiment (pick a design, choose a number of replicates, see what the power is, iterate until happy with the balance of power with sample size). Is this really necessary? It's absolutely possible to do what I'm requesting by hand (or with Minitab) as a way of designing an experiment rather than evaluating an experiment.
Re: How to use sample size and power for factorial DoE to determine number of replicated treatmentsI apologize in advance for the following comments/questions:
What is the purpose of replication? I believe there are several purposes:
1. To increase the inference space of the study,
2. To make the study more representative of future conditions (reality),
3. To determine if the effects of factors studied under one set of conditions are consistent when those conditions change,
4. Robust design: identify noise-by-factor interactions early in design to take action early early to reduce the impact of those noise effects...
So to me, replication is done to ensure the experiment is representative of future conditions so I increase my confidence the results can be extrapolated into the future.
“Unfortunately, future experiments (future trials, tomorrow’s production) will be affected by environmental conditions (temperature, materials, people) different from those that affect this experiment…It is only by knowledge of the subject matter, possibly aided by further experiments (italics added) to cover a wider range of conditions, that one may decide, with a risk of being wrong, whether the environmental conditions of the future will be near enough the same as those of today to permit use of results in hand.”
Dr. Deming
That being said, why would you rely on a statistical formula to provide guidance as to whether the experiment is representative of reality?
Re: How to use sample size and power for factorial DoE to determine number of replicated treatmentsYou can only calculate the prospective power of a given design. It is a function of the model matrix, which, in turn, is a function of the design matrix. Without a design, you cannot compute the power.
Selecting the 'best' design is iterative, using 'trial and error,' but it is also practical by developing a small set of designs that span the choices in which you vary the complexity as you intended (e.g., number of runs, terms in the model, number of required center points, et cetera). Then you can compare them all simultaneously using DOE > Design Diagnostics > Compare Designs to narrow down the choice. It is a pretty quick process.
The choice is also 'good enough' because your assumptions that are used in the power analysis are wrong but they are good enough to get the job done (select a reasonable design).
How would you calculate the design by hand or with MINITAB, given the desired power? I think that the answer must rely on the special nature of a rigid factorial design.
