How to evaluate Alias structures in D-optimal DOE?

Question

My understanding is that D-optimal designs can be good for screening experiments, but you'd better hope there aren't any large interaction effects, otherwise they will bias your main effects, or that you should add some suspected 2-way interactions to your optimal design as a way to avoid aliasing effects.

I've been told to avoid Res III fractional factorials, & Plackett-Burman designs for screening b/c they can have bad alias structures & confound main effects w/ interactions.

My question is - how do I compare a D-optimal DOE to a Res III or Plackett-Burman design - or is there any way to find the resolution for an optimal design to avoid any confusing alias structures? What are the rules of thumb for aliasing on optimal designs, and how are they reported in the JMP output - say if the alias matrix reports an alias >0.5 between a main effect and 2-way interaction, should that be avoided?

statman · Answer

The guidance is as follows...Build your understanding of the causal structure hierarchically and sequentially.  It is absolutely OK to use Res III designs when you have many first order effects (factors) you want to test/compare.  I think too often folks drop factors from consideration based on "gut feel and intuition" for the sake of economy.  What do you do with the factors that are not being studied?  Holding them constant restricts the inference space, allowing them to vary may decrease the precision of the design.  In both cases, it seems a "better" idea to include them and get data to help decide the next iteration.  It is also important to predict factor and interaction effects (and their rank order).  That is, use scientific and engineering knowledge for planning.  If your RANKED list has all main effects, then 2nd order effects, Res III might be a good choice.  As the predicted rank order of 2nd order effects "move up" the predicted list, you start using higher resolution designs. As long as you iterate, you won't compromise efficiency too much while improving effectiveness.

mjz5448 · Answer

Thanks Victor_G.

Is there a rule of thumb regarding aliasing one tries to follow in optimal designs (D or Alias optimal?)?

For example, it's clear to me why we want to avoid Res III designs in screening since some main effects are completely aliased w/ 2-factor interactions. But what's not so clear to me is what minimum threshold value in the alias table I should be looking at for an optimal design before I should think about adding runs. Is say 0.33 or 0.4 considered high in an alias table for main effects & 2-factor interactions, or is 0.25 a good start? Basically at what point should a number give me pause, and cause me to re-evaluate the design and add more runs?

Victor_G · Answer

Hi @mjz5448,

The aliasing structure of a design depends on the design choice, number of runs, and constraints/linear dependencies between factors (if any). The more constraints/linear dependencies on the factors, the harder and more complex it is to estimate independantly model terms of the factors.

If your primary concern is about estimating precisely and in an unbiased way main effects, with the possibility to detect and estimate interaction effects, here are some suggestions about designs with good aliasing properties :

* Plackett-Burman Design with Foldover augmentation for continuous and 2-levels categorical factors, no constraints. Here is a post comparing the Foldover Plackett-Burman design with Definitive Screening Design and Bayesian D-Optimal design. 
* Definitive Screening Designs for mostly continuous factors, no factors constraints.
* Mixed-Level Screening Designs for continuous and 2-levels categorical factors, no factors constraints.
* Bayesian D-Optimal Design for continous and categorical factors, possible with factors constraints. In the Model panel, add the main effects with estimability set as "Necessary" and higher order terms (like 2-factors interactions and quadratic effects) as "If Possible" and make sure the number of runs is sufficient to enable fitting a model with more terms than only main effects. I wrote a post about the use of Bayesian D-Optimal design when factors constraints prevents the use of Definitive Screening Design.

If you want to determine the resolution of your design, this link from NIST will help you understand the differences between resolutions : 5.3.3.4.4. Fractional factorial design specifications and design resolution

When Evaluating Designs, you can look at the correlations between terms on the Color Map on Correlations and save the correlations in the red triangle of this section, "Table of Correlations".

About the correlations and the advices to choose one or another design, the design choice is mostly impacted by the number of runs possible in your case. It is for sure "safer" to avoid high correlations between terms, but the necessary runs number required to reduce some of these correlations is sometimes not feasible. So you have to define the best trade-off between your objective and your experimental budget (runs number, there is no "predefined threshold"), and choose the most relevant design. As @P_Bartell mentioned, create several designs and use the platform Compare Designs to better assess the pros and cons of each design.

And on a positive note, there is no failed DoE ; if you didn't achieve your objective, you can always Augment your Design and reduce/remove the correlations between terms that were preventing you to detect interactions or other higher order terms.

Hope this answer will help you,