Modification of a partially executed DOE

Question

We do many DOEs that involve a lot of process steps and can take a long time to execute. Sometimes we have other results coming in that makes us want to change the DOE that has been partially processed; add or take out factors, levels and effects that pertain to operations we have not executed yet but keep factors and levels fixed for operations that have already been completed.

In my current workaround I regenerate the DOE from scratch and try to match as best as possible the runs already executed, but this requires small compromises and only works for a small number of 'fixed' factors.

Is there a better way?

statman · Answer

Let's imaging I am investigating components of a new coating that is made in batches. I want to experiment on factors in the formulation and then factors in the manufacturing process. In the whole plot, I have the experiment on the formulation factors (e.g., material type, amount of ingredients, additives). I create 1 batch for each treatment combination. Then split each of those batches (into sub-batches the number of which is determined by the factors in the processing experiment) and run experiments on the processing factors in the sub plot (e.g., mix time, temperature, mix speed). Saves a lot of resources. Very useful for testing new product designs (factors in the whole plot) over extreme noise conditions (noise factors in the subplot) to create robust designs. I recommend the definitive paper by Box and Jones "Split-plot Designs for robust product experimentation" and Bisgaard's "Robust Product Design: Saving Trials with Split-plot Confounding".

Wibo · Answer

Thanks for your recommendations, some more context:

We cannot draw conclusions from partially completed experiments. But we do many experiments concurrently, these can take a long time to complete (months). So there are cases where you would like to modify a DOE because you get new insights from other experiments that were not known when the design was created.

We have good results with D-Optimal with important interactions included in the effects.

Why is split-plot better for sequential processes?

statman · Answer

My thoughts/questions/comments:

1. You have completed part of the experiment and drawn conclusions about factors and interactions before the experiment is finished?  Are the effects obvious?

2. Are the process steps sequential?  If they are, you might consider split-plot designs to be more efficient.

3.  Seems like you should do more planning before execution.  I have found predicting ALL possible outcomes to be useful in thinking through the experiment and subsequent actions.

4. Make sure you consider block effects as you modify design factors and level setting.