Hello,
a customer asked me if the DSD make sense to use it for optimization? I mean by that, the optimum found for each parameter (eg if one seeks to maximize response) is it as reliable as an optimal design?
Thanks for your comments, Valerie
Valerie,
It can be in some situations, but it wouldn't be my first choice in most cases. The problem is that, unless there is effects sparsity, you won't be able to estimate two factor interactions. From an optimization standpoint, this is problematic. In addition, two factor interactions are correlated with each other and quadratic effects, so even if you can estimate them, the precision of their estimation may be low. Much of this depends on how many factors you have and how many are active.
Don
I agree with Don that this wouldn't be my first choice. Why would the customer want to go with DSD and not Custom Design (e.g.). There he could add exactly the model Terms he would like to have estimated and if there are any constraints e.g. caused by technical restrictions he also could incorporate these easily. If it's a not a typical process but e.g. optimizing over an area you may want to go with space filling designs then.
Good comments by Don and Martin. I would ask "How many factors does the customer have?" I think this is very important to know. I would discourage them from thinking or voting a 10-factor problem down to 5. They leave the opportunity to learn about the 5 left-behind factors on the conference room whiteboard. If all they truly have is 5 or fewer factors then going directly to an RSM design makes more sense than going with a DSD. The default 5-factor continuous-variable design is 27 unique runs to support the 21-term model.
If the number of factors is larger, then the sparsity of effects is more likely to be true. Numbers to keep in mind are that a DSD for 6 or more factors can collapse to support the full quadratic model for any 3 factors. A DSD for 18 or more factors can support the full quadratic model in any 4 factors. A DSD with 22 or more factors can support the full quadratic model for any 5 factors.
If the customer has 10 factors, then the default Custom DOE RSM design needs 72 unique runs to support the 66-term model. That's quite a lot of work and not something most people would pursue. The 10 factor DSD would be 21 unique runs - quite a bit smaller. If the experimenter is lucky and only 3 factors are important, then the design will support the full quadratic model. BUT, if there are more than three important factors things get a bit trickier. Although Stepwise regression and/or Brad's new DSD Analysis Add-in may actually let you fit a model supporting for example five factors and a subset of the full quadratic model, a more conservative approach would be to augment the DSD to support the full quadratic in the 5 important factors. Adding a block variable to test if the process has shifted between when the DSD was run and when the augmented trials will be run and asking for a 10 new trials to be added to the original 21 - yields a 31-trial augmented design. This is still quite a bit smaller than the RSM design in 10 factors. Asking for 15 new trials instead of 10 yields a 36-trial augmented design - half the size of the 72-trial 10-factor RSM design.
How good are the 31-trial and 36-trial augmented designs compared to the 27-trial Custom DOE in 5 factors? In the Fraction Design Space (FDS) plot below the blue curve is for the 27-trial Custom DOE I-optimal design. The green dashed curve above it is for the 31-trial augmented design. The black dashed and dotted curve falling on top of the blue curve is the 36-trial augmented design. What we want to see here is that the curve is low and flat. And so it would appear that by the FDS metric the 36-trial augmented DSD is as good as the 27-trial Custom DOE. I would also say that the 31-trial augmented design is not all that bad.
Comparing Design Diagnostics such as D, G and A Efficiencies we can also see that the two augmented designs compare well with the 27-trial Custom DOE.
Returning to my earlier point, I think the choice of using a DSD depends highly on how many factors the customer may have. Five or fewer go right to RSM to do optimization. More than 5 (10 in my example) consider doing a DSD and then if NOT lucky enough to have the analysis collapse into supporting the full quadratic, or if NOT lucky enough to have stepwise regression find you an adequate model, then augment the DSD to support the full quadratic in the important factors.
Think of this as a two-step process that allows you to make an intelligent decision after the analysis of the DSD about whether or not the second step is necessary. I would also consider doing a few checkpoints to test the models after the DSD stage. If you can adequately predict the checkpoint observations, then it may give you more comfort if choosing to not run the augmented part of the two-step process.
Tom
Valerie: I agree with all previous comments. One other thought perhaps not yet fully fleshed out is the notion of a window of operability for the process at hand. And in line with Don's comments, with lots of active factors/interactions, or perhaps curvilinear responses for some factors, the DSD might not provide the desired estimate precision/bias of an interaction filled, curvilinear response space. Most design or manufacturing engineers want more than just a single optimal point in a design space...they want to know the window of operability around that optimal set point. So for a DSD, you'll get a point in n factor space...but with interactions or curvature, as the process factors move away from that point, the model might not estimate with the desired degree of precision/bias. So when windows of operability are an important issue...not can we 'just make this thing work', I'm not sure I'd use a DSD for optimization work as my first choice.
Agree with all the points made by my esteemed collegues.
My two cents and a brief answer
1.) DSD is for screening as thee name indicates and allows one to potentially examine many more factors than previously afforded since for instance 10 factors can be evaluated in 21 runs. Compare that to a Resolution IV design (a historical "gold standard") which required roughly the same expenditure of resources to only consider 6 factors.
2.) Sometimes due to effect sparsity and effect heredity one might be quite fortunate and be able to use the DSD to optimize ones process (See the Novomer success story where 10 factors were investigated and effect sparsity/herdity led to significant understanding). No further experimentation required and 3000 gallon scale up 6 months after the experiment.
3.) Realize however that experiments are sequential and many times further delineation through augmentation are required to improve one's understanding of the system.
4.) We don't know what we don't know and even when we think we know the system is more dimensional than our brains and requires experimentation so that we can understand and then explain why.
5.) Finally to me "optimization" means that I have settled in on my operating conditions and then perturb the inputs that are expected to exert variation over time on the process and delineate and understanding of the robustness of my operating space.
