DOE with 5 multi-level categoricals. Best sequential approach to finding optimum?Hi,
Hoping you might have some ideas for this interesting problem.
We have an objective to find the settings that optimise yield for a chemical reaction.
There are 5 categorical factors for different choices of reactants, reagents, ligands, solvents (factors table attached). The factors have between 3 and 12 levels.
There are 4608 possible combinations of the factor levels. It is not feasible to run all of them!
It is possible to carry out 48 runs simultaneously, taking about 2 days of lab work.
So, in the first stage we run a custom main effects screening design with 48 runs (attached).
This first block of runs uncovers some useful insight:
- Reactant_1 and Reactant_2 have the biggest effect.
- Some combinations give yields of >90%.
Distribution of %Yield and 5 Factors for 48 run screening experiment.
The question is: what next? What should the next 48 runs be?
Re: DOE with 5 multi-level categoricals. Best sequential approach to finding optimum?Hi Phil,
Thanks for highlighting this example, this seems important for a number of customers who focus on the big experiment rather than a sequential approach. And this is a head scratcher so am eager to explore what everyone thinks!
Considering whether we have grounds to screen out levels of factors could be useful. Using Effect Details (better than my Fit Y by X initial suggestion) and a T-K HSD (being as conservative as possible), maybe there's grounds to remove 6-Cl-Q (Reactant 1) and Trifluoroborate (Reactant 2) as levels? Also interesting is that whilst "None" for ligand is 2nd to lowest average yield, "None" for Reagent 1 is mid yielding. It's not the same as a null factor but perhaps focus towards if some reagent 1's that are negatively impacting could be useful in the next 48?
Kind Regards,
Ben
Re: DOE with 5 multi-level categoricals. Best sequential approach to finding optimum?If you include the interaction between Reactant 1 & 2 into the model there's a combination of factors in the profiler which bring the yield to 99%. Instead of exploring other combinations of factors with subsequent DOEs perhaps it's best to now move into a set of confirmatory experiments to validate that those identified settings will yield 99% in practice.
Re: DOE with 5 multi-level categoricals. Best sequential approach to finding optimum?Good point @Nick, perhaps we could use the simulator to find out a set of predicted high yield (Obj>0.95) candidates? This leads to 23 candidate sets in the attached.
Re: DOE with 5 multi-level categoricals. Best sequential approach to finding optimum?Thanks @benfrancis and @nick_shelton .
Both good ideas.
Some confirmatory runs around the current predicted best settings would seem like a good idea. But I think we might wish to do more with the next 48 runs that we have.
Expanding on what Ben suggested I looked at the Scaled Estimates and there are category levels from some factors that are significantly worse than the others, e.g. Ligand[Xantphos].
(Xantphos is just the worst ligand ever, isn't it. Personally I would never use Xantphos for anything:)).
It would seem rational and justified to remove these category levels from further runs.
Similarly we would be justified in getting rid of 2 of the levels of Reactant 2 and just using Boronic acid.
Has anyone tried using Augment on categorical variables? I can't seem to figure out how to remove a category level before augmenting.
Cheers,
Phil
Re: DOE with 5 multi-level categoricals. Best sequential approach to finding optimum?I like that idea, @benfrancis .
Ideally we would select a set of 48 runs that are:
- Predicted from our current model to have high yield
- Optimal for testing the model of interest
So maybe we could:
- Generate a very large candidate set of 1000s of runs (say, the full factorial) and then
- Select from this a subset of 100s of runs that are predicted to have yield above some threshold.
- Use these as a candidate set and Custom Design with covariate factors to determine the optimal 48 to test the model
Re: DOE with 5 multi-level categoricals. Best sequential approach to finding optimum?In augment design, you can't remove categorical levels but you can set a factor constraint to prevent new runs from using certain levels.
Re: DOE with 5 multi-level categoricals. Best sequential approach to finding optimum?Thanks. That is really helpful, @nick_shelton .
Re: DOE with 5 multi-level categoricals. Best sequential approach to finding optimum?By the way, the example comes from a published paper where they actually ran the full factorial (4608 runs!).
This is a really nice dataset (attached JMP table). Although it is difficult to know how accurate or precise the data are.
If nothing else you can make a nice heatmap:
