nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

How to Augment a Bayes Opt for DoE applications (QbD, Robustness)

Ben_BarrIngh

Hi All,

In the ENBIS Bayesian Course, the idea of augmenting a series of Bayes Opt experiments to turn the data set into something suitable for QbD/Robustness studies is presented, which is a good use case for our customers.

The question I have is, when using the 'Augment' tool in the DoE menu, how can you accurately decide the number of additional runs required to ensure there is no confounding (ie a clear color map of correlations). In the 3 factor example above, the suggested augmented runs are always 8 additional, regardless of if I've applied a simple linear model, or an RSM.

Compare that to Custom Design, where changing the model alters the number of recommended experiments, I'm wondering if similar can be done here, or at the least, it can become clearer how many additional experiments would be needed for a set criteria (D efficiency, low correlation etc) without manually increasing the numbers and checking every time.

@Florian_Vogt @Phil_Kay @chris_gotwalt1

Thanks!
Ben

Accepted answers

Phil_Kay

Great question. I'm surprised that you don't know that the answer is the classic statistician response: it depends.

It really will depend.

It is very unlikely that you would be able to augment data from a Bayesian optimization project and arrive at a design with 0 correlation. So it's almost always going to depend on how much correlation is acceptable. And that will be really hard to decide.

I'm not totally sure how the recommended number of runs is determined in Custom Design, but I think it's something like the number of parameters in the model to be estimated plus 5 (for error DoFs). There is no right answer there but it is relatively simple to give a sensible "rule of thumb" answer.

I think the best option for the experimenter is to consider how many runs they can realistically afford and create 3 different augmented designs with different numbers of runs up to that maximum. They can then use Compare Designs to quickly and easily compare the 3 options across the different design diagnostics, including effect correlations. I don't think this would be possible with any other software, so it's a nice differentiator for JMP.

All comments

Phil_Kay

Great question. I'm surprised that you don't know that the answer is the classic statistician response: it depends.

It really will depend.

Ben_BarrIngh

Hi @Phil_Kay ,

Of course, I should have known! Thanks for the differentiation, that's a nice way to approach how it can be applied if asked.

Thanks,

Ben

russ_wolfinger

Follow up questions: My (perhaps naive) understanding of JMP's BayesOpt is as an effective Gaussian-process based alternative to DOE Augment Design. Is this misguided? Are there general guidelines on when to use Augment instead of turning the BayesOpt crank again to generate more design points?

Mark_Bailey

As the name implies, to goal is optimization, not characterization and modeling. It is meant to find the optimum settings as quickly as possible (i.e., fewest augmentations/runs). You might be done after finding the optimal settings. In some situations, like pharma and biotech, though, you would then follow through with DOE to model the surround surface. Think QbD and Design Space.