nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Bayes opt with continuous descriptors of a categorical input

Phil_Kay

Hi,

A customer asked about using Bayes opt in a situation where they have a single categorical input (e.g. catalyst type) that they can describe using multiple continuous variables (e.g. chemical descriptors).

I know they could do this by using the table of descriptors as a candidate set. Has anyone seen examples of customers doing this? Or have any SEs experimented with simulated examples?

Thanks

Phil

Accepted answers

Florian_Vogt

Indeed

Yes, in that case a synthetic dataset may be the way to go.

Unilever have basically stated that this (product + process dev) is where BayesOpt does pretty well (just that they use chemical properties of ingredients, instead of catalyst/QSAR and possibly other process parameters like stirring etc.). They are currently evaluating and have not shared any insights, but maybe by the end of October I will get more detailed insight - possibly even a public case study.

All comments

Ben_BarrIngh

Hi @Phil_Kay,

Are they trying to learn what the composition effect of the catalyst has on the overall process (build a model) with aims of controlling/adapting the catalyst makeup?

I've had similar from customers (for example in perfume development), they can use compound A,B or C, they know the make-up of it (ie 10% Oil, 5% Alcohol), but can't do purposeful experimentation on it because they can't tweak the ratios, which meant BayesOpt was little use.

Cheers,
Ben

russ_wolfinger

Hi @Phil_Kay, In case this helps, the attached table is from Assistant with the following prompt: "Simulate QSAR data to predict catalyst type using chemical descriptors. Use realistic chemical relationships and make sure some descriptors are significant."

There is also a script for an XGBoost model, which appears to work well.

QSAR Catalyst Data - Simulated.jmp

Phil_Kay

Thanks, Ben. I believe what they want to do is to ultimately select the best catalyst to give optimum yield. There might be 50 catalysts to choose from. Each catalyst will have unique values for the descriptor variables, reflecting their different chemical properties. They can't just test them all as that is too many runs of the reaction, so the idea would be to use the descriptor variables as factors in Bayes opt as a rational way to explore the catalyst property space and move towards the optimum catalyst.

Phil_Kay

Thanks, @russ_wolfinger . That looks pretty good. But the case I'm thinking of is a candidate set with 50 unique catalysts. Sorry, I think my "catalyst type" wording was not the best. Each catalyst would have unique values for the descriptors.

russ_wolfinger

Thanks for the clarification Phil.

Is there perhaps a continuous target Y (say yield) that is to be optimized as a function of the descriptors? Catalyst name then serves as a row label?

Assuming this is the case, the data could be run through BayesOpt to find descriptor values that optimize Y. Once found I believe this would require ability to synthesize a catalyst with the optimal descriptor combination (or at least a close approximation to it).

Alternatively, QSAR (Quantitative Structure Activity Relationship) modeling fits a predictive model of Y as a function of the descriptors. XGBoost or Torch are good choices and you could then use the profiler to optimize Y or use the fitted model to predict currently unmeasured catalysts with known descriptors.

If you happen to have SMILES strings for the catalysts then Torch has Graph Neural Network models that typically work very well.

I went ahead and added a Yield column to the simulated data.

Russ

QSAR Catalyst Data - Simulated.jmp

Phil_Kay

Yes, Yield would most often be the response variable, although impurity levels and cost could be additional ones. Yield is unknown and only determined by running the reaction. And you can't afford to run the reaction for all 50 catalysts. Hence you would use Bayes opt to (hopefully) efficiently explore the catalyst property space. I think you would just settle on using the catalyst that worked best, although if the model suggests optimum properties that are not similar to one of the catalysts, you might add more to the candidate set for consideration.

I think this should be possible and reasonably straightforward.

But I'm not sure that it would be possible to have additional process variables, like temperature or pH, for example. I expect this would be important. I think that the only way to do that would be to create a candidate set for the catalyst descriptors, and one for the process factors (say a space-filling design), and then take the product of those. So you would have a massive candidate set with multiple process factor treatments for each catalyst and it's descriptors.

Florian_Vogt

Hi @Phil_Kay ,

there is some public data which may fit your case: https://github.com/SigmanGroup/Threshold/tree/main/examples

I have just quickly downloaded the dataset and tested it using it as a candidate set and look up table. (This seems to be a related publication https://www.science.org/doi/10.1126/science.abj4213?referrer=https%3A%2F%2Fgithub.com%2FSigmanGroup%2FThreshold%2Ftree%2Fmain)

Draw back on my quick initial trial is that BayesOpt apparently got stuck on one recommendation (which was spot on for one response btw.)

Maybe synthetic data is the "safer" option for demos...

Hope this helps

Florian

Starting Table.jmp

Candidate Set cleaned.jmp

Lookup Set w Results.jmp

Phil_Kay

Well, I guess it found the optimum very effectively! It should be more interesting when you have process variables in there as well, with interactions between those and the catalyst descriptors. But, as I've said above, I think that will be a challenging workflow currently.

Florian_Vogt

Indeed

Yes, in that case a synthetic dataset may be the way to go.

Phil_Kay

That would be very nice. I think we'll start to see more and more successful case studies coming from customers that will really help us sell the value.

russ_wolfinger

Great data Florian, thanks for sharing. We have a very strong story for these kinds of data from a predictive modeling perspective:

1. Gaussian Process and the iterative conveniences BayesOpt affords. GP is likely best for smooth systems. MixedModel in Torch is similar.

2. XGBoost / LightGBM / Boosted Trees: Best for nonlinear interaction systems with sharp changes or even discontinuities.

3. Neural Nets in Torch and JMP: Best for systems with moderate smoothness, in between GP and boosted trees. Extensive structure richness.

4. Graph Neural Nets on SMILES in Torch: Most faithful to chemical structure, potentially best of all.

3 and 4 are readily combinable into one model and the best possible predictions come from ensembling all of them.