973200_en-US

liliconcom

Bayes opt with continuous descriptors of a categorical input

Hi,

A customer asked about using Bayes opt in a situation where they have a single categorical input (e.g. catalyst type) that they can describe using multiple continuous variables (e.g. chemical descriptors).

I know they could do this by using the table of descriptors as a candidate set. Has anyone seen examples of customers doing this? Or have any SEs experimented with simulated examples?

Thanks

Phil

Re: Bayes opt with continuous descriptors of a categorical input

Hi @Phil_Kay,

Are they trying to learn what the composition effect of the catalyst has on the overall process (build a model) with aims of controlling/adapting the catalyst makeup?

I've had similar from customers (for example in perfume development), they can use compound A,B or C, they know the make-up of it (ie 10% Oil, 5% Alcohol), but can't do purposeful experimentation on it because they can't tweak the ratios, which meant BayesOpt was little use.

 

Cheers,
Ben

Re: Bayes opt with continuous descriptors of a categorical input

Hi @Phil_Kay,   In case this helps,  the attached table is from Assistant with the following prompt: "Simulate QSAR data to predict catalyst type using chemical descriptors. Use realistic chemical relationships and make sure some descriptors are significant."

There is also a script for an XGBoost model, which appears to work well.

Re: Bayes opt with continuous descriptors of a categorical input

Thanks, Ben. I believe what they want to do is to ultimately select the best catalyst to give optimum yield. There might be 50 catalysts to choose from. Each catalyst will have unique values for the descriptor variables, reflecting their different chemical properties. They can't just test them all as that is too many runs of the reaction, so the idea would be to use the descriptor variables as factors in Bayes opt as a rational way to explore the catalyst property space and move towards the optimum catalyst.

Re: Bayes opt with continuous descriptors of a categorical input

Thanks, @russ_wolfinger . That looks pretty good. But the case I'm thinking of is a candidate set with 50 unique catalysts. Sorry, I think my "catalyst type" wording was not the best. Each catalyst would have unique values for the descriptors.

Re: Bayes opt with continuous descriptors of a categorical input

Thanks for the clarification Phil.

Is there perhaps a continuous target Y (say yield) that is to be optimized as a function of the descriptors?  Catalyst name then serves as a row label?   

Assuming this is the case, the data could be run through BayesOpt to find descriptor values that optimize Y.  Once found I believe this would require ability to synthesize a catalyst with the optimal descriptor combination (or at least a close approximation to it).

Alternatively, QSAR (Quantitative Structure Activity Relationship) modeling fits a predictive model of Y as a function of the descriptors.  XGBoost or Torch are good choices and you could then use the profiler to optimize Y or use the fitted model to predict currently unmeasured catalysts with known descriptors.

If you happen to have SMILES strings for the catalysts then Torch has Graph Neural Network models that typically work very well.

I went ahead and added a Yield column to the simulated data.

Russ

Re: Bayes opt with continuous descriptors of a categorical input

Yes, Yield would most often be the response variable, although impurity levels and cost could be additional ones. Yield is unknown and only determined by running the reaction. And you can't afford to run the reaction for all 50 catalysts. Hence you would use Bayes opt to (hopefully) efficiently explore the catalyst property space. I think you would just settle on using the catalyst that worked best, although if the model suggests optimum properties that are not similar to one of the catalysts, you might add more to the candidate set for consideration.

I think this should be possible and reasonably straightforward.

But I'm not sure that it would be possible to have additional process variables, like temperature or pH, for example. I expect this would be important. I think that the only way to do that would be to create a candidate set for the catalyst descriptors, and one for the process factors (say a space-filling design), and then take the product of those. So you would have a massive candidate set with multiple process factor treatments for each catalyst and it's descriptors.  

Re: Bayes opt with continuous descriptors of a categorical inputOne thing I’ve run into that would be worth considering when using yield as a response for BayesOpt: If the goal for yield is set to “maximize” and they are doing yield as a percentage instead of absolute units (like g/L), the algorithm can start predicting non-physical outputs like OLS does (i.e., 103% yields). That might result in odd suggestions for experiments.Re: Bayes opt with continuous descriptors of a categorical input

Hi @Phil_Kay ,

 

there is some public data which may fit your case: https://github.com/SigmanGroup/Threshold/tree/main/examples

I have just quickly downloaded the dataset and tested it using it as a candidate set and look up table. (This seems to be a related publication https://www.science.org/doi/10.1126/science.abj4213?referrer=https%3A%2F%2Fgithub.com%2FSigmanGroup%2FThreshold%2Ftree%2Fmain)

Draw back on my quick initial trial is that BayesOpt apparently got stuck on one recommendation (which was spot on for one response btw.)

Maybe synthetic data is the "safer" option for demos...

 

Hope this helps

Florian

Re: Bayes opt with continuous descriptors of a categorical input

Well, I guess it found the optimum very effectively! It should be more interesting when you have process variables in there as well, with interactions between those and the catalyst descriptors. But, as I've said above, I think that will be a challenging workflow currently. 

Re: Bayes opt with continuous descriptors of a categorical input

Indeed

Yes, in that case a synthetic dataset may be the way to go.

Unilever have basically stated that this (product + process dev) is where BayesOpt does pretty well (just that they use chemical properties of ingredients, instead of catalyst/QSAR and possibly other process parameters like stirring etc.). They are currently evaluating and have not shared any insights, but maybe by the end of October I will get more detailed insight - possibly even a public case study.

Re: Bayes opt with continuous descriptors of a categorical input

That would be very nice. I think we'll start to see more and more successful case studies coming from customers that will really help us sell the value.  

Re: Bayes opt with continuous descriptors of a categorical input

Great data Florian, thanks for sharing.    We have a very strong story for these kinds of data from a predictive modeling perspective:

1. Gaussian Process and the iterative conveniences BayesOpt affords.  GP is likely best for smooth systems.   MixedModel in Torch is similar.

 

2. XGBoost / LightGBM / Boosted Trees:   Best for nonlinear interaction systems with sharp changes or even discontinuities.

 

3. Neural Nets in Torch and JMP:  Best for systems with moderate smoothness, in between GP and boosted trees.   Extensive structure richness.

 

4. Graph Neural Nets on SMILES in Torch:   Most faithful to chemical structure, potentially best of all.

 

3 and 4 are readily combinable into one model and the best possible predictions come from ensembling all of them.

Re: Bayes opt with continuous descriptors of a categorical input

@MikeD_Anderson any good tips for dealing with the % as a response issue? I've come across it a lot for customers (ie in dissolution %) where the only measures are %, I've found it hard to understand how to add a strict ceiling, in running synthetic runs BayesOpt can still end up going over.

Re: Bayes opt with continuous descriptors of a categorical input

@Ben_BarrIngh - the "right" answer is we need the equivalent of GenReg for Stochastic Processes (the surrogate model used by our implementation of BayesOpt). That would let us put in the beta distribution (i.e., BeSP instead of GaSP modeling). There is math out there for that kind of thing - but someone in Dev or PM would have to speak to if there's a roadmap for that capability.

 

For now, the best thing that I've come up with is to switch terms like Yield % from a maximize goal to a target goal and set something like 99.5% as the target.  Then you can adjust the upper and lower bounds of the desirability curve to force it to ignore stuff over 100%.  Since bayes opt is a little fiddly about doing this in the platform at the moment, you'd need to put these settings in the response column property for the response column.  This seems to work okay - but it is a hack. 

 

 

Re: Bayes opt with continuous descriptors of a categorical input

Very useful, thanks @MikeD_Anderson 

Re: Bayes opt with continuous descriptors of a categorical input

You can also try taking the logit of the proportion response. That's how logistic regression converts pass/fail into data on (–∞, +∞). Then take the logistic function of the predictions to transform back to proportion units. (I think JMP might have a Logit Percent function? Then you don't have to carry around the 100.)