One of the most frequently cited characteristics of Bayesian optimization is that it can arrive at the optimal solution with a small number of experiments .
This feature becomes a significant advantage when each experiment is time-consuming and costly.
Version "JMP Pro 19," released last September, added a Bayes optimization platform . In fact, many people are considering adopting it or are interested in it because they find the ability to "reduce the number of experiments" appealing.
1. Other major features besides "requiring fewer experiments"
Bayes optimization has another important feature: it can capture complex nonlinearities without having to pre-assume a specific functional form (such as a response surface model with a quadratic polynomial) for the relationship between the response and the factors.
Therefore, it has the advantage of making it easier to search for optimal conditions even if one does not have sufficient knowledge of statistical models such as interaction and response surface models required for design of experiments (DOE).
Therefore, in this article, we will compare Bayes optimization and experimental design for finding optimal conditions using simulation data for which the theoretical formula (true response) is known, and explain their characteristics.
2. Verification Examples
Here, we will use the optimization of the yield of the following enzyme-catalyzed reaction as an example.
Objective : To explore the optimal conditions for maximizing the yield of ester synthesis by combining three types of lipase enzymes with reaction conditions (concentration of reactants, amount of catalyst).
response
factor
The factor has the constraint condition "0.5 × concentration of reactants + 5 × amount of catalyst ≤ 110".
This time, we are performing a simulation where the true optimal value (theoretical formula) is known.
The diagram below shows the distribution and optimal values of the theoretical formula, with enzyme type = "CALB", reactant concentration = 109.7, and catalyst amount = 6.98.
The contour profile shows the response distribution when the enzyme type is "CALB". Darker red indicates higher yield, and in this example, it can be seen that it has a complex nonlinear structure with two peaks (multimodal).
3. How to compare Bayes optimization and experimental design.
For the same theoretical formula*, we will perform experimental design using two different approaches: Bayes optimization and experimental design.
Bayes Optimization
The Space Filling plan generates three initial experiments that satisfy the constraints and inputs the responses to those experiments.
Using the [Bayes Optimization] platform, experiments are added sequentially, one at a time. Experiment points are added using the experimental candidates displayed in the batch selection (using automatic mode).
The experiment is terminated sequentially when the system enters the optimal value search phase in automatic mode, or when the response value significantly exceeds the target value (70% or more), and the last experimental condition is considered the optimal condition.
Design of experiments
Using "Custom Design," fit the response surface model (quadratic polynomial) with the following settings.
The differences between the two can be summarized as follows:
Bayes optimization is a method that sequentially determines "what to try next" based on previous experimental results, without pre-determining a specific function, and uses a flexible model (Gaussian process model) for exploration.
Experimental Design : A combination of factors is determined in advance, and a model is created assuming a quadratic response surface model.
*In reality, a slight error (noise) is added to the theoretical formula.
4. Comparison and Discussion of Experimental Results
The following results were obtained. It can be seen that the results are overwhelmingly better with Bayes optimization, even though the number of experiments was smaller.
| Technique |
Number of experiments |
Optimal yield found |
| Bayes Optimization |
14 times |
Over 90% (significantly exceeding specifications) |
| Design of experiments |
18 times |
Approximately 15% |
Using Bayes optimization, after three initial experiments, we were able to find conditions that yielded over 90% of the results in 11 additional experiments.
On the other hand, we conducted experiments in advance to construct a response surface model for the entire experimental domain, and estimated the optimal conditions from the results.
We will use "contour profiles" to compare the optimal values obtained using each method.
Bayes optimization reaches the high-yield red region (the peak), while experimental design estimates an optimal condition at a location far from the red region.
The following reasons are thought to be the cause of these results:
- Experimental design methods (response surfaces) rely on models based on quadratic polynomials (single peaks), and therefore cannot capture complex shapes like the one in this case, which has multiple peaks (multimodal properties).
- In Bayes optimization, a specific functional form is not assumed in advance, and a flexible surrogate model is used for sequential exploration. This is thought to allow for the exploration of regions with high yields even when there are multiple peaks in the process.
5. Summary
Bayes optimization has two main characteristics. In this article, we focused particularly on the second characteristic : its ability to capture complex nonlinear relationships .
1. The optimal solution can be reached with a small number of experiments.
- By learning from past experimental results and intelligently selecting the next experimental point, it's possible to reduce unnecessary experiments and lower costs.
2. Capable of capturing complex nonlinear relationships.
- A specific function formBecause it does not make certain assumptions, it can capture complex nonlinearities and interactions between variables without overlooking them.
In the early stages of an experiment, the relationship between the response and the factors is often not fully understood. In such situations, response surface analysis alone may not be able to find the optimal conditions, as in this example.
In cases like this, consider Bayesian optimization as a viable option.
*Including this example, we have prepared videos that allow you to systematically learn about the features and usage examples of JMP Pro's Bayes optimization.
Bayesian Optimization Fundamentals and Applications: Efficiently Searching for Optimal Conditions with Fewer Experiments (On-Demand Version, Japanese)
by Naohiro Masukawa (JMP Japan)
Naohiro Masukawa - JMP User Community
