nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

How are you planning on showcasing Bayesian Optimisation?

Ben_BarrIngh

Hi folks,

Curious to see how people are planning to pitch and position Bayesian Optimisation in their SE workflow, are you going to include it in DoE sessions? Link it into production improvement? Show it everywhere/anywhere? Where do you see yourself talking about it?

Will be interesting to see where people think it will 'fit in' to the way they currently work/talk with customers and prospects, or if a new way will come about!

Thanks!
Ben

Accepted answers

All comments

Phil_Kay

Very good question, @Ben_BarrIngh .

It would be interesting to hear what we have learned from talking to customers about it so far.

I've talked to some customers that are already interested in the method. I've also talked to some of our bigger customers where BayesOpt can provide additional value and/or be a reason for more Pro. Positioning BayesOpt in both of these situations is relatively simple. And those should be our priorities.

I have not talked about BayesOpt with any of our very large number of small customers (remember our median number of JMP users at an account is somewhere between 2 and 3) or with any prospective accounts. How do we talk about BayesOpt in those situations? Should we even talk about BayesOpt with new prospects?

@Ben_BarrIngh - can you give a typical example of the kind of accounts that you are thinking about?

Ben_BarrIngh

Hi @Phil_Kay ,

Thanks for the reply - the areas I'm particularly interested in how we start BayesOpt conversations are:

A) New customers who want to simply improve their experiments - do we offer out BayesOpt in the same breath as DoE? Do we focus on the end goal (understanding vs. reaching an optimum) and then distinguish? What value statements can we make (metric based, time saved, resource % reduced) that help make a convincing argument?

(On that note, @malcolm_moore1 found this infographic https://www.active-learning.uk/2024).

Current customers who know DoE - For customers who know DoE, but aren't Pro customers - what pains do we look to target that brings in BayesOptimisation to the conversation. If we're running a DoE inspire - do we talk about the DoE tools and then jump into Bayesian Optimisation? Or does that confuse things? Do we rename our inspires to 'Modern Experimentation' or similar to get the framing right?

Thanks,

Ben

Lisa_Daroque

Hi guys,

from what I have heard from customers so far, not all find sequential experimentation feasible. Some of them have a time constraint and prefer testing in parallel, even at the expense of a couple of more runs.

I would talk to customers about Bayes Opt if I knew that either their runs are very expensive/done with raw materials that they do not have enough of, or if the underling model is very complex. If time is a concern and they can not do their runs in parallel (e.g. due to a lack of equipment), then I would also talk to them about Bayes Opt.

Many come to us with DoE in mind. If the criteria above are met, I would include Bayes Opt in a DoE demo. As discussed in Amsterdam, we don't want to make it look like a "better alternative" to DoE, but rather a method that is more suitable in some cases, where in others DoE would be the method of choice.

HadleyMyers

Hi,

I'd like to add these Situation Summaries that Florian, Stéphane and Owen developed for BayesOpt to the conversation. They've also added an elevator pitch after the summaries that list some of the challenges customers may face for which BayesOpt could be a solution:

High experimental cost limit number of possible experimentation
Expensive or little available raw material doesn’t allow endless experimentation
Time Pressure doesn’t allow for extensive studies
Optimization in running production doesn’t allow failure
Complexity and high dimensionality require systematic approach

malcolm_moore1

Thanks for starting this discussion @Ben_BarrIngh and great to see what we have learned in such a short time. Big similarity between what @Lisa_Daroque and @HadleyMyers shared regarding target problems for where BayesOpt might be appropriate. At a company level both tools (DoE and BayesOpt) are needed -- lets encourage use of the appropriate tool for the specific problem.

Please keep sharing learning experiences with BayesOpt and JMP 19 generally.

Ben_BarrIngh

Hi @Lisa_Daroque ,

Great graphic, that really helps flesh it out. For those with scarcity/resource constraints do you think that we would need to 'suspend' belief with BayesOpt? I wonder if people would see it as 'magic' when we claim it will reach optimums with a small/smaller number of experiments.

Cheers,
Ben

Peter_Hersh

Great topic. I would love to hear more about this as JMP 19 is released next week. I like @Ben_BarrIngh idea of showing Bayesian Optimization in conjunction with DOE. @bradleybowen has workshop materials he will share that have three case studies of varying complexity. He will need to make sure everything works as expected in the production release, but you should have access shortly after the release.

MarcoSalfi

Thanks @Ben_BarrIngh all the other for your feedback and contribution.

I went through all the 6 videos in the page you posted and I remained with a bittersweet feeling;

Five of six come from our customers (if I’m correct):

All of them have a strong interest for BO (that’s good)
All of them are already using heavily open source for BO (that’s bad)

I noticed, as you already said, that they compare BO with DOE (see a screenshot below), so I want to add 2 more questions to Ben’s:

Do we see the risk to address people to BO with a solution already available in open source – already used in their company?
What we can do to avoid it and justify an extra spent for JMP Pro to use BO?

Thx

Marco

malcolm_moore1

Thanks for digging into this @MarcoSalfi . Here are my thoughts regarding your two questions:

Do we see the risk to address people to BO with a solution already available in open source – already used in their company?
What we can do to avoid it and justify an extra spend for JMP Pro to use BO?

I see nothing different to the usual sales situation here where there is often an incumbent solution of some sort, even if it is Excel or pen and paper. If possible try and get an understanding of their pains and what it is they can't fix/accomplish/avoid with current solutions. If they don't have a pain or they are resistant to change then move on. If they do have a pain that we can align to with BayesOpt (or other capability) then try to complete a specific situation summary for their circumstances. In the case of BayesOpt the consequence of not changing will need to be big enough to motivate for paying the license fee for Pro and a situation summary will be most helpful in advancing such opportunities.

It will be interesting to get different/additional views regarding your questions.

LuizaFabreti

My two cents on open source solutions for BayesOpt (or data analysis in general) is that maintaining code that relies on open source libraries is a nightmare.
Especially if you're using libraries developed in academic environments and suddenly there's no investment in the library anymore.
Besides, you need a mix of many libraries, because many of them only offer one part of the solution.

From what I've seen in Python and R for Bayesian optimization, JMP's BayesOpt is a far superior solution.

Bill_Worley

@LuizaFabreti I wanted to add that there are other software options out there, namely Citrene, which offer an optimization approach that is based on Bayesian Optimization. We have a customer who was using this software but never got any meaningful results. They told us they had spent over $100,000, and probably a lot more in reality, to come up empty handed. They have since become beta users of the new Bayes Opt platform and helped drive a few of the innovations Chris and team have since implemented.

So, the moral of the story is that there are probably more folks out there trying sequential optimization than we think, and it never hurts to let them know that JMP now has a built-in Bayes Opt platform in JMP Pro.