nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Big DoE

Phil_Kay

Hi,

A customer wants to create a very big experiment using high-throughput technology and 96-well plates.

They have 20 continuous factors and a 3-level categorical. They would like to test the RSM model. They would like to block over 3 plates, so 288 runs in total.

Custom Design can create the design with a couple of minutes of processing time.

Ideally the customer would also like to include constraints such that for one of the levels of the categorical factor certain of the continuous factors can only be at their low setting. Custom Design doesn't like this. Even when you remove the interactions between the categorical and these continuous factors.

I need to ask the customer how important these constraints are.

Any thoughts?

If nothing else, it would be useful to give the customer an impression of the scale of the challenge that they are presenting to Custom Design so that they are not surprised when JMP struggles.

Thanks,

Phil

Accepted answers

All comments

DonMcCormack

20 continuous factors and a 3 level categorical makes for 380 2 fi's. 40 more runs for the main effects and quadratics. How's that going to fit on 3 plates? Also, wells are usually not independent, although they may be using a technology that fills them separately.

Sent from my iPhone

Peter_Bartell

Phil: A couple thoughts:

1. Since it sounds like some effects are inestimable due to constraints, is the customer will to break up the single experiment into maybe smaller experiments where all the effects are estimable within the context of that experiment? Can this even be done with the nature of the constraints for the categorical and continuous factors? Just a thought.

2. If you could do what I suggest above, is it then possible to merge all the experiments into one big experiment again, and then let something like the Generalized Regression platform help find active effects? In essence the single big experiment becomes something similar to happenstance empirical results that have been collected and your focus now is just finding a useful or informative model?

Phil_Kay

Hi Don,

I think it is [k(k-1)]/2 to calculate number of 2FIs. So for the 20 continuous factors = (20*19)/2 = 160. Then there are 20 more 2FIs with the 3-level categorical (each requiring 2 more parameters to be estimated, I think). Plus the main effects and quadratics.

I think it is all estimable in 288 runs. And JMP seems to agree. It is a large number of parameters though!

Thanks,
Phil

Phil_Kay

My recollection is that they can fill the well independently. I will check that though.

Phil_Kay

Hi Peter,

That could work. I think they were hoping that they could just cover everything in one big experiment.

The experiment is a formulation problem. The 3 level categorical represents 3 different kinds of buffer. They know that one of these buffers will not work with certain salts. Phosphate buffers bind to divalent salts and form insoluble salts, which is a bad thing, I guess!

So they want to exclude combinations where phosphate is present and these salts are present. There are 4 factors representing the concentration of the 4 divalent salts that could be present. They would need to these factors to be at their lowest level (negligible concentration) whenever buffer = phosphate.

So I agree it might make more sense to create a separate experiment for this buffer.

Or just create the big design without the constraints and then manually set the divalent salt concentrations to low when buffer = phosphate. Not an elegant solution!

DonMcCormack

My mistake. 40 for main effects and quadratics, 190 for 2 fi's continuous + 20 more for categorical 2 fi's for 250 total. Estimable, but I'd be curious about the power.

Sent from my iPhone

Peter_Bartell

Phil: The more I thought about this I seem to recall somewhere in the recesses of my memory that as you add effects in the Model Specification window for the Generalized Regression platform, the response speed slows down considerably? Or maybe I'm just misremembering? With so many effects that you are looking for, the number of models and associated penalties etc. makes one's computer work really really hard and slows it down? Maybe I'm not remembering this properly, but it might be an issue for my second thought above? Maybe Brady Brady or someone else could comment?

Phil_Kay

Yes, I can imagine that the model fitting could get difficult with all of the 200+ interaction effects in there. It is a good point, as I had not thought to ask how they were expecting to analyse the results. Possibly they will just pick the best one. However, they are big on QbD so understanding the system is important.

Thanks,

Phil

martindemel

Hi Phil,

I'm way from being an expert, but just a thought that came into my mind:

Do they really think all 20+1 factors will be active? Wouldn't it make sense to create a smaller main effects model first to identify the most potential effects and do then a RSM model? So Augment the first design with a second one?

As I said, just a thought.

Martin.

Phil_Kay

Hi Martin,

Yes I completely agree. And, in fact, that is the approach that they would have taken before they had the HTS technology.

From talking to them yesterday, it seems that their attitude is: why compromise? We can easily carry out 100s of runs so we should just go for the RSM design from the start. So they want to create an experiment that meets this objective. And their expectation is that Custom Design should allow them to do this.

They were happy with my workaround, which was to use column formulae to reduce the factor levels to low when buffer = phosphate. But they would still prefer not to have to use a workaround. Constraints like this in these settings are apparently common.

From playing around a bit more I've found that including constraints and blocking does not work. Custom design seems unable to handle both at the same time. So we can block it later. My feeling is that for such a large number of runs and small number of blocks, random assignment to the block will be okay. But I could look at blocking in Custom Design with the other factors as covariates if necessary.

I added one of the constraints and managed to create a design. I increased the number of random starts and the design search time. Not sure which had the desired effect (confounded factors!). Perhaps I need to create a DoE to understand the factors that affect my ability to create this DoE!

Mark_Bailey

Besides blocking, you might consider a split-plot design. I think that the rows and columns on a plate lead to correlation of estimates because the runs are not fully randomized. If the HTT, in fact, allows for essentially complete randomization without regard to column, row, or plate, then disregard my concern!

Phil_Kay

Thanks, Mark.

Yes, I have seen a really interesting Discovery paper describing that approach.

But in this case full randomisation is possible. It is a system by TECAN.

I got a useful response from tech support that I can share once I am back in the office.

Thanks,

Phil

Sent from my iPhone

martindemel

Hi Phil, thow it's felt like a decade ago

do you still remember what support told you in end 2015?

Chris_Kirchberg

3 years later and more familiararity with DSDs, would a DSD in this case make sense to tease out which factors are active and since now in 14 you can do blocking? I guess the contraints is the main reason to do Custom Design though, correct?

Jeff_Perkinson

Phil doesn't have to remember. The tracking system, and its search tool, TESSA, never forget.

Here's his track, #7611649845: Custom design with constraints does not converge for an RSM model.