809872_en-US

liliconcom

A well-plate DoE question on blocking

Hi all,

 

I have a customer question I am not so sure about what the correct answer is. Here is the situation:

- Biotech Process, where they try to optimize a complex buffer formulation

- Factors: 20 ingredients + Temperature

- They can afford 3 well plates, with 84 wells for the experimental conditions

- they have one linear constraint, some interactions and quadratic effects

- Temperature can only be varied per well plate.

Now here is the question:

Since it is a DoE factor, how would that be seen from a statistical point of view? Blocking Factor, Hard-to-Change Factor? On the one hand, they are of course interested in Temperature as regular DoE factor (fixed effect), but it only has 3 repeats. And these 3 repeats might be confounded with plate-to-plate variability.

And what would be the appropriate way to analyse the results then?

 

I hope this makes sense

Thanks

 

Florian

 

 

Re: A well-plate DoE question on blocking

Byron knows more about this than me - but with plates you also need to understand the logistics of the lab to design a "good design" that is also executable. Often you need to use a "good design" rather than the "best design" to have an experiment that can be executed. For example, is this a manual or automated experiment in terms of plate loading? Often you will need blocks within the plate as a totally random experiment (without automation) would likely be very hard to implement.  

Re: A well-plate DoE question on blocking

Why not try a 252 run split plot design with Temperature as a hard to change factor and making all the two factor interactions If Possible? This will make the analysis easier since you wouldn't have to add a whole plot column and random effect to the model as would be the case if you treat Temperature as a block. I bumped the number of starts to 10 and it took a few minutes to complete. This also gives you the option of adding constraints.

Re: A well-plate DoE question on blocking

Morning @Byron_JMP ! Thanks for your detailed response, I like the approach and I like the push for Pro.

Two limitations though: 1) DSD's don't accommodate the linear constraint and 2) the option to add more than 8 extra runs is only EA

 

What I was able to generate so far, is a 252 run RSM design with Temp as 3 level blocking factor.

The Split-Plot-Design with Temp as Hard-to-change didn't converge after a few hours.

It did converge by creating random blocks (group runs into...).

Following your approach, of starting by assuming temperature effect is much larger than plate to plate variation, to me the easiest way to go seems be a 252 run RSM design with Temp as 3 level blocking factor, and then using your analysis approach with GenReg. 

 

What do you think?

 

 

Re: A well-plate DoE question on blocking

I think I might disagree with everyone's answers so far.

Temperature is likely going to have a far larger effect than anything else, even the plate to plate variation independent of temperature
I would set up the experiment exactly the same way in each of the three plates, and then run each plate at the desired temperature. 

With only 84 wells and 20 factors, they are looking at a definitive screening design 

this will give you 72 runs with 12 center points (all zeros)
On the plate this means rows 1-7 have experimental runs and row 8 is open for controls and such. There will be one center point in each of the 12 columns on the plate. 

Then in the analysis, the blocks are kind of irrelevant (in a practical sense), so delete ":Block" from the model script

The analysis for each individual plate will let them discover with factors and maybe with higher level terms are important.  With all the extra runs, they will be able to estimate a lot of the quadratic and interaction terms if they are there. 

 

For the final concatenate all three plates, being careful to create a source column.
Recode that source column in to a continuous numeric temperature value.

Go into fit model, select all 20 variables and temperature, click the macro button for RSM
Choose personality = generalized regression and then use a two stage forward selection

If the don't have jmppro, sell it to them, because honestly, if they really think they have 20 factors that actually do something they'll buy anything, even if they only have budget for three plates.  They should have spent a lot more time deciding what they needed to experiment on. 

back to the real answer - 
If they don't have pro, then use the Fit Definitive Screening (:Block deleted) for each temperature. Compare parameter tables to see if the results are different across temperatures.    Fit Definitive screening will help figure out which variables aren't important.  
In the combined data table, use fit model and  RSM, but only on the active factors, including temperature. 

The key detail here is that they are working on a 96 well format. It's more important to have an executable experiment than to have a perfect design.  (The perfect design would have around 808,182,895 runs) 

It's also super important to sell them on the idea that they will need a follow up experiment to tune the factors that were actually important. 

 

Re: A well-plate DoE question on blocking

yes, that's true. they calculated 3 plates is feasible for this task (CDMO, they want to sell this as a procedure). 

So you could argue plate as random effect (and as discussed even well location), but they also want to investigate temperature, which they can only adjust per plate.

Re: A well-plate DoE question on blocking

Thanks @russ_wolfinger @Chris_Kirchberg @DonMcCormack,

makes much sense already. I spoke with him about randomization across the plate and he assumes no effect from the well location.

So to make sure I understand the suggestion for the setup of the design. In custom design, he would create the design by

- Specifying the 20 continuous factors

- Specifying Temp as continuous factor, but hard-to-change

- Set the linear constraint

- Add the model effects

- Set the number of whole plots to 3

 

Correct?

Re: A well-plate DoE question on blocking

Sorry, I was not thinking straight in the morning. @DonMcCormack has a more correct answer than I, but I still would not do temp changes with only 3 plates. one temp per plate or two plates for one temp and one plate for one temp might give a general idea that there is an effect, but as Don says, the power would be so low, there would not be any real confidence in that difference.

Re: A well-plate DoE question on blocking

I agree with @russ_wolfinger that your dealing with a somewhat complex split-plot design. I would treat temperature as a hard to change factor. With only three well plates, the power for temperature will be very low, possibly so low only a gigantic effect will be statistically significant. That said, it's still good to do the statistical test to keep from concluding there's a difference for something large but within experimental noise. While the most sound approach is to treat the rows and columns of the plates as random effects, it has the potential of complicating the analysis for the user. If they're comfortable with this, go for it. When I've had to deal with this situation in the past a common alternative was to ignore the wells on the periphery of the plate and assume the variability across rows and columns was negligible, simplifying the analysis by removing the two additional random effects.

Re: A well-plate DoE question on blocking

For clarity, they have 3 plates and there can only be one temperature per plate?  With only 3 plates they would not even be able to estimate variance  or effect for two different temperature settings. They would need at least 2 per temperature. Especially since temperature is the plot/block for the split plot.

Re: A well-plate DoE question on blocking

A good starting approach to help avoid confounding would be to treat temperature as a continuous covariate and specify plate and potentially also row(plate) and col(plate) as random effects.

This is a split-plot scenario and some in depth discussion with examples is in https://www.amazon.com/JMP-Mixed-Models-Ruth-Hummel/dp/195236521X?asin=1951684028&revisionId=&format=4&depth=1 

Re: A well-plate DoE question on blocking

I'm still sticking with the idea of completely replicated design for each level of temperature/plate.

20 factors is impractical for an RSM model both from the required number of runs and the practicality of pipetting out 252 combinations without making a mistake is going to be really rough. Programming the robot to do the combinations might be possible but I don't know of a robot system that can handle 20 components, maybe they have one?

If they really need 20 factors, then a DSD is really the right direction. For runs that violate the linear constraints, leave the well on the plate empty and report the data as missing for the analysis, or just do the run anyway, there won't be enough bad runs to ruin the experiment. For the extra runs, just add dummy variables just like JMP does.  It is exactly the same as adding extra factors. For example, make the design with 20+8 factors, then delete the extra 8 columns. The fold over structure will be there.  The extra runs option was simpler than explaining the dummy run concept. (note: 8 extra factors adds 16 runs.. total runs=factors*2+1. Should be able to detect 10 or 11 active factors and their higher order terms, assuming about half of the 20 factors are active, which might be a stretch).   @DonMcCormack can double check me on this?

  

Re: A well-plate DoE question on blocking

For runs that violate the linear constraints, leave the well on the plate empty and report the data as missing for the analysis, or just do the run anyway, there won't be enough bad runs to ruin the experiment.

Keep in mind that missing runs in a DSD tends to break things (you need to also hide/exclude the corresponding foldover run and/or use something like GenReg to do the analysis).

Re: A well-plate DoE question on blocking

I have two concerns with running the experiment like this. First, I loathe the idea of throwing away runs. It counterproductive to why DOE is better than ad hoc experimentation. Second, with so many two factor interactions and so little information (I'm assuming if they knew more, there would be fewer factors) you run the risk of having a lot of active 2FIs. DSDs perform poorly under these circumstances. I'd treat this as a screening experiment and not worry about quadratic effects. With 252 runs per plate you have enough to estimate all main effects and 2FIs. Given that there are three plates, the power for the 2FIs should be decent depending on the constraints.

BTW, when I try to create this design I wind up killing the design generation after 5-10 minutes. I'm not sure if I'm not letting it go long enough or it just won't work. I'll try it again overnight and let you know what happens.