nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

DOE: more runs or balanced design?

anne_sa

Hello everybody,

I have the data of an experiment designed with JMP (160 runs). In addition to this design, the experimenter replicated a control treatment 16 times. I wonder if I should include these 16 more runs in the analysis or not. Based on the "Compare design" platform I will increase the power if I include the 16 control runs. However it will create imbalance in my dataset and I wonder if there is any disadvantage of that, and which tool I can use to know that.

Thanks for your help!

Accepted answers

P_Bartell

To add a few thoughts to the comments from @statman :

1. Experimental design imbalance in and of itself usually isn't a showstopper from an experimental design analysis point of view. Especially if it's caused by adding treatment combination(s) as opposed to losing treatment combinations. So read on...

2. You don't say where in the experimental design space the 'control' factor settings reside. Are they within the experimental design space? Or outside? If outside, how far? If outside, what does your process knowledge tell you wrt to expected response values? If the responses are not what you would have expected proceed with caution adding the control responses since you may be in a different operating domain from the 160 run space.

2.a. A bit of a related issue to the above...back in the day, we would include 'control' treatment combinations in an experimental execution event as a means to check if all the noise and nuisance variables in a system (experimental and measurement) were NOT unduly influencing the system in some untoward way that would cast suspicion on the meat of the experiment. How do the control results compare in this regard?

3. When all else fails, why not model with and without the control? Does your answer to the practical problem at hand change? If so, why do you think this is the case?

All comments

statman

Wow, you must have a lot of resources available. Are the 16 replicates of the one treatment run randomly throughout the experiment or how were they run. I don't understand what a "control treatment" is? This data can be analyzed as a time series and possibly to give you an estimate of the MSE (which may be more representative of the the random errors). It is certainly easy enough to analyze those runs separately and the DOE with and without those treatments (they will be pooled into the MSE error).

P_Bartell

To add a few thoughts to the comments from @statman :

3. When all else fails, why not model with and without the control? Does your answer to the practical problem at hand change? If so, why do you think this is the case?

Mark_Bailey

Create two data tables, one is the original design and the other is the original design plus the control points. Of the two tables, the current data table will be the reference for the comparisons. Select DOE > Design Diagnostics > Compare Designs. Select the other table and click OK.

You will see all the usual design diagnostics for a side-by-side comparison.

Plus ideas from @statman and @P_Bartell ...

anne_sa

Thank you very much @statman , @P_Bartell , @Mark_Bailey for your inputs.

To give more details, the "control treatment" correspond to a well-know treatment (the treatment used by default). This treatment is within the experimental space.

Actually during the experiment two 96-well plates have been used and since they were 160 runs, the experimenter decided to put this control treatment in some of the remaining wells (borders of the plates). The idea was to validate that the results of these runs were consistent with the usual results for this treatment, and to check that there were no unexpected effects. No specific issue was detected based on these runs.

So if I understand correctly what you suggest, there is no major blocking point to include these runs in the analysis, but before I can compare the two designs with the Compare Design platform?