Random vs Fixed Blocking Factor in DOE

Question

Hello All,

I am trying to setup a custom DOE experiment where I am evaluation 4 continuous factors with 1 response. I have a limit of 30 runs and am interested in main effects (later I would like to do two-way interactions as well if possible). I set this run up but because these 30 runs have to be split over two imaging platforms I use a blocking of two with 15 runs per. I tried with a fixed blocking factor which yields high power and low variance values for the model. However, to me it makes sense to use a random blocking factor because it will free up more degrees of freedom and also from a physical standpoint the two platforms should be very similar and just two random ones out of a population of many such platforms.

However, when I computer this design (I am using D-optimal design) I get good power values for all terms except the intercept which is a whole order of magnitude lower. At the same time the average variance jumps from about 0.1 to 0.7 from fixed to random blocking.

Could someone indicate the reason for this to me and possibly how to circumvent this problems. I always assumed that random blocking gives more power to estimate effects (especially main effects).

Thank you

Ekarshalev · Answer

Hello Mark and Statman,

Thank you for the input. I do believe that my platforms (which are not imagers but a consumable part over which the testing happens) should be set as random effects since there will be many more of them and in theory there should not be significant differences over them that need to be categorized.

The main questions was why I was getting such a difference in average variance and low intercept power with random blocks versus fixed. In general fixed would give more unknowns but in my case they should be the same I believe since I only have two very large blocks of 15 runs in each. For the fixed if I remember correctly you add k-1 unknowns for k blocks and for fixed the only added unknown is the variance between blocks. The color maps of both designs also look exactly the same so additional correlation between factors should not have occurred correct?

Any help with this would be appreciated.

statman · Answer

First, welcome to the community.

I'll share my thoughts with perhaps, a different perspective:

1. How data should be analyzed and what tools you use for analysis is a function of how the data was acquired.  (actually this extends to what questions you can answer, what conclusions you can draw and your confidence in extrapolating the results is a function of how the data was acquired)

2. While the appropriate "designation" of blocks can be debated, if you can specifically assign what factors are confounded with the blocks and as Mark suggests, those "levels" can be reproduced, those blocks can be treated as fixed effects.  I would debate whether taking those randomly (or treating them as random effects) increases the inference space vs. specifically manipulating them at "Bold levels".  I personally think you have a greater likelihood of have a broader inference space by specifically manipulating the blocks.

3. Treating the blocks as a fixed effect has the huge advantage of being able to quantify block-by-factor interactions (essentially noise-by-factor interactions) which is perhaps the best method to quantify the robustness of your design structure.  Are the effects of your design factors consistent over changing noise?

4. Interestingly, blocking has the additional benefit of simultaneously increasing the inference space and increasing the precision of the design (as defined by Box).  This because the within block is biased to the factor effects (the noise is held constant within the block) and then the noise is changed between the blocks so it can be assigned.

"Block what you can, randomize what you cannot" G.E.P. Box

Block what you can identify and manage over the execution of the experiment, randomize for the noise that has not been identified or cannot be managed over the duration of the experiment.

Mark_Bailey · Answer

Whether to treat an effect as fixed or random is usually not a matter of DF or power but how you see the variation. If it is a reproducible level, if the levels are the only ones of interest, then treat it as a fixed effect. If the levels are a sample from a larger population, if you are not really interested in these two particular levels, but variation across the population, then treat it as a random effect.

So how do you see the two imagers? Are they the only two devices? Will they be the only two devices for the foreseeable future? Then treat the blocks as a fixed effect. Are the two imagers selected from a large and ever changing pool? Then treat them as a random effect.