D OPTIMAL DESIGNS

Question

Hello,

I am a JMP user and I have a doubt about
how JMP finds out the significance of the
effects in DOE after performing a D OPTIMAL
design.

If for instance I have a 16 run design and it turns out

that I can only perform 13, then I can use D-OPTIMAL

to find out which ones to skip.

The issue is that I don't know if the only way to

asses significance to the effects is using

a p-value through the ordinary least square

regression.

If that is the case I see it very simplistic,

since there are bayesian alternatives

to do that, more over we can estimate

the missing values.

I will really appreciate feedback on this topic.

Thanks.

rafel · Answer

Hi Dan,

Thanks a lot for the answer. You insited in having

HUGE caution when it comes to imputing missing

values.

I have tried with JMP but it does not give the

reasonable option to imput these missing

values estimating them using the contrasts that can be

considered null, such as, 3 and 4 factor

interacions.

What do you think about that way of imputing missing values? Because

George J.P. Box insisted that this is a good way to

imput the missing values up to 2, maximum 3.

These values are "made up" but using a different and

maybe appropiate technique having less influence

on the model.

I will appreciate your opinion about that.

Thanks again,

Rafel

Dan_Obermiller · Answer

I think you are misunderstanding how D-optimal designs work. D-Optimal designs can only be created for the model that you have specified and are not to "choose runs to skip".

For example, suppose I have 4 factors and wish to fit a model with all main effects and all 2-way interactions. That will give me 11 terms in my model (including the intercept). A D-optimal design will require at least 11 runs (that is the minimum number of runs JMP offers in the design creation process). This is a requirement because The D in D-optimal refers to a determinant of a matrix that needs to be inverted.

Now, if I take that same situation but list all of my 2-way interactions as "If Possible" rather than "Necessary", you will see the minimum number of runs drop to 5 (only intercept and main effects). When doing this you are actually telling JMP to create a Bayesian D-Optimal design. In such a situation you cannot fit the model with main effects and all 2-way interactions. You will need to use something like forward stepwise regression (or some other techniques that JMP offers) to determine which of the 2-way interactions are considered "active".

Now perhaps your question is more about a 16 run D-optimal design that allows estimation of all main effects and all 2-way interactions, but for some reason 3 runs are not completed. For this situation you have 13 runs, which is more than the 11 runs required. Therefore, ordinary least squares regression will work and provide testing for all of the parameters. Unfortunately, the design is not optimal for 13 runs, it is optimal for 16. Therefore, you are likely to have higher multicollinearity of the parameter estimates and more variance of the parameter estimates. This is not a JMP limitation, it is a limitation of least squares regression and not completing the design as originally laid out.

JMP offers alternatives in such situations. You can impute missing values. You can try alternative modeling techniques that handle multicollinearity and missing values differently (PLS, Neural Networks, partition models, etc.). If you go to JMP Pro, you have even MORE options for fitting models to handle these situations differently.

A HUGE caution though, especially when it comes to imputing missing values. If this is a designed experiment, each trial is chosen to have high influence on the results. After all, you are trying to get maximum information for minimal number of data points. I would NOT recommend imputing values for a designed experiment because there is not much data to determine an appropriate structure to accurately impute the values and that "made up" result will have lots of influence on the model.