74639_en-US

liliconcom

# of DOE Levels

Hi-

 

I got the following question from a customer:

 

"My question we more focused on better confirming the difference between quadratic and linear. I don’t expect it to be cubic or higher. This mainly comes from a concern about error in my experiments, where I don’t want to erroneously conclude that a correlation is quadratic because a few of the sample points are outlying enough to fit a quadratic better than a linear fit. Let me know if this makes sense or I am misunderstanding. To me, having 4 or more levels would provide stronger evidence of curvature. This would make me feel more comfortable concluding that curvature is not associated with the randomness of my test."

 

Essentially, he wants to know if he should add more levels (more than 3) to have more confidence in whether he has a linear or quadractic model.

 

My answer to him, although I told him I would check with other colleagues, he should:

 

1) Look a the graphical relationship in the Prediction Profiler to see if there was curvature

2) Finally, to be sure if there is a statistical relationship, to look at the Effects Table, and if the quadratic is statistically significant, then keep it in.  

3) If you have the resources to run additional runs, then give it a go! 

Does anyone else have any different thoughts?  

Re: # of DOE Levels

Hi Alisa,

 

I should note for the data table you added, you may find Evaluate Design complains (it looks at the Model and Disallowed Combinations, and can't find the A-F) - it may be easier to remove those if you look at Evaluate (maybe we'll have to see about addressing that for JMP 15...)

 

I like Peter's idea, to try a weighted A-optimal design.

 

I'll also add, sometimes I like to use the orthogonal polynomials to check quadratics (jsl: ortho poly), as sometimes using the simple square for the quadratic leaves it underpowered. That is, I'll take something like the column A and break it up into to columns (one for the linear component, and one for the quadratic), and use that for modelling. They'll may prefer to go back to the simpler model after using the original coding, but it can help point to quadratic effects that may be there.

If you have the Wu and Hamada book, they talk about it in Chapter 1 - at least the first edition did. 

 

Cheers,

Ryan

Re: # of DOE Levels

@phersh Absolutely!  Here it is.  I created some fake data to show him how to analyze it in JMP. 

 

Re: # of DOE Levels

Alisa,

 

Would it be possible to share the file that you and Don created?

 

Also have you considered a weighted A-optimal design with putting the focus on the quadratic effect(s) of interest?

 

I would think that more runs at end points and center points would give you a better feel for if there is truely an active quadratic effect.  Instead of picking 4 levels in between the two end points.

Re: # of DOE Levels

Honestly, it sounds like an awful lot of work was done! I'm not sure I understand why, but that is not important for now. I'm sure that you and Don did something that was appropriate. As for evaluating the power, I think you still can. Take the data table that you are analyzing and just choose DOE > Design Diagnostics > Evaluate Design. You specify the factors and model and it should do the evaluation. It does not need to be something created by Custom Design.

Re: # of DOE Levels

Yes.  I thought about Power after I created the post.  With his particular DOE, I cannot check the power (or at least I don't think I can).  He has 6 factors, but only 3 of which he wanted to use at one time. With the help of @DonMcCormack, we created through CD a DOE using X1->X3  to represent the 3 levels as discrete numerics & X4->X6 as categorical factors with 6 levels. The six levels of X4->X6 represented the 6 factors. We created the design & it had 36 runs.  We then used the Disallowed Combinations Filter to break any choose 2 of the same factor situations.  Then we had to do a Multiple Series Stack to stack the levels and factors & then a split to put them in the appropiate columns.  Then we added in the response column & I showed him the way to analyze using Fit Model since the Model script would not work. Does this make sense? So, at the end of the day, the DOE isn't one that can be evaluated with the Power Analysis.  I don't know of a way to check the Power of this "by hand/brute force" DOE.  Am I missing something? 

 

Re: # of DOE Levels

You can check to see which is better. Essentially, he wants high power for the quadratic terms. He really needs to know if they are real or not. The default design is a D-Optimal design, but an I-Optimal design is better for estimating quadratic terms, so he should be using an I-Optimal design (slightly better power on the quadratics, but lower power on the main effects and interactions so there is a trade-off there).

 

I created an I-optimal design for one factor (the results will hold for more factors) and a quadratic model, 12 runs. This will result in a 3 level design. The power analysis looks like this:

 

Now I create an I-optimal design in 12 runs with 4 levels. I have to choose Discrete Numeric to do this, but as long as I still only estimate the quadratic model, that is fine. Here is the power analysis on this design:

 

As you can see, you actually get LOWER power on the quadratic term when adding more levels (you do increase the power on the main effects). We have a demonstration similar to this in our custom experimental design class. You have a better chance of detecting the curvature by using fewer levels. 

 

Here is the overall comparison of power PLUS, I added the Fraction of Design Space plot. The 3-level design provides better power on quadratics (not main effects) AND it provides better prediction variance. Overall, I think 3-levels outperforms the four levels when it is only a quadratic model.