nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

DOE without replicate runs not working

VladimirB

Hello,
I encountered a problem when trying to create a custom design without replicate runs.
I have 3 variables, all three are Discrete Numeric with (7,15, and 7) values.
When I enter the Custom design I enter my disallowed combination, model, desired number of runs (30), check that the number of replicated runs = 0 and ask to make design.

The resulting design has replicated runs (in my case it 8 out of 30).

Amy suggestions on how to avoid that?

The JMP file is attached. I use JMP 18.1 Regular, not PRO.

Thank you very much for your help and suggestions!

Vladimir

Custom_Design_for_Vlad.jmp

Accepted answers

Dan_Obermiller

It sounds like you are trying to do something different than the typical designed experiment. So my note will have two parts: one about the model terms and the other about creating a possible design.

Part 1) Model terms

I will try to explain the "necessary" and "if possible" model terms for your situation. Keep in mind that all optimal designs (which is what custom design creates) are about estimating a specific model and will use as few factor levels as possible.

Let's start with a single discrete numeric factor, a quadratic model, and 6 runs. For simplicity, suppose it has five levels: 0, 0.5, 1, 1.5, and 2. JMP adds the cubic term and the quartic term to the model, making them "if possible" in order to specify the maximum number of levels (5). You need a fourth order polynomial to get to 5 levels, which is what I have. Because I only wanted a quadratic model, the "extra" levels are still not needed. The design only uses levels 0, 1, and 2. But if I increase the number of runs to say, 10, I will get designs that start using the other "discrete levels". So, if enough runs are allotted, you will get runs that allow the estimation of the "if possible" terms. JMP will ALWAYS strive to get a design to estimate the "necessary" terms first. The "if possible" is kind of like the icing on the cake. If you can estimate it, please do.

Part 2) Design creation

From your description, you are not looking for a model-based design. That is where the issues occur for you. You have some other criterion for choosing runs. A couple of options that MIGHT work for you:

What if you were to choose a space-filling design, but add a dummy continuous factor so that the design algorithm works? I tried that for your situation, and I get a pretty reasonable distribution of the discrete numeric factors. I just ignore that extra continuous column. There is no guarantee of no replicates, but it is unlikely given the large number of combinations and the small number you are choosing. This approach seems most promising to me.

The second approach is to create a Classical > Full Factorial design. In this platform you can specify continuous factors with the desired number of levels and what those levels are. The design is created. Now create a column for your constraint and delete the rows that do not match your constraint. Now go into Custom Design, but do not specify any factors. Instead, click the "Specify Covariate Factors" button and point to the modified Full Factorial Design table. You can now specify the model and number of runs. Again, no guarantee of no replicates, but this approach should work, too.

VladimirB

Thank you again, Dan!
In addition to thank that I posted to your solution below

And your solution of using a Full Factorial table actually pointed me to another possible solution: After the Full Factorial design with all the levels is created, it is possible to use Full Factorial design in the Custom Design and use all factors as Covariates and make JMP select the desired number of points using some optimality criterion.

Thank you!

All comments

Dan_Obermiller

It turns out that JMP is actually working just fine. Let's back up a bit.

Why did you make all of your factors Discrete Numeric? When you do that, JMP will try to use EVERY level of a factor. In order to do that, it will add model terms that are "necessary" and "if possible". That will force every level to be used (or as close as it can). However, you removed all of the "If Possible" terms. If you had left the model as JMP originally specified it, you would not have any replicates.

Because you DID remove the "if possible" terms, JMP realized that not all of those levels are needed to fit the model that you specified. Therefore, it ignores all of the other levels. It picks the ones that provide the best I-optimal design, and that means that you will get replicates with the desired number of runs. You can see this because you are not getting all of the different levels of those discrete numeric factors.

I am guessing that you wanted to make all of the factors discrete numeric to ensure each factor is used at all of those levels. But that is a testing mentality, not a designed experiment mentality. For example, if you want to fit a quadratic model, no more than 3 levels are needed to estimate that curve. You then use the model to determine the best setting of the factor, which may or may not be a level that has been tested. Models can do that. There is no need to test more than the three levels. (Yes, models will need to be verified, but this is what makes designed experiments more efficient).

My recommendation is to make the factors continuous and create the design with your desired model. It will save you experimentation and should still lead to good results and allow optimization. I hope this point of view helps somewhat.

Dan_Obermiller

One more thing. The replicates box on the designed experiment dialog is a way to force replicates INTO the design. It does not guarantee that replicates will not be used, as you have seen. JMP will replicate when possible as long as the replicate run(s) will improve the optimality criterion.

VladimirB

Thank you for your reply, Dan!

I agree with most of your considerations. And use them myself too

I will try to follow your suggestion about continuous factors and see if I make it work using rounding outside of JMP (means - extra work)

My goal was to make a space-filling design of a sort.

My expectation would be that JMP would at least warn me that it could not (or didn't want to) fulfill the condition that I explicitly requested - no duplicated runs. And ideally tell why and how I could go around it.

I agree about small number of runs for designed experiments utilizing smaller number of levels. At the same time there are situations when "space filling" designs are needed. For a variety of reasons. For example, when fitting a model is a secondary goal. Unfortunately in JMP such designs are only available if at least one continuous variable is present. If there are no continuous variables - we are back at custom design with selecting the model to fit.

I think it is a disadvantage in JMP, as I am sure some type of optimization algorithms are used inside in both space filling and custom design cases and could be extended from one to another. But that's another discussion.

Also I don't quite understand the logic behind your explanation of "necessary" vs "if possible". Specifically, if I say that quadratic term is "if possible" for a variable, to my mind it means I am giving JMP an option to use 2 levels instead of 3 and allow avoid fitting the quadratic term. Thus, reducing the number levels used by JMP even further. And that is not what I want.

Ideally, I would have liked to exploit JMP capabilities to distribute the various discrete levels among different runs in various combinations. Not necessarily all the discrete levels, and definitely not all possible combinations.

I agree that this is along the lines of "testing everything" mind set, but with the DOE mentality to reduce all possible combinations, if you will.

VladimirB

Thank you for the clarification, Dan!

I didn't know about the "recommendation" and not "enforcing" powers of the replicated box.
I wonder if there is there way to enforce it.

Thank you again for both of your answers and for taking time to go over my questions.

statman

There is no description of the actual situation, so providing specific advice is impossible. I do have some questions though, you have 3 factors; 2 at 7 levels and 1 at 15 levels? What model do you intend to test? Do you really need to fit a 14th order polynomial? How would you use that? How do you intend to estimate the MSE? Are you sure your design space is around the "true" optimum? What strategies have you used to understand the "noise"? Pardon my comments, but it does seem like a strategy to "pick a winner" rather than understand causal relationships.

VladimirB

Thank you for the comments, Statman.

The goal is to generate a space-filling design.

I don't need to fit any model. As well as I don't need to optimize anything.

I had to use a model, because optimality designs require some model. Specifying 14 levels doesn't necessarily equate to a need in a14-th order polynomial. That is a too simplistic interpretation.

And I am not familiar with a strategy to "pick a winner".

Dan_Obermiller

It sounds like you are trying to do something different than the typical designed experiment. So my note will have two parts: one about the model terms and the other about creating a possible design.

Part 1) Model terms

Part 2) Design creation

VladimirB

Thank you very much for your great explanation and suggestions, Dan!
I appreciate it very much!

My mistake in understanding was to restrict the "if needed" terms to quadratic. And you explained everything very well. Thank you!

Both of your suggested solutions are very appropriate. I will try them both. And will keep them in mind for future similar problems.

Thank you again!

VladimirB

Thank you again, Dan!
In addition to thank that I posted to your solution below

Thank you!