I have a 6 parameters process , all of them are continuous factors. We know how to run a DSD and find opimized setting for these factors. But now the question is, we suspect that the hardware machining error is also important , and we want to evaluate the impact from it So our idea is to prepare 2 hardware equipment, they should be the same except there is some degree of machining error, and we will run a DSD design: X1~X6 + 2 blocks by14 runs (add center run to estimate quadratic effect). So 7 runs will go by 1st equip while the other 7 runs by 2nd equip. Qeustioin as below : * When we do model fitting, we choose response surface to estimate all 1st and 2nd order terms for X1 ~ X6, how about block factor? only main effect should be estimate or quadratic and interactive terms can be estimated? (We don't know if they exist or not at this point, but it is possible) * Our idea is to get a number from the model says that: the machining error account for x% of the model, can we get such information from the model? (Normally we use "access variable importance" function to see the contribution from each factor, but this does not include block factor) * The next step is that we will measure all properties of the hardware and try to find out which machining error is key to process variation. Any comment on this whole procedure? Again, the target is to evaluate machining error impact to process.

nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

How to focus on block factor in a DSD if it is my top interest item

BayesRabbit7133

I have a 6 parameters process , all of them are continuous factors. We know how to run a DSD and find opimized setting for these factors. But now the question is, we suspect that the hardware machining error is also important , and we want to evaluate the impact from it

So our idea is to prepare 2 hardware equipment, they should be the same except there is some degree of machining error, and we will run a DSD design: X1~X6 + 2 blocks by14 runs (add center run to estimate quadratic effect). So 7 runs will go by 1st equip while the other 7 runs by 2nd equip.

Qeustioin as below :

When we do model fitting, we choose response surface to estimate all 1st and 2nd order terms for X1 ~ X6, how about block factor? only main effect should be estimate or quadratic and interactive terms can be estimated? (We don't know if they exist or not at this point, but it is possible)
Our idea is to get a number from the model says that: the machining error account for x% of the model, can we get such information from the model? (Normally we use "access variable importance" function to see the contribution from each factor, but this does not include block factor)
The next step is that we will measure all properties of the hardware and try to find out which machining error is key to process variation. Any comment on this whole procedure? Again, the target is to evaluate machining error impact to process.

Accepted answers

statman

I'm sorry for the multitude of questions. It is extremely hard to provide specific advice when the engineering or science is not adequately described, which I completely understand is beyond the scope of this forum. Most of the equations I ask, don't need to be answered here, but are for your own thought processes. On one hand you are running a DSD which, by definition, is a screening design meant to identify active factors/interactions for further investigation. These typically use bold level settings to exaggerate factor effects. Your reactor issue sounds challenging because you can't exaggerate the volume effect (Or can you?) Since the DSD experiment looks at the relative effect of the design factors, and their levels are exaggerated, it can be challenging to compare the relative effect of a factor with a much narrower level setting. In this context, the comparison may be biased.

I will continue the beatings until the morale improves...

These reactors seem like micro-reactors. Are these for production or development? Is the reaction ego or end thermic? How is the temperature managed? How is it the volume of the reactor impacts the performance measures of the material being made in the reactor? How much space is need for the reaction? Does the reaction happen in the reactor at all places in the reactor simultaneously? Have you assessed/measured within reactor variation? Why would you have to spend the money to fix the supplier's issue of variation in reactor? Is it possible to have different processing factors and levels for each reactor?

"manufacturing variation of this reactor is +/- 0.1mL". what is +/-, is this the total distribution or 1 standard deviation? What I don't understand is, for each reactor, the "volume" is not changing, correct? It is only that you have additional reactors that there is a potential effect of the volume, correct? So in your first experiment (the DSD), you did these treatments in one reactor, correct?

Are you concerned the effects of the factors in your initial DSD experiment will change if the reactor volume changes? (this is, BTW an interaction of reactor and design factors)

As I see it, here are some options for you:

1. Run the second replicate of the first DSD with the second reactor (make sure all of the levels set in the first replicate are "identical". There is no need for center points as the reactor is not a continuous variable. Model the design factor effects, reactor and 2nd order design factor by reactor effects.

2. Record a measure of the actual reactor volume for each treatment of the first DSD, run a replicate of the DSD with the second reactor and record the actual value of the volume. Treat the volume as a covariate in your analysis.

3. Treat the reactor as a whole plot factor and the DSD as a sub-plot of a split-plot design.

4. Take the results of your first DSD and sample those "preferred settings" over multiple reactors where within reactor is within subgroup and between reactor is between subgroup.

"So currently we just ignore the confounding."

My advice is never to ignore the confounding, but identify it. If the confounding is specifically identified, strategies to un-confound are more readily found.

statman

My original plan was created a 13+1 run DSD table, and 7 of them by 1st reactor while the other 7 by 2nd reactor.

OK, sorry for my first suggestion, I did not realize you had results over both reactors. I'm not sure whether you have sufficient DF's to add the reactor to the model. Without the data table, I can only guess...Make sure you have a column called reactor and put a 1 for one reactor and 2 for the other. You might try analyzing the design factors of the DSD, removing the insignificant terms, rerunning the model adding reactor to the model.

Treat the volume as covariate, then what should I do in JMP ?

If you had run the experiment over multiple different reactor volumes, you could analyze the volume as a part of a mixed model. The DSD factors are considered fixed effects and the reactor volume would be the random effect. Proceed with typical covariate analysis.

I look into split- plot design from JMP help, it seems that this is created from custom design with very hard to change factor, how can we include a DSD into a custom design?

Actually there a different reasons to run split-plot designs, only one of which is the difficulty in changing factor levels. But regardless, you would assign reactor as the hard-to-change factor, this putting it in the whole plot. The design factors would be in the sub-plot. TBH, I'm not sure whether JMP handles this appropriately. I think you might be able to get by with adding the reactor as a categorical factor, create table and sort by the categorical factor. Analysis will take some help.

Not sure the actual operation in JMP for this comment

It is not an operation in JMP, but an iterative plan to study the process.

Definitive Screening Design.jmp

All comments

Victor_G

Hi @BayesRabbit7133,

The first thing you need to assess is the type of blocking effect you want to account for : fixed or random.

Fixed blocking factor are factors that you expect to have an influence on the mean response, and whose levels are defined and "fixed" (only 2 equipments in your case). It also means you can have different predicted values based on the machine chosen.
Random blocking factor are factors that you expect to have an influence on the variance of the response, and whose levels are only a subset of all possible levels (population). In your case, you might be interested by the change of variance depending on the equipment, and these 2 machines are only a subset of all machines available.

Regarding your questions :

Blocking factors are not involved in interactions or quadratic effects, they are supposed to have only an influence on the intercept or variance of the model, hence you can only add main effect for blocking factor. See the more complete response from @Mark_Bailey here :Solved: Re: Block effects in JMP - JMP User Community
Yes, if your blocking effect is considered as random, then you can check the percentage of variance accounted by your blocking effect, you will have this information when fitting a mixed model, under "REML Variance Components" panel (see Pct of total):
One option could be to consider all process parameters of your machine (completely map/describe your equipment as a system with all inputs/outputs), and use data analysis on historical data (data mining approach) and/or create a DoE to better assess how each machine factor contribute to the variance and mean result (it can also be helpful to validate the findings of the data mining). You may need to repeat the measurements for each condition in order to model/analyze the mean response and variance response.

Hope this answer will help you,

statman

I have some questions, comments and clarifications.

"We know how to run a DSD and find opimized setting for these factors"

DSD is a screening design not meant for determining "optimum settings".

"we suspect that the hardware machining error is also important". I'm confused by this phrase. What do you mean by hardware machining error? If you know it is an error, why not proceed to understand it and perhaps "fix it"? Is this a mechanical machining operation? Are you are calling the variation in this operation error? How do you set it to different levels? Why not find out what factors affect this variation? To do this you will need to model the variation, not the mean.

There may be some confusion as to the use of the term block. As Victor points out, the block can be treated as a fixed or random effect. What differentiates these designations is the degree to which you understand what is confounded with the block. If you have identified all of the factors confounded with the block, then it would be reasonable to treat the block as a fixed effect. What is key is the factors confounded with the block are considered noise to the investigator. That is, factors that one is NOT willing to manage (for technological, economic or convenience reasons). If you have not assigned the factors to the block, the block is typically considered a random effect. I believe you can be more effective and efficient in understanding your process and robustness (the absence of noise by factor interactions) if you do the due diligence of assigning and confounding the noise in the blocks.

"Block what you can, randomize what you cannot" G.E.P. Box

"only main effect should be estimate or quadratic and interactive terms can be estimated?"

It is nonsensical to estimate a non-linear term associated with the block. It is similar to a categorical factor and since it is not continuous, a quadratic term makes no sense (there is no center point). However, a block by factor interaction is indeed possible. This is actually a very important piece of knowledge. It answers the question; Are the design factor effects consistent over changing noise (represented by the block)? This is a very effective way to investigate robustness of the product or process design.

"Our idea is to get a number from the model says that: the machining error account for x% of the model, can we get such information from the model?"

I'm not sure about a direct calculation to determine the % that the machining error contributes, but you could calculate and compare R-square Adj with the block and block by factor interactions in the model and when they are not in the model. But why would you do this? If the block is significant, what will you do? Investigate the machining error more thoroughly...If there are block by factor interactions, what will you do? This is an indication that factor effects depend on machining error, these will need to be further studied.

If the block or block by factor interactions are not significant, will you investigate the machining error anyway?

BayesRabbit7133

Let's assume I am doing chemical reaction in a very small reactor. The process parameters include weight of reactant A, B, C, temperature, reaction time, UV light intensity, so there are 6 factors in total.

Now the reactor is designed to be 1mL size, but the actual reactor I got may be 0.9ml, or 1.1ml, or any possible number around 1mL. So in theory, the reactor size is continuous number, but it's expensive and we only have 2, 3 reactors on hand for now, so I'll put it as block factor.

During the whole complicate chemical reaction process, there is gas phase reaction in the middle, so the size of the reactor should have influence on gas concentration, and may eventually impact the final yield. So that's why we suspect there might be interaction between block and process parameter.

If the small difference in size is important, we will want to control the size more precisely, for example, +/- 0.01mL rather than +/- 0.1mL. This won't be easy to achieve so we need to make sure it's important before we proceed.

Based on the application, we will use different reactor size, and the required process window is not the same, so the sensitivity of reactor size may be different in each case, for example if it's a 10mL size reactor, +/- 0.1mL should not be important. But how about for a 2mL size reactor? when will it become important enough that we should start to control it better

statman

As you describe the volume of the reactor, that appears to be a controllable factor (although it may be costly). Not even sure why you call it error?

Blocks are intended for noise factors (Factors you are NOT WILLING to manage in the long-term and in the future.). The purpose of blocking is to simultaneously increase the inference space (by varying the noise to make it more representative of future conditions) while improving the design's precision (by partitioning and assigning the noise, hence lowering the MSE) . If you are running a RCBD, confounding the volume with the block increases the resolution of the block effects and confounds the volume with other noise (reactor-to-reactor, set-up, batch-to-batch of raw materials, etc.) Why not just include volume as a design factor?

BayesRabbit7133

The size of reactor is not a controllable factor, it is designed to be 1mL for some reasons (not process related), we cannot change it.

I mentioned there are different size reactors previously, those are for different application / product, and same as the 1mL reactor, they cannot be treated as process factor either, in each case, the size is always fixed.

Back to the 1mL size case, it is designed to be 1mL, but the manufacturing variation of this reactor is +/- 0.1mL currently, so we call it error. We want to evaluate the impact from this hardware variation, if it's important, we may need to spend resource try to improve the manufacturing variation to, for example, +/- 0.01mL

And one more thing, the reactor is super expensive, each reactor can cost our entire year budget, so you cannot possibly get a lot of reactors and form a process window for its size.

As for the counfouding you mentioned, Yes, there will be other reactor related noise counfounding with the reactor size (reactor-to-reactor, set-up. etc). Many of them are not measurable, but the size is mearsurable and should be the most important one based on our knowledge. So currently we just ignore the confounding.