nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Factor inherent variability when using the Profiler simulator

Hugo_JOHAN

Dear community members,

I have a question regarding factor inherent variability when using the Profiler simulator.

Let’s say I would like to define the acceptable factor range that enables me to always meet the predefined acceptance criteria I set.

Let’s also say that this factor has an inherent variability, meaning that when I want to set the factor at 2, the true value is between 2 +/-0.1%.

I run a DoE to get a model and the related transfer function.

I launch the Profiler simulator so as to define this acceptable factor range:

I select the distribution as random
But which standard deviation should I state: I should take in account the model RMSE but also the factor inherent variability ? Or the RMSE only considering that it already includes the factor inherent variability, meaning that the model already captured this inherent variability in the model error ?

Thanks,

Hugo

@martindemel

Accepted answers

martindemel

"then I understand that indeed the model error is not necessarily taking in account the factor inherent variability ?"

The model will only take this variability into acvcount if it is already within the data. So yes, the model error does not take into account the additional simulated error you set. The simulation allows you to add additional noise you may know from experience or technical details of the response measurements/instruments/input variation ... (e.g. ph will most likely not be exact equal 5.5, you usually have a small variation even it you set it with the instrument to this value.

That said, the answer to your second question: If you replicate your design, you would have more data for the "same settings". This will allow a better estimate of the error. So the model probably is more accurate. However I would not necessarily say that this therefore includes the inherent factor variability. From a conservative point of view I would still want to know the worst case scenario. Then you can - together with your expertise - judge over your model how likely these scenarios are and if a smaller variation differs a lot from the conservative approach. Unfortunately it is still an approximation but probably a (slightly?) better one than before.

All comments

martindemel

Hugo, when I want ot be sure that my simulation takes into account your factor's inherent variability in addition to the model error, I would set the distribution of the simulation to that range (2 as mean, std 0.1%). If I can be sure that the variability will not be more than .1% in your example then I would use truncated normal or univariate to capture the whole range of variation rather than using just normal.

As the goal is (as I suggest) to gather the largest factor range to stay in your acceptance criteria it always is better to be rather conservative than too optimistic.

My2cents

Others may have additional input from their daily work and usage.

Hugo_JOHAN

Hi Martin,

Thanks for the input.

I do agree with your comment about the conservative approach.

When you state "when I want to be sure that my simulation takes into account your factor's inherent variability in addition to the model error,..." then I understand that indeed the model error is not necessarily taking in account the factor inherent variability ?

In case I would replicate the entire design, then I would have more confidence that the model error is taking in account such an inherent factor variability ?

martindemel

"then I understand that indeed the model error is not necessarily taking in account the factor inherent variability ?"

Hugo_JOHAN

Many thanks Martin for the nice discussion on the topic.

Mark_Bailey

Just to clarify, you specify the factor variability, if any, for each factor. The RMSE is the random variation in the response that is not associated with changing factor levels. So you add the RMSE separately. That way both sources (fixed and random) are included in the simulation.

Hugo_JOHAN

Hi Mark,

Thanks for your input. So in the profiler, selecting the random distribution, I set my set point value as the mean and then the factor variability as the SD, is that correct ? Finally, I would not put the RMSE as SD of a factor as, as you mentioned, the RMSE is the error related to the response and is already captured in the model so in the confidence interval of each simulated Y value ?

Mark_Bailey

Here is an example of using the simulation feature in the Prediction Profiler:

Notice that I specified random variation for age and height but fixed sex. The target value for the process variables is the mean. The SD is the expected variation. So I want to simulate what happens when the height = 60 inches but varies according to a normal distribution with a SD = 1.5 inches.

Notice at the bottom that I also added random noise for Y (weight) that is inherent and independent of the factor levels. In this case, the weight varies by 10 pounds even if the factor levels do not change. You might have a historical value for this SD or you might use the estimated RMSE from the regression analysis to fit this model.