Which alias terms to include in fit modelGood Day
I have performed a main effects screening design experiment to determine which of my factors have the most significant effect on the mechanical properties of a material. My factors were:
L -> 5 level discrete numeric
R -> 2 level categorical
S -> 2 level categorical
T -> 2 level categorical
I used the default number of runs and added one replicate run as this was suggested by a colleague and tried my best to minimize correlation values when using the design evaluation tools, but had to keep the runs at a minimum as I did not have a lot of material to work with. The two factor interactions were automatically excluded from the design if I remember correctly and were added as alias terms.
When I now run my analysis after entering my experimental data using the fit model platform and effect screening, the effects which are significant vary depending on which alias terms I include in the fit model platform. I also cannot include all the two factor interactions (e.g L I can only include in a two factor interaction term twice) as most of the effects are just not shown in the resulting effect summary then. Does this have something to do with the degrees of freedom in the design? Is it correct to try and include as many of the two factor interaction terms as possible with priority given to the ones with highest correlation with main effects in the alias matrix? In my alias matrix the 4 highest correlations are as follows:
L*(T*R) - 0.69
L*(T*S) - 0.58
L*(S*R) - 0.58
R*(T*L) - 0.56
where the factors in brackets are the two factor interaction term partially aliased with the main effect in front of the bracket. 4 more correlations follow these having values around 0.4 and the rest of the design includes terms with very low correlation values.
Are these alias terms the reason why the effect L pops up as being the only significant factor (p<0.05) in most cases when I include different two factor interactions? Should I include the 4 listed above (with the highest correlations) as my best bet at finding which factors are significant if there are any? Also do I need to include two factor interactions in the fit model analysis at all?
AnalyticsDOERe: Which alias terms to include in fit modelI'm a bit confused , so here are some questions/comments:
1. If you are running a screening design, why are you testing a factor at 5 levels? This would allow you to estimate a 4th order polynomial which would be more likely associated with an optimization design.
2. I assume you used Custom Design and added the model terms you were most interested in?
3. I don't understand what you mean by "The two factor interactions were automatically excluded from the design if I remember correctly and were added as alias terms." You can't have terms that are aliased in the fit model or you will get singularity.
4. I also don't understand "added one replicate run"? Did you do this to get 1 DF for error for your statistical test? How did you choose which treatment to replicate? Realize the statistical test MSfactor/MSerror (F-test) assumes the errors are unbiased, randomly distributed and hopefully REPRESENTATIVE of the true random errors (not sure how you get this with 1 replicated treatment?). And the residuals meet NID(0,variance) assumptions.
5. Each coefficient in the model is conditional and its magnitude, and sometimes the sign, change. They depend upon the other variables in the model (and noise).
6. Don't understand "priority given to the ones with highest correlation with main effects in the alias matrix". If your model variables are correlated this is multicollinearity. You don't want this. Make sure you check VIFs in the parameter estimates table.
Perhaps attach your JMP table and the community can have a look.
Re: Which alias terms to include in fit modelHi and thank you for taking the time to respond to my questions!
1 and 2): I was referred to a colleague for help who has experience in experimental design. I started off with a custom design but was told to rather do a classical screening design as I am trying to find out which factors have the most significant effect on my material mechanical properties so that we can take those and focus on them in further research.
3 and 6): I meant that during creation of the screening design only the main effects were included in the model and the two factor interactions were added as alias terms. After I gathered my research results and I had to analyze using the fit model platform I was confused about whether I should add alias terms into the fit model section before I run the analysis as the alias matrix and the colour map showed partial aliasing and correlation of some main effects and two factor interactions but not between the main effects themselves. I understand the alias matrix to reveal which main effects may be biased by certain two factor interactions? and from what I have read I thought that these two factor interactions should then be included in the fit model section? If I just run the fit model analysis without adding alias terms and get my results, what should I look at to know whether the main effects indicated as significant are really significant? I am not familiar with singularity but will go and read up on this.
4): I was told to manually add one replicate run to the screening design after the table was generated but am not sure how this helps.
I am attaching my JMP design table here.
Re: Which alias terms to include in fit model
My first comments are, I don't understand:
- the process being experimented on (did you create a process map of the experiment?),
- the factors nor the hypotheses that led you to experiment on these factors (do you have a thought map),
- the response variables nor how much change in the response variables if of practical value,
- how the experiment was executed (did you keep track of run order? Was it run in random order?),
- how you were handling all of the other variables that changed during the course of the experiment (that were not explicitly manipulated).
So my answers may be less than complete.
1. What you have attached is not what I would call a classical screening design as you have a factor (L) at more than 2 levels. Since you did this, you have created a rather complex aliasing structure. If you had stuck to 4 factors at 2-levels, you would have a straight forward alias structure resolution IV design in 8 treatments. I don't understand why you would test just one factor at 5 levels. This biases the results as there is more information regarding factor L than others in the study (4 degrees of freedom for L).
2. I still don't understand your use of the word alias "only the main effects were included in the model and the two factor interactions were added as alias terms". You don't add terms that are aliased to the same model or the resultant analysis will invoke a singularity warning in JMP. In the table you provided, you have a total of 10 DF. Each 2-level factor is 1 DF and there are 4 DF for L (1 for linear, 1 for quadratic, 1 for cubic and 1 for quartic). The non-linear terms are partially aliased with 2nd order (and higher) linear effects. Since L is being considered a continuous variable compared to the others in the model, this may create a mixed model scenario (Treated as a random effect vs. the other factors that are fixed effects).
There are multiple approaches to analysis...some do it additively by starting with 1 term and add terms terms to the model (stepwise). Some prefer to start with a saturated model and remove terms deemed insignificant. Both of these require an understanding and ability to interpret: R-square, R-square adjusted (and the delta between the 2), p-values, VIF's (for unbalanced designs), MS error and whether this is representative of the true error, practical significance, etc. Realize when you leave a term out of the model it ends up in the error term and then becomes part of the MSerror estimate impacting the F-test. Removing insignificant terms from the model further biases the MSerror term (making p-values look even more significant). Daniel was the first to use normal plots to perform statistical tests unbiased by the MSerror term.
3. Adding 1 replicate run does little to provide a representative, unbiased estimate of the error, IMHO. And which run was selected to replicate? Randomized replicates certainly has been used/recommended for many years to get an unbiased estimate of the error. There are, perhaps, more efficient and effectives means of understanding the error, how representative it is of future conditions and therefore whether it is a good basis for a statistical test.
