375773_en-US

liliconcom

Factor not statistical significant in Effect Tests although there is statistical significance difference among factor levels from Tukey Multiple Comparison

Hi, 

I am using Generalized Regression model to perform analysis in my data, with Binomial as response distribution (because my response values - ErrorAT - are either 0 or 1). I am using Lasso as the estimation method. 

I opted to used generalized regression due to the large correlation among some factors. 

 

I have a quick question related to this model: one of the main factors (seedAT) was removed from the model. However, when I used Tukey multiple comparison test on the factor, all the three levels of the factor (seedAT) are statistically significantly different among themselves. If they are statistically significantly different among themselves, shouldn't the factor be significant too?

Could someone shed some light on why the levels of the factor are statistically significantly different but the factor is still not significant to the model?

Thank you!

 

AnalyticsDOERe: Factor not statistical significant in Effect Tests although there is statistical significance difference among factor levels from Tukey Multiple Comparison

Statistical significance is a conditional statement.  It depends on what is being compared, what is in the model and what is not, what constitutes the error terms, what is in the inference space.

Re: Factor not statistical significant in Effect Tests although there is statistical significance difference among factor levels from Tukey Multiple ComparisonRight, but the comparison here is the same. So, if the levels of the factor are significantly different in the model, shouldn't the factor itself be significant for the model?Re: Factor not statistical significant in Effect Tests although there is statistical significance difference among factor levels from Tukey Multiple Comparison

I'm not understanding your model.  Since seedAT is in some cross effects that are significant, it is only the direct effect that is not showing significance.  While this seems unusual, it suggests to me that the variable's significance lies in its interaction with other variables - once those are accounted for, the effects of seedAT have been incorporated. It appears like seedAT was removed from the model - that is not insignificance, but an indication that the model is redundant (I believe - I have little experience with Generalized regression, so I might be wrong).

Re: Factor not statistical significant in Effect Tests although there is statistical significance difference among factor levels from Tukey Multiple Comparison

It would be extremely helpful to understand the data set and how you acquired the data, but I understand you may not want this posted in a public forum.  This does make it challenging to know what exactly you did or are trying to do. I don't necessarily agree with you that the comparisons are the same.  One comparison is the factor (seedAT) with the other factors, interactions and error in the model.  When looking at the level differences in the levels of seedAT, you are comparing within level to between level.  There are no factors in this comparison and the basis for comparison is the within level variation (not the unassigned DF's of the model, error).  

 

I'm not sure about your model, but it appears you removed the 1st order effect and kept 2nd order effects.  This does not follow the principle of hierarchy when model building?  Somewhat to Dale's point.

One additional comment: I do not have any idea of what the differences are in practical terms. Don't forget, practical significance is always more important than statistical significance.