How to quantify the significance of a model with a random effect?

clane

When models including a random effect are run in JMP, the ANOVA table cannot be calculated so no p-value for the entire model is generated. Instead, the REML Varaince Compenent Estimates is calculated. I was wondering if there is any way to get a p-value for the whole model when a random effect is included. I know p-values are not always the way to go (I am using AICc as well), but I think a p-value is needed to help other people interpret my models.

stevedenham

Think about what a p value is--the probability that a null hypothesis is rejected is usually the answer given.  But it has some assumptions, the first being that some sort of test exists.  About the only test I can think of is a likelihood ratio test, comparing the log-likelihoods of the model that is fit, to the null model.  It turns out, that for reasonably sized datasets and a limited number of model parameters, AICc is as good a comparator as you can find.  However, the chi-square value from comparing the -2 log-likelihoods is the only "p value" generating test that I can think of.

Steve Denham