generalize regression, std error is 0 for some predictors,

Question

predictor itself looks fine. what does this mean?

I observed that this occur more often while the interaction terms are included in the model. The std err=0 happened in some model but not all models including the same predictors.

The estimation method is Lasso while the distribution of the dependent variable is ZI Binomial, since the DV represent a rare event (coded as 0 or 1).

thank you for any feedback or insight!

yunyunda · Answer

Hi Kevin,

thank you very much for your reply and recommended paper. I am just getting familiar with generalize regression lately. The Tibshirani, R. (1996) paper will be a good reading material for me to get more technical details on this approach.

I should have state my question more clearly. Those predictors that looks a bit concerning to me are those with an non-zero estimate (e.g., 0.11)  but with a std error=0 and identical lower 95% & upper 95%. For some other statistical models, when there is a unusual value in standard error for some predictor, it raise a red flag for the entire model/analysis to me.

So in general, I wonder if after running JMP generalized regression there are a small numbers of predictors (in some analysis, including the intercept term) with non-zero values in estimated coefficient but with std error equal to zero, shall I trust estimates/output from this particular analysis? Or the estimates for other predictors and overall model are still good while overlooking those std err=0 predictors?

We have plenty of predictors, around 40. some of them are categorical. the dependent variable is a rare event, but the predictors are not.

And if anyone can point out what's the possible explanations for the std error=0 while with non-zero estimates (does this means that these predictors are on their way to be shrunk? haha, just kidding), or ways to test out what could be the reason,  that would be quite helpful to understand potential causes of this issue.

thank you!

Kevin_Anderson · Answer

I recommend reading Tibshirani, R. (1996); "Regression Shrinkage and Selection via the Lasso"; Journal of the Royal Statistical Society Series B (Methodological), Vol 58, Issue 1; pp 267-288.  Section 2.5 is on Standard Errors, in which is stated "Since the lasso estimate is a non-linear and non-differentiable function of the response..., it is difficult to obtain an accurate estimate of its standard error."  It notes that a common approximation of the covariance matrix of the estimates yields estimated variances of 0 for predictors where the estimates have shrunk to 0.