Looking for some guidance: Logistic regression

Question

Hello all,

New to stats overall and have watched many videos but unfortunately only made it so far. I am trying to determine the relationship between PTB (nominal Y/N) and 16 water contaminants (also nominal, i.e. yes the water contaminant occurred in a year or no it did not).

Multicollinearity is present between many of the water contaminants and I'm not sure how to adjust the model to reduce multicollinearity. Must I remove water contaminants? My concern with that is I would go from 16 contaminants to 2 that are not impacted by any others. Is there a better option? I'm not entirely sure of the purpose of creating interactions but would this be a way to manage it? Also, if no other solution besides to remove the majority of the covariates could I run a model for each individual contaminant? Or would something like this be inaccurate?

Thanks in advance!

Mark_Bailey · Answer

You are correct. My reply was hasty.

You can still use logistic regression with categorical predictors in the linear predictor.

You could also use Multiple Correspondence Analysis to explore and interpret the associations (as opposed to the correlations with continuous predictors).

dale_lehman · Answer

But didn't the post indicate that all of the predictors are also nominal?  Wouldn't that call for some other kind of analysis?  I'm not sure what that would be, but I would probably try to explore which contaminants occur together and then create some new composite features for those combinations.  I think your suggestion of principal components was along those lines, but would that be a correct approach for nominal predictors?

Mark_Bailey · Answer

Collinearity is common, except for some designed experiments. The questions are how strong is it and how many variables are involved? For example, if two predictors are extremely collinear (e.g., r = 0.97), then you might omit one of them from the model. If you have several of the 16 predictors which are naturally collinear, then you might create principle components and then 'factors' that represent the underlying force. For example, you might have several measures of water hardness that could be replaced with one or two principle components or factors.

So, 'it depends.'

You can study the colinearity in several ways. I good place to start is Analyze > Multivariate Methods > Multivariate with the 16 predictors.