Is it okay for a variable to have multiple interactions?In my data, I have about 15 different variables (water contaminants) and I am trying to determine if there are any interactions. I am unable to use VIF (it doesn't show as an option) so I am running a Fishers Exact Test to determine if there is collinearity. Starting with the first variable, I run the test on each other variable and will be doing this for my whole list of variables. The first variable has an interaction with 10 other variables in the data. This is where my question comes in, is it bad to have a variable with multiple interactions with other variables, or is this just something that can happen under certain data sets?
Another question I have, I know there can be three-way interactions, but when is it appropriate to do so? Can you have a four and five-way interaction, or higher?
Thank you.
AnalyticsRe: Is it okay for a variable to have multiple interactions?We usually use the term 'interactions' to refer to a non-linear effect on a response involving two or more factors. Is that your meaning? Or do you mean correlations among the predictors? You do not mention a response variable.
Regarding the complexity of interaction effects that you might expect, the answer is 'it all depends.' There is a principle (not a theory or law) from screening a large number of potential factors that most of the contribution is from the first order terms and the contribution decreases as the order increases (so called 'hierarchy principle'). So, for example, a 4th-order interaction is possible but it is either very unlikely or provides a very small contribution. On the other hand, it is very common for many predictors to be correlated (that is, 'multi-collinearity').
My final thought is that your one at a time analysis is fraught with pitfalls. A better approach would be a 'principle components analysis' for correlations or a 'linear model' for interactions.
Re: Is it okay for a variable to have multiple interactions?Sorry for the late response. I was using the term interactions incorrectly. I meant correlations among the predictors (thanks for the clarification). The response variable is a birth outcome (i.e. pre-term birth yes/no). How do I determine when to incorporate 3rd-order interactions to reduce multicollinearity in the logistic regression model?
Re: Is it okay for a variable to have multiple interactions?In addition to the comments already made, I offer the following thoughts. The only way to "fix" multicollinearity is to remove the offending terms from the model. Failure to do this will result in a model that over estimates the usefulness of the model. Unfortunately there is no statistical procedure that is foolproof for identifying which of the correlated terms should be removed (though there are several available from latent variables to PCA). My advice is to use your knowledge of the situation (the predictor variables and the response variables) and consider the usefulness of the predictor variables in the model. Then iterate...use the model to predict the next set of data, re-evaluate, modify and repeat.
