nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Multiple Regression

data_analysis

Hello,

I have an scenario where my response is a continuous variable and predictors are mixed(continuous and categorical with more than 2 levels)

In the response i see a trend and my goal is investigate which predictor are trending with it.(Other words which predictor is mostly contributing more in response trend because trend is a bad sign for given proccess.)

I am trying different ways by using multiple regression but i have some concerns:

Would be better to run different regression for each type of variables(one time with only numeric and next time with only categorical) to detect the causing trend predictor?

When i run the model with all variables i am getting some NA for some estimates and this is a sign of correlation. In this point i am not sure how to decide which variable should i have to remove to avoid multicollinearity.

I hope for some ideas.

Thanks.

Accepted answers

All comments

P_Bartell

When you say, 'trend'...I'm presuming you mean a time based systematic change in the response. For the continuous predictor variables I suggest a simple overlay plot in Graph Builder leveraging the time sequence of collection of the responses (maybe with a local data filter to highlight specific x's efficiently). For the categorical variables is there still a lurking time element? Say, like batch 1, 2, 3, 4, etc. where each subsequent batch is bounded in a time sequence? If you've got a time sequence to the categorical variables, use the Value Ordering column property to get them to show up in the correct time sequence in Graph Builder. If you all you are really interested in is detecting this time based trend...you should be able to 'see it' in Graph Builder and not even need to go to any modeling.

data_analysis

Actually , i am interesting more at variable that is causing the trend than when it happened.
Thanks.

Mark_Bailey

It sounds like you suspect collinearity and singularity with the data set in hand. You could try Stepwise in the forward direction starting with the complete linear predictor.

You could also use Partition with extensive over-fitting and then examine the column contributions. The top contributors are likely to be real.

data_analysis

Hello,
I am trying to use this method but it does not work because it gives error related multicollinearity.
Warning message:
In leaps.setup(x, y, wt = wt, nbest = nbest, nvmax = nvmax, force.in = force.in, :
2 linear dependencies found
Thanks for your help.

Mark_Bailey

That message indicates a singularity. That means that the collinearity is perfect, or the correlation of the estimates is -1 or +1. You model is over-specified. That is why I suggested using forward stepwise or recursive partition.

P_Bartell

My suggestion will offer guidance on which variables are correlated, and note, correlation is not causation, especially with happenstance data, which I presume you have. Just look for changes in the x’s that align in time (noting there may be lags) with changes in the response.

statman

My thoughts. What Pete is suggesting is to look for the relationships graphically (rather than quantitatively). This is an excellent idea and graph builder is your friend. Mark suggests a more quantitative approach. He suggests an additive approach to building the model.

What none of us know is how you got your data? Is this an observational data set, directed sampling or an experiment? If it is an observational data set, then the best you can get regardless of the analysis tool is correlation. There are many methods to look at the data. You could try the Analyze>Multivariate Methods>Multivariate enter the y's and predictors to get scatter plots. If you do some regression modeling, I suggest you start with hypotheses. What factors would make sense to be affecting the trend in the response? Build a model with those factors first. And always look at VIF's in the parameter estimates table.

data_analysis

Thank you.
It makes sense.

data_analysis

I do not have full information but looks they are more from an experiment.
I do appreciate your help.