Stepwise Regression analysis vs Least Square regression

albiruni81

Hi,

 

I am having problem whereeby using the stepwise regression it actually shows that only 1 factor is significant but running a direct Least Square regression it actually shows multiple factors is significant. Any reason for this to be this way?

 

Rgrds

 

Irfan

gzmorgan0

This is not an uncommon occurrence.  This link provides an example using a JMP Sample Data table.

https://www.seas.upenn.edu/~ese302/extra_mtls/STEPWISE_REGRESSION.pdf

 

It recommends using mixed stepwise ( <0.25 to enter and  > 0.10 to remove) or to use something like finding the best model with K factors.  There are many data issues that can cause this to happen. The most common are:  multicollinearity, interactions, missing categorical/grouping factors and missing data.  Least squares regression assumes independent factors; muticollinearity means some of the factors are correlated. 

 

A scenario that might help explain this phenomenon is: Suppose Y, an electrical parameter, is a function of conductive film thickness and area of the structure and secondarily the smoothness of the film (no lumps or bubbles or pits). The data set has film thickness, structure length and strucure width.  Stepwise might include thickness, but not length nor width alone, but together the two factors are highly significant.  But in this case if area (length * width, a length and width interaction)  were modeled the model would be even stronger. 

 

I hope you find the pdf recommendations useful.