difference in EFFECT TESTS between Generalized regression and Fit Least Squares platforms

Question

I'm on JMP Pro 18.2.2 and have noticed that the Effect Test table differs substantially between the Generalized Regression platform and Fit Least squares (and also all other platforms in JMP) when there is an interaction term.  This is true even when the design is perfectly balanced.  The Generalized Regression platform generates sums of squares that depend on the parameterization of the factors.  If the order of the levels changes then the effect tests table also changes. I have illustrated the issue in the attached JSL file.  Fit Least Squares generates the same SS tables as SAS and other common stats software (eg Stata, SPSS etc) but the Generalized Regression platform produces the same SS tables as R using the car package and LM type III SS with default effect parameterization.

Note that this is not a slight difference and the conclusions for the main effects can change.

Is there any documentation or white papers from SAS that would explain this?

please see the attached JSL file which illustrates my concerns using the JMP popcorn data set.

popcorn comparison 2.jsl

GUhartel · Answer

I understand that the different platforms encode dummy variables differently (cf http://www.jmp.com/content/dam/jmp/documents/en/newsletters/jmper-cable/11_winter_2003.pdf). SAS GLM also uses the indicator parameterization, however, this doesn't affect the type 3 SS anova tables, only the parameter estimates, whereas in the Generalized Regression platform the anova tables are also changed.  This might come as a surprise to some.  it might be worth for there to be a warning.

Victor_G · Answer

Hi @GUhartel,

Welcome in the Community !

The differences you can see between the two modeling platforms is linked to the way these platforms encode nominal (categorical) variables differently : Nominal Factors & Statistical Details for Nominal Effects Coding

In summary, the estimates of a categorical effect are different, because the encoding of the nominal variable is different, so the estimates represent different hypothesis. As far as I understand the differences between these two nominal effects coding, I would say that if you expect to evaluate levels from a nominal factor by comparing each level to one specific level, then the nominal coding from Generalized Regression might be more appropriate. You can use column property "Value Order" to specify the order needed so that it fits your study design. The last level of the factor will be used for the comparison to all other levels.

But when you want to know how each levels may influence the average response calculated on all levels, then the Standard Least Squares from Fit Model platform may be more useful.

You can read my previous responses on these topics here : How DOE analysis handle categorical factor in regression analysis & Random effect test

Hope this answer will help you,