nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

JMP ANOVA not matching Python results

gchesterton

Using JMP Pro 15.0 on a Mac. I conducted a 2^4 factorial experiment with the following results:

Factor1	Factor2	Factor3	Factor4	Y
P	L	U	C	77.90909
P	U	U	C	60.63636
A	U	U	C	103.5833
P	L	L	C	92.76923
A	L	L	T	81.41667
P	L	U	T	80.92308
P	L	L	T	88.61538
A	L	L	C	108
A	U	L	C	116.3333
A	L	U	T	77.41667
A	L	U	C	88
A	U	L	T	105.75
A	U	U	T	105.75
P	U	U	T	120.3333
P	U	L	C	109.9231
P	U	L	T	99.69231

I fit a model, limiting it to main effects and 2-way interactions. I found only one significant effect. Anyone care to try it on their end? My colleague is getting different results in Python, even after ensuring that he's calling for a type III anova rather than the Python default of type II. His results appear to be more closely aligned to JMP's sequential (i.e. type I) results. But I was at a loss as to how to get my results to converge on his or vice versa. I thought I better start by verifying that I have correctly implemented the model in JMP.

Again, I fit the factorial to degree 2 only.

Is anyone familiar with JMP-Python inconsistencies? And are my results ok for starters?

Effect Tests

Source	Nparm	DF	Sum of Squares	F Ratio	Prob > F
Factor1	1	1	192.1560	1.3957	0.2906
Factor2	1	1	1007.2948	7.3161	0.0425*
Factor3	1	1	483.4297	3.5112	0.1198
Factor4	1	1	0.4703	0.0034	0.9557
Factor1*Factor2	1	1	42.9518	0.3120	0.6006
Factor1*Factor3	1	1	13.0468	0.0948	0.7706
Factor1*Factor4	1	1	551.1892	4.0034	0.1018
Factor2*Factor3	1	1	1.6620	0.0121	0.9168
Factor2*Factor4	1	1	393.5865	2.8587	0.1517
Factor3*Factor4	1	1	700.2053	5.0857	0.0738

Accepted answers

jules

Hi @gchesterton,

Inconsistent results are always a hassle, and I do hate a mystery. Using your data I find the identical results in JMP. Are you able to post the results your colleague is finding with python? The python code to fit the model would also be helpful.

You mention that his results appear to match the jmp sequential tests. I find this confusing. The method for finding sums of squares would not make a difference in this kind of designed experiment. These factors are orthogonal, so Type I SS are identical to Type III SS. Below is the the Fit Model output with both the Type III and Type I SS confirming this.

As for your general question of JMP-Python inconsistencies, I am not aware of any. Inconsistencies in almost all cases come down to how the model is defined, and it seems to me that you have defined it correctly in JMP. This leads me to believe your colleague is doing something different from this in Python.

I hope this helps some -

@jules

All comments

jules

Hi @gchesterton,

I hope this helps some -

@jules

gchesterton

Thanks @jules. I enjoy your YouTube lessons. Glad to see my results weren’t completely off base. I’ve asked my colleague to share his code.
In the meantime I’m still unclear about when I should expect type I and III anova to be the same. Is this orthogonality condition satisfied any time we have independent factors from a designed experiment? Or do the factors need to exhibit no interaction for this to hold?

jules

Hi @gchesterton,

I'm so glad to hear that you've found those videos useful! When the design is balanced in the way certain designed experiments ensures (without missing data), the factors are orthogonal, and types I, II and III all give the same results. Not all designs exhibit complete orthogonality, though, and in those cases there are correlations among the factor estimates, and the Type I,II,II estimates will differ. The presence or absence of an interaction is not a consideration. I quite like Matt Cooper's article on the differences between Type I,II,III sums of squares. It's a little older, but not an overly technical read and might help clarify some of the differences. Hope that helps!

@jules