How reliable is post hoc means comparison in anova with unequal variance?

xiaoyaj0

In the attached data, I want to do an ANOVA analysis with MathSS as Y, and School Name as X. The variance between groups are not equal, after the "unequal variance" test. Can I still proceed with means comparison such as HSD Tukey? How reliable is this post hoc test? Thank you so much!

Test Data Fall 2014.jmp

KarenC

I am going to make the assumption that you wish to understand the similarities and differences between the schools for the various test score that you have. I would do some or all of the following:

Use distribution to look at scores by school. Most schools have a few really good test takers but you are probably more interested in what the middle part of the distribution looks like.

Use graph builder to explore the data. Here is a busy GB that I would use as an exploratory tool - I would simplify before sharing  (copy to a new script and with your data table open run the script)

Graph Builder(

Size( 880, 544 ),

Variables( X( :School Name ), Y( :MathSS ) ),

Elements(

Box Plot( X, Y, Legend( 7 ) ),

Points( X, Y, Legend( 8 ) ),

Line( X, Y, Legend( 9 ), Summary Statistic( "Median" ) ),

Contour( X, Y, Legend( 10 ) )

),

SendToReport(

Dispatch( {}, "Graph Builder", FrameBox, {Marker Size( 2 )} ),

Dispatch( {}, "400", LegendBox, {Set Title( "" )} )

)

);

Use the Fit Y by X platform to explore as you have started to do. Use the quantiles option to compare the quantiles (important measures for test scores - in my opinion), consider non-parametric tests if you must have p-values. Consider the ANOM with transformed ranks to compare across all schools.