F-distribution for power and sample size calculator - why this one?

Question

I had the following question from a customer - I am looking for any insight here.. I am including his thoughts on distributions, so if these are faulty please feel free to throw the customer under the bus!

-----------

...Apparently the JMP sample size and power calculator for "comparision of two means" uses an F-distribution (as opposed to either a normal distribution or a t-distribution). might you be able to point me towards a good resource on why the F-distribution is "obviously" (to a JMP statistician) the right thing to use in this case? my understanding has been limited to the following (and I'd like to take it a step forward):

- normal distributions are great pedagogically, but in the real world we are often testing new or unstable processes with small sample sizes, such that the sample standard deviation (or an assumed standard deviation) is often a poor approximation of the true standard deviation.

- t-distributions address the problem of testing new or unstable processes with small sample sizes by allowing for the likelihood that the true standard deviation is > sample standard deviation (i.e., as indicated by chi-2 distribution)

- F-distributions are t-distributions squared, for whatever that's worth...

I totally would have expected JMP to use a t-distribution for comparison of two means since the "t-test" is the classic "comparison of two means" test...

Peter_Bartell · Answer

The only other thing I'll add is for the customer to not get the notion of the distributional function/shape of the actual populations of interest get in the way of using typical t and F tests for hypothesis testing/sample size determination work. The magic of the central limit theorem insures that the sampling distribution of the test statistic in question is appropriate, at desired alpha/beta risks, etc.

Laura_Higgins · Answer

Thank you for all the input! I appreciate it!

jiancao · Answer

The reason why F distribution is used is because a two-sample comparison is a special case of a k-sample test. When the following assumptions can be made: (1) samples are independent, (2) they are drawn from normal distributions and (3) they have equal variances, an F test can be constructed to compare the equality of k sample means.

For two-sample comparisons, a t-test is equivalent to an F-test result only when variances are (assumed to be) equal. In fact, JMP Fit Y by X offers a t-test assuming unequal valance (i.e., t-test option). P-values usually differ.

As for his questions on normal distribution vs. t-, F or Chi^2 distributions, t, F and Chi^2 are sampling distributions of random samples drawn from normal distributions. They are used for various statistical testing and inferences on these samples.

.