Wilcoxon Signed Rank test - approximation vs. exactHi JMP team,
In Distribution and in Matched Pairs, JMP currently calculates an exact Wilcoxon Signed Rank test for n <= 20 and an approximation to that test for n > 20. What is the backstory on why the cutoff of 20 is used? It appears that for p-values in the neighborhood of 0.05 that the approximation is conservative but noticeable in at least the third decimal place. For my purposes, the approximation is okay, but we have some researchers who are in fields where the standard is to run an exact test on larger sample sizes and running into some conflict because the JMP approximation is not the same as the exact test.
Given the increase in compute power over the past few years, is it reasonable to change the threshold from 20 to some higher number? How high could we go without a major performance issue? I know the exact tests are more computationally intense...
Thanks,
Michael
Re: Wilcoxon Signed Rank test - approximation vs. exactHi Michael- This is a great feature request! If you have not yet put it on the wish list, please do - that list goes to Development!
-LauraRe: Wilcoxon Signed Rank test - approximation vs. exactHi Laura, thanks for the link.
I would like to know why we need to approximate this statistic. I know exact statistics can be computationally intensive but I believe we have enough compute power to exactly estimate for sample sizes somewhat larger than 20. Not sure how much larger than 20 we can go but with increases in compute power over the years it should be possible. We have some colleagues here who are using other means to get the p-value because they believe (rightly or wrongly) that the exact p-value is better than an approximation.
Re: Wilcoxon Signed Rank test - approximation vs. exactHi Michael -
Here's what I've found out about this.
JMP is using Iman (below) as a reference, as is SAS.
Iman,R.L.(1974), Use of a t-statistic as an Approximation to the Exact Distribution of the Wilcoxon Signed Ranks Test Statistic, Communications in Statistics 3, 795-806
If you have another specific recommendation that you would like to see JMP Development implement, please let me know as well as add it to the Wish List here in the P&G Community.
- Laura Higgins
