
Hi
Fabulous library you've put together.
I'm curious about your MannWhitney test.
How are you handling the situation where you get tied ranks? I've looked at the source code where you do your merge sort (without forming a new merged list), but I can't quite figure out what happens with tied ranks.
(The reason I'm asking is because I'm getting different results with different MannWhitney calculators, and I'm trying to figure out why!)
Thanks. (And, of course, I quite understand if you don't have the time to answer this, etc).


Coordinator
Mar 2, 2013 at 6:46 AM

Hi Jerry, I'm glad you have found the library useful. You ask a perfectly reasonable question, and while I do have a reasonable answer, I'm afraid it won't be very satistying.
The short answer is that we do absolutely nothing to account for ties when doing the MannWhitney test. If a particular value appears three times and ties for 7th, 8th, and 9th place, which instance gets assigned which rank will basically be random. (Not random
in the sense of not reproducable, but random in the sense that it will depend on the orderings of other values in the samples, which sample given as the first argument and which as the second, etc.) This will affect the computed value of the test statistic
(how much depends on how many ties there are), and thus the computed Pvalue.
We did not, however, take this approach out of ignorance or lazyness. The basic issue is that the derivation of the null distribution of the MannWhitney test statistic assumes that the variable is continuous, and the chance of getting truly exactly equal values
when sampling from a continuous distribution is exactly zero. Now if your variable really is continuous (e.g. the length of an inchworm) and you get ties only because you only record a finite number of signficant digits (e.g. you record inchworm lengths of
1.2474... and 1.2531... both as 1.25), then you will probably get so few ties that this whole debate is purely academic. But in the more likely case that you actually have ordinal data (e.g. "very satisfied? somewhat satisfied? ...") then the real
problem is that using the MannWhitney test is not really justifiable. We could certainly pick a reasonable rule that would produce consistent test statistic values (e.g. ties all get assigned their midrank). But the null distribution of that statistic wouldn't
be the MannWhitney distribution, so the computed P value would be wrong. (No, not very wrong for a small number of ties, but not right either, and certainly wrong in the case of a large number of ties.) So in the end we decided the implement in accordance
with the assumption that the derivation of the test makes, namely no ties.
By the way, we are not the first people to face this quandry. There are whole academic papers on differently ways of handling ties:
http://www.tandfonline.com/doi/abs/10.1080/00031305.2000.10474513. My own view is basically the same as this post on the Rhelp mailing list:
https://stat.ethz.ch/pipermail/rhelp/2009December/222787.html.



Hi
Thank you for your response.
Actually, it is an entirely satisfying answer. (And also an interesting issue  though I don't know enough about stats to have an opinion  not least because it suggests the possibility that at least some people will have reported significant results simply
because they chose a stats package that implemented the MW in a particular way).
Thanks again, And many thanks for your library. Great work.
Best,
Jerry

