This project has moved. For the latest updates, please go here.

Mann-Whitney

Mar 1, 2013 at 5:38 PM
Hi

Fabulous library you've put together.

I'm curious about your Mann-Whitney test.

How are you handling the situation where you get tied ranks? I've looked at the source code where you do your merge sort (without forming a new merged list), but I can't quite figure out what happens with tied ranks.

(The reason I'm asking is because I'm getting different results with different Mann-Whitney calculators, and I'm trying to figure out why!)

Thanks. (And, of course, I quite understand if you don't have the time to answer this, etc).
Coordinator
Mar 2, 2013 at 6:46 AM
Hi Jerry, I'm glad you have found the library useful. You ask a perfectly reasonable question, and while I do have a reasonable answer, I'm afraid it won't be very satistying.

The short answer is that we do absolutely nothing to account for ties when doing the Mann-Whitney test. If a particular value appears three times and ties for 7th, 8th, and 9th place, which instance gets assigned which rank will basically be random. (Not random in the sense of not reproducable, but random in the sense that it will depend on the orderings of other values in the samples, which sample given as the first argument and which as the second, etc.) This will affect the computed value of the test statistic (how much depends on how many ties there are), and thus the computed P-value.

We did not, however, take this approach out of ignorance or lazyness. The basic issue is that the derivation of the null distribution of the Mann-Whitney test statistic assumes that the variable is continuous, and the chance of getting truly exactly equal values when sampling from a continuous distribution is exactly zero. Now if your variable really is continuous (e.g. the length of an inchworm) and you get ties only because you only record a finite number of signficant digits (e.g. you record inchworm lengths of 1.2474... and 1.2531... both as 1.25), then you will probably get so few ties that this whole debate is purely academic. But in the more likely case that you actually have ordinal data (e.g. "very satisfied? somewhat satisfied? ...") then the real problem is that using the Mann-Whitney test is not really justifiable. We could certainly pick a reasonable rule that would produce consistent test statistic values (e.g. ties all get assigned their mid-rank). But the null distribution of that statistic wouldn't be the Mann-Whitney distribution, so the computed P value would be wrong. (No, not very wrong for a small number of ties, but not right either, and certainly wrong in the case of a large number of ties.) So in the end we decided the implement in accordance with the assumption that the derivation of the test makes, namely no ties.

By the way, we are not the first people to face this quandry. There are whole academic papers on differently ways of handling ties: http://www.tandfonline.com/doi/abs/10.1080/00031305.2000.10474513. My own view is basically the same as this post on the R-help mailing list: https://stat.ethz.ch/pipermail/r-help/2009-December/222787.html.
Mar 2, 2013 at 11:30 AM
Hi

Thank you for your response.

Actually, it is an entirely satisfying answer. (And also an interesting issue - though I don't know enough about stats to have an opinion - not least because it suggests the possibility that at least some people will have reported significant results simply because they chose a stats package that implemented the M-W in a particular way).

Thanks again, And many thanks for your library. Great work.

Best,

Jerry