OT: Nice article on "the allure of nonparametrics"

[Bruce Weaver]

In the past, I have referred people to Zimmerman's (2003) simulation study which shows that the Wilcoxon-Mann-Whitney test is extremely sensitive to heterogeneity of variance.  A later study by Fagerland & Sandvik (2009) shows the same thing.  I just learned of another short and very readable article (with a good title) that emphasizes the same point:

Johnson, D. H. (1995). Statistical Sirens: The Allure of Nonparametrics. Ecology, 76, 1998–2000.

HTH.
Bruce

[Kirill Orlov]

Many thanks for the articles, Bruce.
However lines like "The Wilcoxon–Mann–Whitney (WMW) test is often used to compare the means or medians" offend the eye. This test tests the difference in the so called 2-sample Hodges-Lehmann location estimator, or , in other words, it tests the stochastic dominance. In doing so, the test doesn't need any assumptions about the shape of the distributions.

With the assumption of equality of the distributional shapes (in population) added the test do tests for difference in shift of any specific quantile (such as median or the quantile corresponding to mean). But it is bit a foolish to use WMW to test medians at all, - since we have Median test (found in SPSS too) which compares medians without any assumptions!

[Bruce Weaver]

Hi Kirill.  Johnson is saying that people often do use the WMW test to compare means or medians, not that they should.  So I don't understand why it is offending the eye.  I think he would agree with your comment about testing for stochastic superiority.  But maybe I've misunderstood your point.

Cheers,
Bruce

Kirill Orlov wrote

Many thanks for the articles, Bruce.
However lines like "The Wilcoxon–Mann–Whitney (WMW) test is often used
to compare the means or medians" offend the eye. This test tests the
difference in the so called 2-sample Hodges-Lehmann location estimator,
or , in other words, it tests the stochastic dominance. In doing so, the
test doesn't need any assumptions about the shape of the distributions.

With the assumption of equality of the distributional shapes (in
population) added the test do tests for difference in shift of any
specific quantile (such as median or the quantile corresponding to
mean). But it is bit a foolish to use WMW to test medians at all, -
since we have Median test (found in SPSS too) which compares medians
without any assumptions!

[Rich Ulrich]

As Bruce pointed out originally, the reference in its title to
"Statistical Sirens" gives away the attitude -- the Sirens of
myth sang their "siren songs" to lure sailors to shipwreck
on certain famous rocks.

I think we all agree that the WMW test uses a measure for
stochastic superiority, and not any direct assessment of
medians or whatever.  But my sense of these other articles
on the hazards of testing ranks is that the test can give the
wrong p-value for "stochastic superiority" when certain
assumptions fail.  So far as I remember, it is not true (as
Kirill asserts) that the "test doesn't need any assumptions
about the shape of the distributions."

--
Rich Ulrich

> Date: Thu, 29 Aug 2013 15:51:33 -0700

> From: [hidden email]
> Subject: Re: OT: Nice article on "the allure of nonparametrics"
> To: [hidden email]
>
> Hi Kirill. Johnson is saying that people often *do* use the WMW test to
> compare means or medians, not that they *should*. So I don't understand why
> it is offending the eye. I think he would agree with your comment about
> testing for stochastic superiority. But maybe I've misunderstood your
> point.
>
> Cheers,
> Bruce
>
>
>
> Kirill Orlov wrote
> > Many thanks for the articles, Bruce.
> > However lines like "The Wilcoxon–Mann–Whitney (WMW) test is often used
> > to compare the means or medians" offend the eye. This test tests the
> > difference in the so called 2-sample Hodges-Lehmann location estimator,
> > or , in other words, it tests the stochastic dominance. In doing so, the
> > test doesn't need any assumptions about the shape of the distributions.
> >
> > With the assumption of equality of the distributional shapes (in
> > population) added the test do tests for difference in shift of any
> > specific quantile (such as median or the quantile corresponding to
> > mean). But it is bit a foolish to use WMW to test medians at all, -
> > since we have Median test (found in SPSS too) which compares medians
> > without any assumptions!
>

[Kirill Orlov]

Rich, I will owe to you if you point/explicate/prove the notion that the general-formulated (= stochastic superiority) Mann-Whitney test *do* needs assumptions regarding distributions (besides that the data are at least ordinal and, maybe [I'm not sure], that there is no ties, ideally).

30.08.2013 4:48, Rich Ulrich пишет:

As Bruce pointed out originally, the reference in its title to
"Statistical Sirens" gives away the attitude -- the Sirens of
myth sang their "siren songs" to lure sailors to shipwreck
on certain famous rocks.

I think we all agree that the WMW test uses a measure for
stochastic superiority, and not any direct assessment of
medians or whatever. But my sense of these other articles
on the hazards of testing ranks is that the test can give the
wrong p-value for "stochastic superiority" when certain
assumptions fail. So far as I remember, it is not true (as
Kirill asserts) that the "test doesn't need any assumptions
about the shape of the distributions."

--
Rich Ulrich

> Date: Thu, 29 Aug 2013 15:51:33 -0700
> From: [hidden email]
> Subject: Re: OT: Nice article on "the allure of nonparametrics"
> To: [hidden email]
>
> Hi Kirill. Johnson is saying that people often *do* use the WMW test to
> compare means or medians, not that they *should*. So I don't understand why
> it is offending the eye. I think he would agree with your comment about
> testing for stochastic superiority. But maybe I've misunderstood your
> point.
>
> Cheers,
> Bruce
>
>
>
> Kirill Orlov wrote
> > Many thanks for the articles, Bruce.
> > However lines like "The Wilcoxon–Mann–Whitney (WMW) test is often used
> > to compare the means or medians" offend the eye. This test tests the
> > difference in the so called 2-sample Hodges-Lehmann location estimator,
> > or , in other words, it tests the stochastic dominance. In doing so, the
> > test doesn't need any assumptions about the shape of the distributions.
> >
> > With the assumption of equality of the distributional shapes (in
> > population) added the test do tests for difference in shift of any
> > specific quantile (such as median or the quantile corresponding to
> > mean). But it is bit a foolish to use WMW to test medians at all, -
> > since we have Median test (found in SPSS too) which compares medians
> > without any assumptions!
>

[Rich Ulrich]

Kirill,
From the evidence of the two articles that are clickable, you might be right.
However -  Here is some criticism of those two articles. (As a reviewer, I
would have accepted neither article as they stand.)

I think that this point may be demonstrated in the Zimmerman article (the name
ending in WMZIM) that Bruce cited.  It may not show much else that is useful,
but it does show "too many rejections" of the general null hypothesis, for
certain specific simulations, if their manipulations did not invalidate the test.
Preferably, the step, "Set the mean to zero and standard deviation to 1" ... was
performed on the combined pair of samples in a test, rather than on two
samples separately.  That might legitimize one aspect of their results.

I am not very pleased with either of those two articles.

Zimmerman gives an obtuse discussion of how variance will change with
differences of the means, as with "log-normal" and so on.  I call this obtuse,
because variance differences induced by the shift of mean is totally irrelevant
to the rank test.  Consider:  If the data are all log-normal, then taking the logs
results in two normal samples, which will yield *exactly* the same ranks (and
thus, the same rank-test) whichever version you look at.   The same goes for
exponential, or whatever.  The mean-shift induced difference in variances will
never affect the robustness of the test.  That is the meaningful substance of the
rule, that the samples should be "drawn from the similar distributions."
  - If you know what transformation makes data normal, then the powerful
test for shift is to do the transform and use the t-test.

The simulation does *not* give data comparisons that are similar to realistic
data sampled from log-normal, exponential, or whatever.  Once they zero-center,
divide by the SD, and then multiple one-half the scores by a constant (to create
a larger SD in that set), they no longer have numbers that can be back-transformed
to a reasonable imitation of two sets of data drawn from log-normal, or whatever.

I really don't like the demonstration if they set means equal by separately setting
them to zero.  I *think* they might validly demonstrate a too-large rejection rate
with what I presume that they did, but even that much is questionable.

The other paper leaves too much documentation on the Website maintained by
the journal to determine what they did, so far as I could tell.  It seems like the
whole paper might be an extensive demonstration that "sometimes it doesn't
test the mean" and "sometimes it doesn't test the median" -- which is more easily
"proved" (in my opinion) by drawing up examples where the test rejects in the
"wrong" direction.   The paper mentions separate tests of rejection of equal means
and equal medians.  It gave no hint of anything that I would consider a proper
procedure for untangling those tests, which isn't anything I've seen done.  It *did*
hint that the authors might have assumed a simplistic solution.  The "stronger
effects with larger N"  also suggests that they are measuring artifact, since a
larger sample *should* be better at detecting stupid bias.

--
Rich Ulrich



 

---

Date: Fri, 30 Aug 2013 06:30:05 +0400
From: [hidden email]
Subject: Re: OT: Nice article on "the allure of nonparametrics"
To: [hidden email]

Rich, I will owe to you if you point/explicate/prove the notion that the general-formulated (= stochastic superiority) Mann-Whitney test *do* needs assumptions regarding distributions (besides that the data are at least ordinal and, maybe [I'm not sure], that there is no ties, ideally).

30.08.2013 4:48, Rich Ulrich пишет:

As Bruce pointed out originally, the reference in its title to
"Statistical Sirens" gives away the attitude -- the Sirens of
myth sang their "siren songs" to lure sailors to shipwreck
on certain famous rocks.

I think we all agree that the WMW test uses a measure for
stochastic superiority, and not any direct assessment of
medians or whatever. But my sense of these other articles
on the hazards of testing ranks is that the test can give the
wrong p-value for "stochastic superiority" when certain
assumptions fail. So far as I remember, it is not true (as
Kirill asserts) that the "test doesn't need any assumptions
about the shape of the distributions."

--
Rich Ulrich

> Date: Thu, 29 Aug 2013 15:51:33 -0700
> From: [hidden email]
> Subject: Re: OT: Nice article on "the allure of nonparametrics"
> To: [hidden email]
>
> Hi Kirill. Johnson is saying that people often *do* use the WMW test to
> compare means or medians, not that they *should*. So I don't understand why
> it is offending the eye. I think he would agree with your comment about
> testing for stochastic superiority. But maybe I've misunderstood your
> point.
>
> Cheers,
> Bruce
>
>
>
> Kirill Orlov wrote
> > Many thanks for the articles, Bruce.
> > However lines like "The Wilcoxon–Mann–Whitney (WMW) test is often used
> > to compare the means or medians" offend the eye. This test tests the
> > difference in the so called 2-sample Hodges-Lehmann location estimator,
> > or , in other words, it tests the stochastic dominance. In doing so, the
> > test doesn't need any assumptions about the shape of the distributions.
> >
> > With the assumption of equality of the distributional shapes (in
> > population) added the test do tests for difference in shift of any
> > specific quantile (such as median or the quantile corresponding to
> > mean). But it is bit a foolish to use WMW to test medians at all, -
> > since we have Median test (found in SPSS too) which compares medians
> > without any assumptions!
>

[Marta Garcia-Granero]

Hi Bruce:

I just dropped by to add another paper to your collection:

Anna Hart. Mann-Whitney test is not just a test of medians: differences
in spread can be important. BMJ 2001;323:391
http://www.bmj.com/content/323/7309/391

Best regards
Marta (running back to Statalist before someone spots me here)

El 29/08/2013 23:46, Bruce Weaver escribió:

> In the past, I have referred people to  Zimmerman's (2003) simulation study
> <http://imaging.mrc-cbu.cam.ac.uk/statswiki/FAQ/paranp?action=AttachFile&do=view&target=MWZim.pdf>
> which shows that the Wilcoxon-Mann-Whitney test is extremely sensitive to
> heterogeneity of variance.  A later study by  Fagerland & Sandvik (2009)
> <http://imaging.mrc-cbu.cam.ac.uk/statswiki/FAQ/paranp?action=AttachFile&do=get&target=MannW.pdf>
> shows the same thing.  I just learned of another short and very readable
> article (with a good title) that emphasizes the same point:
>
> Johnson, D. H. (1995).  Statistical Sirens: The Allure of Nonparametrics
> <http://www.jstor.org/stable/1940733>  . Ecology, 76, 1998–2000.
>
> HTH.
> Bruce
>
>
>
>
> -----
> --
> Bruce Weaver
> [hidden email]
> http://sites.google.com/a/lakeheadu.ca/bweaver/
>
> "When all else fails, RTFM."
>
> NOTE: My Hotmail account is not monitored regularly.
> To send me an e-mail, please use the address shown above.
>
> --
> View this message in context: http://spssx-discussion.1045642.n5.nabble.com/OT-Nice-article-on-the-allure-of-nonparametrics-tp5721786.html
> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
> .
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Kirill Orlov]

Marta
Seemingly this same article is available in free PDF here
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1120984/pdf/391.pdf

03.09.2013 12:19, Marta García-Granero пишет:

Hi Bruce:

I just dropped by to add another paper to your collection:

Anna Hart. Mann-Whitney test is not just a test of medians: differences
in spread can be important. BMJ 2001;323:391
http://www.bmj.com/content/323/7309/391

Best regards
Marta (running back to Statalist before someone spots me here)