post hoc correction of non-parametric test

[Robinson Aschoff]

Hello,

maybe you can give me some advice:

I have calculated a Kruskal Wallis test over six groups which turned out
to be significant. Now I want to use Mann Whitney tests as post hoc test
to determine the specific differences between those groups. Is it right
that I have to use the Bonferroni correction for this post hoc tests? Are
there any other corrections which are adequate but which are not as
conservative as the Bonferroni correction?

Thanks a lot,
Robinson

[Marta García-Granero]

Hi Robinson

Wednesday, November 22, 2006, 9:58:24 PM, You wrote:

RA> I have calculated a Kruskal Wallis test over six groups which turned out
RA> to be significant. Now I want to use Mann Whitney tests as post hoc test
RA> to determine the specific differences between those groups. Is it right
RA> that I have to use the Bonferroni correction for this post hoc tests? Are
RA> there any other corrections which are adequate but which are not as
RA> conservative as the Bonferroni correction?

There are several modified (stepwise) Bonferroni methods: Holm,
Hommel, Finner...

Depending on the SPSS version you are using (older/newer than 12) you
can use one of these two approaches:

1) Older SPSS versions, use the following syntax:
http://pages.infinit.net/rlevesqu/Syntax/Unclassified/AdjustedP-ValuesAlgorithms.txt

(I have given you the link to the old -now frozen- site because I had
problems trying to access the new one, at http://www.spsstools.net/SampleSyntax.htm
I hope it is temporary...).

There is a complete set of references (for each method) at that link.

2) Newer versions, try this macro (see references on link above):

Warning: This MACRO extracts and adjusts exact p-values. If you have
big sample sizes, then it should be modified to keep asymptotic
p-values (see below the commands after this line: "* Keep only exact
p-values *.") or replaced by another MACRO (I can send it to yo, too)
that performs asymptotic testing.

********************************************************************
* Multiple Mann-Whitney's U tests exact p-values adjusting methods *
********************************************************************
*   Warning: All the code requires SPSS 12 or newer (uses OMS)     *
********************************************************************
*   (C) Marta García-Granero ([hidden email]) 04/20/2006   *
********************************************************************
* Important: 'C:\Temp' folder must exist *.

DEFINE MultiU (!POSITIONAL !TOKENS(1)/!POSITIONAL !CHAREND('(')/
  !POSITIONAL !CHAREND(',')/!POSITIONAL !CHAREND(')') ).
SET OLANG=ENGLISH./* This step is important if other language is used *.
OMS /SELECT TABLES
 /IF COMMANDS='NPar Tests' SUBTYPES='Mann Whitney Test Statistics'
 /DESTINATION FORMAT=SAV
  OUTFILE='C:\Temp\MultiMWU&P.sav'.
DO IF $CASENUM=1.
- WRITE OUTFILE "c:\temp\multiman.sps" /"NPAR TESTS".
- NUMERIC #I #J (F2.0).
- LOOP #I=!3 to !4-1.
-  LOOP #J=#I+1 to !4.
-   WRITE OUTFILE "c:\temp\multiman.sps"
    /"  /M-W= "!QUOTE(!1)" BY "!QUOTE(!2)" (" #I #J ")".
-  END LOOP.
- END LOOP.
- WRITE OUTFILE "c:\temp\multiman.sps" /".".
END IF.
EXECUTE.
INCLUDE FILE='C:\Temp\multiman.sps'.
OMSEND.
* SET OLANG=SPANISH. /Unremark this line & set language back to yours *.
* Direct adjustment of Exact p-values *.
GET FILE='C:\Temp\MultiMWU&P.sav' /DROP=Command_ TO Label_.
* Keep only exact p-values *.
SELECT IF Var1='Exact Sig. [2*(1-tailed Sig.)]'.
EXECUTE. /* Necessary before DELETE VARIABLES *.
DELETE VARIABLES Var1.
RENAME VARIABLES (ALL=pvalue).
COMPUTE id = $CASENUM.
FORMAT id (F2.0).
SORT CASES BY  pvalue (A) .
COMPUTE pos = $CASENUM.
FORMAT pos (F2.0).
PRESERVE.
SET ERRORS=NONE RESULTS=NONE. /* Output is shut-down *.
RANK pvalue /n into N /PRINT = NO.
RESTORE.                      /* Output is restored *.
COMPUTE bonferr=MIN(pvalue*n,1).
COMPUTE sidak=1-(1-pvalue)**n.
COMPUTE holm = MIN(1,(n-pos+1)*pvalue).
IF (holm LT LAG(holm)) holm = LAG(holm).
COMPUTE downsidk = 1-(1-pvalue)**(n-pos+1).
IF (downsidk LT LAG(downsidk)) downsidk = LAG(downsidk).
COMPUTE finner = 1-(1-pvalue)**(n/pos).
IF (finner LT LAG(finner)) finner = LAG(finner).
COMPUTE cn = cn+1/pos.
LEAVE cn. /* With thanks to Richard Ristow *.
SORT CASES BY pos(D).
IF cn LT LAG(cn) cn = LAG(cn).
COMPUTE hommel = MIN(1,cn*n*pvalue/pos).
IF (hommel GT LAG(hommel)) hommel = LAG(hommel).
COMPUTE hochberg = (n-pos+1)*pvalue.
IF (hochberg GT LAG(hochberg)) hochberg = LAG(hochberg).
COMPUTE simes = n*pvalue/pos.
IF (simes GT LAG(simes)) simes = LAG(simes).
EXECUTE./* Needed to DELETE VARIABLES in next line *.
DELETE VARIABLES pos,n,cn.
* FINAL REPORT *.
FORMAT pvalue bonferr to simes (F9.4).
VARIABLE LABELS id 'Nr.' /pvalue 'Original p-value'
  /bonferr 'One-step Bonferroni' /sidak 'One-step Sidak'
  /holm 'Step-down Holm' /downsidk 'Step-down Dunn-Sidak'
  /finner 'Step-down Finner' /hommel 'Step-up Hommel'
  /hochberg 'Step-up Hochberg' /simes 'Step-up Simes'.
SORT CASES BY id (A).
OMS /SELECT TABLES
 /IF COMMANDS='Summarize' SUBTYPES='Case Processing Summary'
 /DESTINATION VIEWER=NO.
 SUMMARIZE
  /TABLES = pvalue bonferr TO simes
  /FORMAT = LIST NOCASENUM TOTAL
  /TITLE = 'Original & adjusted (several methods) p-values'
  /MISSING = VARIABLE
  /CELLS = NONE.
OMSEND.
!ENDDEFINE.

* Example *.
DATA LIST FREE/group glucose (2 F8.0).
BEGIN DATA
1 51 1 56 1 58 1 60 1 62 1 63 1 65 1 68 1 72 1 73
2 60 2 65 2 66 2 68 2 68 2 69 2 73 2 75 2 78 2 80
3 69 3 73 3 74 3 78 3 79 3 79 3 82 3 85 3 87 3 88
4 70 4 75 4 76 4 77 4 79 4 80 4 82 4 86 4 88 4 89
END DATA.
VALUE LABEL group 1'Control' 2'Respiratory' 3'Metabolic' 4'Mixed'.

* MACRO call *.
MultiU glucose group(1,4).



--
Regards,
Dr. Marta García-Granero,PhD           mailto:[hidden email]
Statistician

---
"It is unwise to use a statistical procedure whose use one does
not understand. SPSS syntax guide cannot supply this knowledge, and it
is certainly no substitute for the basic understanding of statistics
and statistical thinking that is essential for the wise choice of
methods and the correct interpretation of their results".

(Adapted from WinPepi manual - I'm sure Joe Abrahmson will not mind)