Is Dunn's test good/right as the pairwise host hoc after Friedman?

[Kirill Orlov]

SPSS Nonparametric Tests offer pairwise multiple comparisons post hoc
test in a form of Dunn's test (1964) with Bonferroni adjustement.

Dunn originally invented it to be used after Kruskal-Wallis independent
samples test. SPSS uses it also after other omnibus tests, for example,
Friedman related samples test.

I want to question whether it is right/good to use the Dunn's test after
Friedman test. I'd like to ask multiple comparison gurus here, including
SPSS statisticians, - what they think.

For, as one recent post on CrossValidated:
http://stats.stackexchange.com/questions/175441/friedmans-test-spss-22-significant-but-post-hoc-comparisons-not-significan
has demonstrated, Dunn's test can be nonsignificant (even before
Bonferroni adjustment) when Friedman omnibus test is far significant;
i.e. it fails to discover which samples differ despite that there surely
some must differ.

In my (ttnphns) answer to the aforementioned post I considered possible
reasons why Dunn (after Friedman) is so weak, but I can't arrive at any
conclusion: is Dunn's test, as SPSS adopts it, a due pairwise post hoc
after Friedman? If yes should it be better "calibrated"?


=====================
To manage your subscription to SPSSX-L, send a message to
[e-mail] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Kirill Orlov]

P.S.
One hint. Friedman test seems to adjust for ties (see it formula in SPSS
Algorithms, Nonparametric tests). But Dunn's test (as is used by SPSS
after Friedman)  lacks any similar adjustment.  There. Am I right?

=====================
To manage your subscription to SPSSX-L, send a message to
[e-mail] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Kirill Orlov]

Publishing Friedman / post hoc syntax, - for those who wishes to follow and play around with the problem.
Take any variables or those three from the question on CrossValidated.

*Perform ranking within case. Do it as you wish - by transposing the dataset and then ranking or using my macro from http://www.spsstools.net/en/KO-spssmacros ("Horizontal tools").
!hrank  vars= v1 to v3
       /cap= rnk     
       /ties= MEAN
       /order= A.

*Statistical testing (Following SPSS Algorithms).
set mxloops 1000000.
matrix.
get rnk /vari= rnk1 to rnk3.
comp n= nrow(rnk).
comp k= ncol(rnk).
comp c= csum(rnk).

* Processing ties.
comp sumties= make(n,1,0).
loop i= 1 to n.
-comp case= rnk(i,:).
-do if rmax(case)=rmin(case).
- comp sumties(i)= k**3-k.
-else.
- comp des= design(t(case)).
- comp ties= csum(des).
- comp sumties(i)= rsum(ties&**3-ties).
-end if.
end loop.
comp sumties= csum(sumties).

* Friedman statistic and test.
comp stat= ( (12/(n * k*(k+1))) * rsum(c&**2) - 3*n*(k+1) ) /
           ( 1 - sumties/(n * k*(k**2-1)) ).

print (c/n) /title 'Mean rank in each sample (variable)'.
print stat /title 'Chi-sq test statistic'.
print (k-1) /title 'df'.
print (1-chicdf(stat,k-1)) /title 'Asymp Sig' /format= f8.6.
*----------------.

* Post hoc pairwise comparisons (SPSS "Dunn's, 1964" approach with Bonferroni correction).
comp ncompar= k*(k-1)/2.
comp sigma= sqrt(k*(k+1)/(6*n)).
loop i= 1 to k-1.
-loop j= i+1 to k.
- comp pwstat= (c(i)-c(j))/n.
- comp stpwstat= pwstat/sigma.
- comp sig= 2*(1-cdfnorm(abs(stpwstat))).
- comp adjsig= sig*ncompar.
- do if adjsig>1.
-  comp adjsig= 1.
- end if.
- print {i,j,pwstat,sigma,stpwstat,sig,adjsig}
     /clab= 'Sample1' 'Sample2' 'TestStat' 'StError' 'Z' 'Sig2side' 'AdjSig'
     /title ' ' /format= f8.6.
-end loop.
end loop.
end matrix.

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Kirill Orlov]

Conclusion from the cited post
http://stats.stackexchange.com/questions/175441/friedmans-test-is-very-significant-but-its-post-hoc-comparisons-spss-are-not/175529#175529

In the presense of only k=2 samples in data a correct post hoc pairwise
comparison test must give the same result (statistic and p-value) as the
omnibus test - it is actually a property which proves that the post hoc
test corresponds (is isomorphic) to the parent omnibus test. It is
indeed so with Kruskal-Wallis test and Dunn's test - just program it
following SPSS Algorithms and test with |V1| and |V2| as two independent
groups, and you'll get same |p=.0153| both for KW and for Dunn. But we
saw that a similar equivalence is absent in relations between Friedman
test and "Dunn's approach" post-Friedman comparison test.

Conclusion. Post hoc multiple comparison test being performed by SPSS
(version 22 and earlier) after Friedman's test is defective. Maybe it is
correct when there is no ties, but I don't know. The post hoc test does
not treat ties the way Friedman does it (while it should). I cannot say
anything about the formula of st. error, |sqrt[k*(k+1)/(6n)]|, they are
using: it was derived from discrete uniform distribution, but they
didn't write how; is it correct? Either the "Dunn's test approach" was
adapted to Friedman inadequatly by SPSS or Dunn's test cannot be adapted
to Friedman at all.

Some other nonparametric post hoc test might turn to be affected, too,
especially tests after k-related-samples tests.


=====================
To manage your subscription to SPSSX-L, send a message to
[e-mail] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Kirill Orlov]

Here is syntax to perform Kruskal-Wallis with its pairwise post hoc test (Dunn's test). For those who happens to wish to play around the problem.
This procedure (it follows SPSS Algorithms exactly)  is correct.
(Similar procedure for Friedman + its post hoc published earlier on this thread also follows SPSS Algoritms exactly. However, the post hoc there is incorrect.).

*Variable V and grouping variable GR.
*Rank cases (all groups together).
RANK
  VARIABLES= v (A) /RANK /PRINT=NO
  /TIES=MEAN .

*Rank sums and frequencies in groups.
*Save as external file.
AGGREGATE
  /OUTFILE= 'C:\bygroups.sav'
  /BREAK= gr
  /sum= sum(rv)
  /n= N.

*Frequency of distinct values (i.e. number of ties for each value).
*Save as external file.
AGGREGATE
  /OUTFILE= 'C:\tied.sav'
  /BREAK= rv
  /n= N.

matrix.
*Read in rank sums and frequencies by groups.
get bygroups /file= 'C:\bygroups.sav'.
comp gr= bygroups(:,1). /*Group codes
comp r= bygroups(:,2). /*Rank sum in groups
comp grn= bygroups(:,3). /*n in groups
comp n= csum(grn). /*Total N
comp k= nrow(grn). /*Number of groups

*Read in Ties frequencies.
get tied /file= 'C:\tied.sav' /var= n.
comp sumties= csum(tied&**3-tied). /*Process it

*test statistic and significance.
comp stat= ( 12/(n*(n+1)) * csum(r&**2/grn) - 3*(n+1) ) /
           ( 1 - sumties /(n**3-n) ).
print {gr,r/grn} /title 'Mean rank in each sample (group)'.
print stat /title 'Chi-sq test statistic'.
print (k-1) /title 'df'.
print (1-chicdf(stat,k-1)) /title 'Asymp Sig' /format= f8.6.
*----------------.

*Post hoc pairwise comparisons (Dunn's, 1964 approach with Bonferroni correction).
comp ncompar= k*(k-1)/2.
loop i= 1 to k-1.
-loop j= i+1 to k.
- comp pwstat= r(i)/grn(i)-r(j)/grn(j).
- comp sigma= sqrt(( n*(n+1)/12 - sumties/(12*(n-1)) ) * ( 1/grn(i)+1/grn(j) )).
- comp stpwstat= pwstat/sigma.
- comp sig= 2*(1-cdfnorm(abs(stpwstat))).
- comp adjsig= sig*ncompar.
- do if adjsig>1.
-  comp adjsig= 1.
- end if.
- print {gr(i),gr(j),pwstat,sigma,stpwstat,sig,adjsig}
     /clab= 'Sample1' 'Sample2' 'TestStat' 'StError' 'Z' 'Sig2side' 'AdjSig'
     /title ' ' /format= f8.6.
-end loop.
end loop.
end matrix.
erase file 'C:\bygroups.sav'.
erase file 'C:\tied.sav'.
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Bruce Weaver]

Hi Kirill.  If you use the data on that Stackexchange site, what p-values do you get for the 3 pair-wise comparisons using your code?  

I ask, because it seems to me that another option (especially with a reasonably large N) is to just perform a repeated measures ANOVA (with all pair-wise contrasts) on the ranks within ID.  I did that (see syntax below), and get the following results:

Overall Test:  F(2, 798) = 6.136, p = .002 (same p-value as the Friedman test)

V1-V2:  p_unadj = .001, p_adj = .002
V1-V3: p_unadj = .007, p_adj = .021
V2-V3: p_unadj = .696, p_adj = 1.000

Where p_unadj = the unadjusted p-value, and p_adj = the p-value with the Bonferroni adjustment.  

I suspect that these p-values will be pretty close to what you're code is giving.  

Cheers,
Bruce

-----------------------------------------------------------------

* This example uses the data found at
* http://stats.stackexchange.com/questions/175441/friedmans-test-is-very-significant-but-its-post-hoc-comparisons-spss-are-not.
.
* A long DATA LIST to read in the data is appended below.

* When the sample size is relatively large (and it is here),
* a repeated measures ANOVA on the ranks (within ID) should
* be more or less equivalent to the Friedman test.  

* Restructure the data to LONG format.
VARSTOCASES
  /ID=id
  /MAKE V FROM V1 V2 V3
  /INDEX=i(3)
  /KEEP=
  /NULL=KEEP.

* Rank the scores within ID.
RANK VARIABLES=V (A) BY id
  /RANK
  /PRINT=YES
  /TIES=MEAN.
FORMATS RV (F5.2).

* Restructure back to WIDE format.
CASESTOVARS
  /ID=id
  /INDEX=i
  /GROUPBY=VARIABLE
  /SEPARATOR="".

* Friedman test via NPAR TESTS.
NPAR TESTS
  /FRIEDMAN=V1 V2 V3
  /STATISTICS DESCRIPTIVES
  /MISSING LISTWISE.

* Friedman's test with NPTESTS with pairwise comparisons.
* Must set variable level to ORDINAL first.
VARIABLE LEVEL V1 to V3 (ORDINAL).
*Nonparametric Tests: Related Samples.
NPTESTS
  /RELATED TEST(V1 V2 V3) FRIEDMAN(COMPARE=PAIRWISE)
  /MISSING SCOPE=ANALYSIS USERMISSING=EXCLUDE
  /CRITERIA ALPHA=0.05  CILEVEL=95.

* With large N, Friedman test is more or less the
* same thing as repeated measures ANOVA on the ranks
* within ID.

GLM RV1 RV2 RV3
  /WSFACTOR=Condition 3 Polynomial
  /METHOD=SSTYPE(3)
  /EMMEANS=TABLES(Condition) COMPARE ADJ(LSD)
  /EMMEANS=TABLES(Condition) COMPARE ADJ(BONFERRONI)
  /CRITERIA=ALPHA(.05)
  /WSDESIGN=Condition.

* Here's that LONG DATA LIST command.

NEW FILE.
DATASET CLOSE all.
DATA LIST list / V1  V2  V3 (3F1).
BEGIN DATA
5   5   5
4   4   5
5   3   5
4   5   5
5   5   5
5   5   5
5   5   4
5   5   5
5   5   4
5   5   5
5   5   5
4   4   4
4   4   4
4   5   5
3   3   3
4   4   5
3   5   2
5   5   5
3   3   5
4   4   4
5   5   5
5   4   5
5   5   5
5   5   5
4   4   5
5   5   5
5   5   5
5   4   5
5   5   5
5   5   5
4   4   4
4   4   4
5   5   5
4   4   4
4   5   4
5   5   5
4   4   4
4   4   4
4   5   4
5   5   5
5   5   5
5   5   5
5   4   4
5   5   5
4   5   5
5   5   5
5   5   5
5   5   5
5   5   5
4   4   4
5   5   4
5   5   5
5   5   4
5   4   4
5   5   5
4   4   4
4   4   4
5   4   3
5   5   4
4   5   4
5   5   5
5   5   5
4   4   4
5   5   4
5   4   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   4   5
5   5   5
5   5   5
5   4   5
5   5   5
5   5   5
5   5   4
4   4   4
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   4   5
5   5   5
5   5   5
5   5   5
4   4   3
4   4   4
5   5   4
4   4   5
4   5   4
4   3   4
4   4   4
4   4   4
4   4   4
5   4   4
5   4   4
2   2   3
4   4   5
4   4   4
5   4   5
4   4   3
4   4   4
4   4   5
5   2   5
4   3   5
4   4   4
4   5   4
4   4   4
4   5   5
5   5   5
5   5   5
4   5   4
5   3   5
5   5   5
5   4   5
5   3   5
2   3   5
5   5   5
5   5   5
4   4   4
5   5   4
4   5   5
5   5   5
5   5   5
3   4   4
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   4
5   5   5
5   5   5
5   5   3
5   5   3
5   5   5
5   5   3
5   5   4
5   5   3
5   5   3
5   5   5
5   5   5
5   5   3
5   5   4
5   5   3
5   5   5
5   5   3
5   5   5
5   5   3
5   5   4
5   5   5
5   5   5
4   4   4
4   4   4
3   4   4
4   5   5
3   5   4
3   5   4
5   5   5
3   3   4
5   5   5
5   5   5
5   5   4
4   4   4
4   4   4
4   4   4
5   5   5
3   2   4
3   2   4
4   4   5
5   5   5
3   1   2
5   4   1
5   4   5
5   5   5
5   4   3
4   5   4
2   3   5
3   2   1
3   2   2
5   5   5
4   4   5
5   5   1
5   3   3
3   3   4
5   3   4
4   5   5
5   4   3
5   1   4
4   2   2
4   4   2
5   2   1
4   4   5
5   3   5
5   3   5
2   5   4
4   3   4
5   4   4
5   2   1
5   4   2
3   1   5
4   4   5
5   4   2
3   4   1
5   3   2
5   4   5
4   1   5
5   4   5
4   3   5
5   4   5
4   5   5
5   4   4
5   2   2
4   5   4
4   4   5
5   5   3
4   5   4
5   4   4
5   4   4
5   5   5
4   4   4
5   5   5
5   4   3
5   5   5
5   5   5
5   4   5
5   5   5
5   5   5
5   5   5
5   5   5
4   5   5
5   4   4
5   5   5
4   4   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
2   4   5
4   4   4
5   4   4
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
4   4   4
5   5   5
4   5   4
5   4   5
5   5   4
5   4   4
5   5   5
5   2   3
5   2   2
5   2   1
1   1   1
4   4   3
4   4   4
5   4   4
5   5   4
5   4   5
5   4   3
3   5   5
4   3   4
4   3   4
4   4   5
4   4   3
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
4   4   4
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
4   4   4
5   5   5
5   5   5
5   5   5
5   5   5
4   4   4
4   4   4
5   5   5
5   5   4
4   5   5
5   4   4
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
4   4   5
2   4   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
4   4   4
5   5   5
5   5   5
5   4   4
5   4   4
5   5   5
5   5   5
4   5   4
4   4   4
4   3   4
4   4   3
5   4   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   4   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
4   5   5
5   5   5
4   5   4
5   5   5
1   5   4
5   4   5
5   5   5
5   5   5
4   4   4
4   2   5
5   5   5
3   4   5
5   5   5
4   4   4
5   4   4
5   4   5
5   5   5
4   3   4
4   4   4
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
END DATA.

Kirill Orlov wrote

Publishing Friedman / post hoc syntax, - for those who wishes to follow
and play around with the problem.
Take any variables or those three from the question on CrossValidated.

*Perform ranking within case. Do it as you wish - by transposing the
dataset and then ranking or using my macro from
http://www.spsstools.net/en/KO-spssmacros ("Horizontal tools").
!hrank  vars= v1 to v3
        /cap= rnk
        /ties= MEAN
        /order= A.

*Statistical testing (Following SPSS Algorithms).
set mxloops 1000000.
matrix.
get rnk /vari= rnk1 to rnk3.
comp n= nrow(rnk).
comp k= ncol(rnk).
comp c= csum(rnk).

* Processing ties.
comp sumties= make(n,1,0).
loop i= 1 to n.
-comp case= rnk(i,:).
-do if rmax(case)=rmin(case).
- comp sumties(i)= k**3-k.
-else.
- comp des= design(t(case)).
- comp ties= csum(des).
- comp sumties(i)= rsum(ties&**3-ties).
-end if.
end loop.
comp sumties= csum(sumties).

* Friedman statistic and test.
comp stat= ( (12/(n * k*(k+1))) * rsum(c&**2) - 3*n*(k+1) ) /
            ( 1 - sumties/(n * k*(k**2-1)) ).

print (c/n) /title 'Mean rank in each sample (variable)'.
print stat /title 'Chi-sq test statistic'.
print (k-1) /title 'df'.
print (1-chicdf(stat,k-1)) /title 'Asymp Sig' /format= f8.6.
*----------------.

* Post hoc pairwise comparisons (SPSS "Dunn's, 1964" approach with
Bonferroni correction).
comp ncompar= k*(k-1)/2.
comp sigma= sqrt(k*(k+1)/(6*n)).
loop i= 1 to k-1.
-loop j= i+1 to k.
- comp pwstat= (c(i)-c(j))/n.
- comp stpwstat= pwstat/sigma.
- comp sig= 2*(1-cdfnorm(abs(stpwstat))).
- comp adjsig= sig*ncompar.
- do if adjsig>1.
-  comp adjsig= 1.
- end if.
- print {i,j,pwstat,sigma,stpwstat,sig,adjsig}
      /clab= 'Sample1' 'Sample2' 'TestStat' 'StError' 'Z' 'Sig2side'
'AdjSig'
      /title ' ' /format= f8.6.
-end loop.
end loop.
end matrix.


=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Kirill Orlov]

Hi Bruce.

-->If you use the data on that Stackexchange site, what p-values do you get for the 3 pair-wise comparisons using your code?

As I've said in my answer to that question on Stackexchange (user ttnphns who answered was me), "I've just sat and programmed Friedman's test and its post-hoc pairwise comparisons following the above SPSS algorithms documentation, and I confirm that there is no bug and that my results were identical to what SPSS output". In other words, the results are what SPSS NPTESTS output:

Mean rank in each sample (variable)
   2.081250000   1.951250000   1.967500000
Friedman's test [Correct]:
Chi-sq test statistic
   12.11698113
df
  2
Asymp Sig
  .002338
 
Pairwise comparisons [Incorrect]:
  Sample1  Sample2 TestStat  StError        Z Sig2side   AdjSig
    V1       V2    .130000   .070711 1.838478  .065992  .197976
    V1       V3    .113750   .070711 1.608668  .107689  .323067
    V2       V3   -.016250   .070711 -.229810  .818240 1.000000

You may check it yourself using my code.

-->it seems to me that another option (especially with a reasonably large N) is to just perform a repeated measures ANOVA (with all pair-wise contrasts) on the ranks within ID. I did that (see syntax below)... With large N, Friedman test is more or less the same thing as repeated measures ANOVA on the ranks within ID.

This is a good observation. Indeed, it would make a reliable work-around until SPSS fixes their defect. However, note that the RM-ANOVA on the Friedman ranks, proposed by you, is not exactly the true post-Friedman multiple comparison test (which SPSS haven't yet formulated correctly). For example, if you use only V1 and V2 variables of the three and perform your RM-ANOVA you'll get significance p=.000221, whereas Friedman's (which is expected to be identical to its own post hoc in case of 2 samples) sig. is p=.000244. Not exactly the same.

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Martin Holt-3]

I've watched this with interest.

An excellent forum for this sort of query is MedStats (I'm biased...I'm the founder). It hosts such notable statisticians as Martin Bland, Doug Altman, Frank Harrell Jnr, Michael Campbell, Stephen Senn....to name but a few..

Once you join you can submit posts, and members will discuss your problem in an open way: the whole discussion is clear to see.

In addition, once you have joined you have access to the Group's archives...truly a fantastic asset.

Just so that you know, anyone in the world who matters () can view your post and subsequent discussion, even if they haven't actually joined.

Finally, the Group's Rules include one: no flaming...so don't be shy......

Best Wishes,

Martin

 

---

From: Bruce Weaver <[hidden email]>
To: [hidden email]
Sent: Friday, 9 October 2015, 19:12
Subject: Re: Is Dunn's test good/right as the pairwise host hoc after Friedman?

Hi Kirill.  If you use the data on that Stackexchange site, what p-values do
you get for the 3 pair-wise comparisons using your code? 

I ask, because it seems to me that another option (especially with a
reasonably large N) is to just perform a repeated measures ANOVA (with all
pair-wise contrasts) on the ranks within ID.  I did that (see syntax below),
and get the following results:

Overall Test:  F(2, 798) = 6.136, p = .002 (same p-value as the Friedman
test)

V1-V2:  p_unadj = .001, p_adj = .002
V1-V3: p_unadj = .007, p_adj = .021
V2-V3: p_unadj = .696, p_adj = 1.000

Where p_unadj = the unadjusted p-value, and p_adj = the p-value with the
Bonferroni adjustment. 

I suspect that these p-values will be pretty close to what you're code is
giving. 

Cheers,
Bruce

-----------------------------------------------------------------

* This example uses the data found at
*
http://stats.stackexchange.com/questions/175441/friedmans-test-is-very-significant-but-its-post-hoc-comparisons-spss-are-not.
.
* A long DATA LIST to read in the data is appended below.

* When the sample size is relatively large (and it is here),
* a repeated measures ANOVA on the ranks (within ID) should
* be more or less equivalent to the Friedman test. 

* Restructure the data to LONG format.
VARSTOCASES
  /ID=id
  /MAKE V FROM V1 V2 V3
  /INDEX=i(3)
  /KEEP=
  /NULL=KEEP.

* Rank the scores within ID.
RANK VARIABLES=V (A) BY id
  /RANK
  /PRINT=YES
  /TIES=MEAN.
FORMATS RV (F5.2).

* Restructure back to WIDE format.
CASESTOVARS
  /ID=id
  /INDEX=i
  /GROUPBY=VARIABLE
  /SEPARATOR="".

* Friedman test via NPAR TESTS.
NPAR TESTS
  /FRIEDMAN=V1 V2 V3
  /STATISTICS DESCRIPTIVES
  /MISSING LISTWISE.

* Friedman's test with NPTESTS with pairwise comparisons.
* Must set variable level to ORDINAL first.
VARIABLE LEVEL V1 to V3 (ORDINAL).
*Nonparametric Tests: Related Samples.
NPTESTS
  /RELATED TEST(V1 V2 V3) FRIEDMAN(COMPARE=PAIRWISE)
  /MISSING SCOPE=ANALYSIS USERMISSING=EXCLUDE
  /CRITERIA ALPHA=0.05  CILEVEL=95.

* With large N, Friedman test is more or less the
* same thing as repeated measures ANOVA on the ranks
* within ID.

GLM RV1 RV2 RV3
  /WSFACTOR=Condition 3 Polynomial
  /METHOD=SSTYPE(3)
  /EMMEANS=TABLES(Condition) COMPARE ADJ(LSD)
  /EMMEANS=TABLES(Condition) COMPARE ADJ(BONFERRONI)
  /CRITERIA=ALPHA(.05)
  /WSDESIGN=Condition.

* Here's that LONG DATA LIST command.

NEW FILE.
DATASET CLOSE all.
DATA LIST list / V1  V2  V3 (3F1).
BEGIN DATA
5  5  5
4  4  5
5  3  5
4  5  5
5  5  5
5  5  5
5  5  4
5  5  5
5  5  4
5  5  5
5  5  5
4  4  4
4  4  4
4  5  5
3  3  3
4  4  5
3  5  2
5  5  5
3  3  5
4  4  4
5  5  5
5  4  5
5  5  5
5  5  5
4  4  5
5  5  5
5  5  5
5  4  5
5  5  5
5  5  5
4  4  4
4  4  4
5  5  5
4  4  4
4  5  4
5  5  5
4  4  4
4  4  4
4  5  4
5  5  5
5  5  5
5  5  5
5  4  4
5  5  5
4  5  5
5  5  5
5  5  5
5  5  5
5  5  5
4  4  4
5  5  4
5  5  5
5  5  4
5  4  4
5  5  5
4  4  4
4  4  4
5  4  3
5  5  4
4  5  4
5  5  5
5  5  5
4  4  4
5  5  4
5  4  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  4  5
5  5  5
5  5  5
5  4  5
5  5  5
5  5  5
5  5  4
4  4  4
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  4  5
5  5  5
5  5  5
5  5  5
4  4  3
4  4  4
5  5  4
4  4  5
4  5  4
4  3  4
4  4  4
4  4  4
4  4  4
5  4  4
5  4  4
2  2  3
4  4  5
4  4  4
5  4  5
4  4  3
4  4  4
4  4  5
5  2  5
4  3  5
4  4  4
4  5  4
4  4  4
4  5  5
5  5  5
5  5  5
4  5  4
5  3  5
5  5  5
5  4  5
5  3  5
2  3  5
5  5  5
5  5  5
4  4  4
5  5  4
4  5  5
5  5  5
5  5  5
3  4  4
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  4
5  5  5
5  5  5
5  5  3
5  5  3
5  5  5
5  5  3
5  5  4
5  5  3
5  5  3
5  5  5
5  5  5
5  5  3
5  5  4
5  5  3
5  5  5
5  5  3
5  5  5
5  5  3
5  5  4
5  5  5
5  5  5
4  4  4
4  4  4
3  4  4
4  5  5
3  5  4
3  5  4
5  5  5
3  3  4
5  5  5
5  5  5
5  5  4
4  4  4
4  4  4
4  4  4
5  5  5
3  2  4
3  2  4
4  4  5
5  5  5
3  1  2
5  4  1
5  4  5
5  5  5
5  4  3
4  5  4
2  3  5
3  2  1
3  2  2
5  5  5
4  4  5
5  5  1
5  3  3
3  3  4
5  3  4
4  5  5
5  4  3
5  1  4
4  2  2
4  4  2
5  2  1
4  4  5
5  3  5
5  3  5
2  5  4
4  3  4
5  4  4
5  2  1
5  4  2
3  1  5
4  4  5
5  4  2
3  4  1
5  3  2
5  4  5
4  1  5
5  4  5
4  3  5
5  4  5
4  5  5
5  4  4
5  2  2
4  5  4
4  4  5
5  5  3
4  5  4
5  4  4
5  4  4
5  5  5
4  4  4
5  5  5
5  4  3
5  5  5
5  5  5
5  4  5
5  5  5
5  5  5
5  5  5
5  5  5
4  5  5
5  4  4
5  5  5
4  4  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
2  4  5
4  4  4
5  4  4
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
4  4  4
5  5  5
4  5  4
5  4  5
5  5  4
5  4  4
5  5  5
5  2  3
5  2  2
5  2  1
1  1  1
4  4  3
4  4  4
5  4  4
5  5  4
5  4  5
5  4  3
3  5  5
4  3  4
4  3  4
4  4  5
4  4  3
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
4  4  4
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
4  4  4
5  5  5
5  5  5
5  5  5
5  5  5
4  4  4
4  4  4
5  5  5
5  5  4
4  5  5
5  4  4
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
4  4  5
2  4  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
4  4  4
5  5  5
5  5  5
5  4  4
5  4  4
5  5  5
5  5  5
4  5  4
4  4  4
4  3  4
4  4  3
5  4  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  4  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
4  5  5
5  5  5
4  5  4
5  5  5
1  5  4
5  4  5
5  5  5
5  5  5
4  4  4
4  2  5
5  5  5
3  4  5
5  5  5
4  4  4
5  4  4
5  4  5
5  5  5
4  3  4
4  4  4
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
5  5  5
END DATA.



Kirill Orlov wrote

> Publishing Friedman / post hoc syntax, - for those who wishes to follow
> and play around with the problem.
> Take any variables or those three from the question on CrossValidated.
>
> *Perform ranking within case. Do it as you wish - by transposing the
> dataset and then ranking or using my macro from
> http://www.spsstools.net/en/KO-spssmacros ("Horizontal tools").
> !hrank  vars= v1 to v3
>        /cap= rnk
>        /ties= MEAN
>        /order= A.
>
> *Statistical testing (Following SPSS Algorithms).
> set mxloops 1000000.
> matrix.
> get rnk /vari= rnk1 to rnk3.
> comp n= nrow(rnk).
> comp k= ncol(rnk).
> comp c= csum(rnk).
>
> * Processing ties.
> comp sumties= make(n,1,0).
> loop i= 1 to n.
> -comp case= rnk(i,:).
> -do if rmax(case)=rmin(case).
> - comp sumties(i)= k**3-k.
> -else.
> - comp des= design(t(case)).
> - comp ties= csum(des).
> - comp sumties(i)= rsum(ties&**3-ties).
> -end if.
> end loop.
> comp sumties= csum(sumties).
>
> * Friedman statistic and test.
> comp stat= ( (12/(n * k*(k+1))) * rsum(c&**2) - 3*n*(k+1) ) /
>            ( 1 - sumties/(n * k*(k**2-1)) ).
>
> print (c/n) /title 'Mean rank in each sample (variable)'.
> print stat /title 'Chi-sq test statistic'.
> print (k-1) /title 'df'.
> print (1-chicdf(stat,k-1)) /title 'Asymp Sig' /format= f8.6.
> *----------------.
>
> * Post hoc pairwise comparisons (SPSS "Dunn's, 1964" approach with
> Bonferroni correction).
> comp ncompar= k*(k-1)/2.
> comp sigma= sqrt(k*(k+1)/(6*n)).
> loop i= 1 to k-1.
> -loop j= i+1 to k.
> - comp pwstat= (c(i)-c(j))/n.
> - comp stpwstat= pwstat/sigma.
> - comp sig= 2*(1-cdfnorm(abs(stpwstat))).
> - comp adjsig= sig*ncompar.
> - do if adjsig>1.
> -  comp adjsig= 1.
> - end if.
> - print {i,j,pwstat,sigma,stpwstat,sig,adjsig}
>      /clab= 'Sample1' 'Sample2' 'TestStat' 'StError' 'Z' 'Sig2side'
> 'AdjSig'
>      /title ' ' /format= f8.6.
> -end loop.
> end loop.
> end matrix.
>
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> [hidden email]

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Is-Dunn-s-test-good-right-as-the-pairwise-host-hoc-after-Friedman-tp5730744p5730755.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD