Cohen's Kappa for multiple raters

Posted by Paul Mcgeoghan on Nov 09, 2006; 4:33pm
URL: http://spssx-discussion.165.s1.nabble.com/Cohen-s-Kappa-for-multiple-raters-tp1072037.html

Hi,

I am using the syntax below from Raynald's SPSS Tools website:
http://www.spsstools.net/Syntax/Matrix/CohensKappa.txt

In my case I have 6 raters rating 5 subjects and there are 2 categories so the data is as below:

Subj   rater1            rater2    rater3         rater4       rater5    rater6
1         2                         2            2               2               2               2
2         2                         2            2               2               2               2
3         2                         2            2               2               2               2
4         1                         2            2               2               2               2
5         2                         2            2               2               2               2

This gives a value of -.0345 which indicates no agreement according to the following article:
http://en.wikipedia.org/wiki/Fleiss%27_kappa

Most of the raters agree in the above table so why is Cohen's Kappa negative, indicating no
agreement.

Also in the output, it gives Cohen's Kappa where Kappa is -.0345 and
Cohen's Kappa Fleiss - adjusted standard error as .1155

Which value do I use for Cohen's Kappa among multiple raters, is it -.0345 or .1155?

Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases),
University Infrastructure Group (UIG),
Information Services,
Cardiff University.
Tel. 02920 (875035).