Posted by
Maurice Vergeer on
Aug 03, 2011; 7:50pm
URL: http://spssx-discussion.165.s1.nabble.com/Interrater-reliability-tp4663573p4663838.html
I can't help you with the Kappa coefficient. Still, you might want to read Hayes & Krippendorff (2007) Answering the Call for a Standard Reliability Measure for Coding Data. COMMUNICATION METHODS AND MEASURES, 1(1), 77–89 arguing in favor of Krippendorff's alpha.
Follow this link for the pdf and a spss-macro for calculating alpha:
http://www.afhayes.com/spss-sas-and-mplus-macros-and-code.htmlHTH
Maurice
On Wed, Aug 3, 2011 at 8:41 PM, Lovins, Brian (lovinsbk)
<[hidden email]> wrote:
Good afternoon,
I am looking to calculate Kappa to determine a measure of interrater reliability. I currently have 125 subjects being rated by different pairs of staff. Each pair assesses the same person, but there are 125 different pairs of staff. I want to calculate the overall Kappa for the entire group. I can do it for the individual pairs and average the scores, but was hoping there was a syntax/macro that I could use that would calculate the overall Kappa. The data are formatted as follows, but I can restructure the data if needed:
|
Rater1a | Rater1b
| Rater2a |
Rater2b |
Item 1-Subject 1
| Item 1-Subject 1
| Item 1-Subject 2
| Item 1 – Subject 2
|
Item 2 – Subject 1 |
Item 2 – subject 1 |
Item 2-Subject 2 |
Item 2 – Subject 2 |
Thanks
Brian
--
___________________________________________________________________
Maurice Vergeer
Department of communication, Radboud University (
www.ru.nl)
PO Box 9104, NL-6500 HE Nijmegen, The Netherlands
Visiting Professor Yeungnam University, Gyeongsan, South Korea
Recent publications:
-Vergeer, M., Eisinga, R. & Franses, Ph.H. (forthcoming). Supply and
demand effects in television viewing. A time series analysis.
Communications - The European Journal of Communication Research.
-Vergeer, M. Lim, Y.S. Park, H.W. (forthcoming). Mediated Relations: New Methods to study Online Social Capital. Asian Journal of Communication.
-Vergeer, M., Hermans, L., & Sams, S. (forthcoming). Online social networks and micro-blogging in political campaigning: The exploration of a new campaign tool and a new campaign style. Party Politics.
-Pleijter, A., Hermans, L. & Vergeer, M. (forthcoming). Journalists and journalism in the Netherlands. In D. Weaver & L. Willnat, The Global Journalist in the 21st Century. London: Routledge.
Webspace
www.mauricevergeer.nl
http://blog.mauricevergeer.nl/www.journalisteninhetdigitaletijdperk.nlmaurice.vergeer (skype)
___________________________________________________________________