How to obtain interrater reliability in SPSS

[Xu, Lihua]

Hi all,

 

A colleague of mine asked me how to obtain interrater reliability, Cohen's Kappa, for 7 raters over 13 items. I figured out that by using cross-tab function and it is possible to get interrater reliability between two raters. My question is what SPSS syntax i could use to get the reliability among 7 raters.

 

Thanks a lot for your help

 

Lihua

 

Lihua Xu, Ph.D.

Research, Evaluation, Measurement and Statistics

College of Education

Oklahoma State University

Email: [hidden email]

Phone: [phone]

[Bruce Weaver]

The most common form of ICC (and the default in SPSS, IIRC) is equivalent to weighted kappa (with quadratic weights).  So one option is to just compute an ICC, and call it a weighted kappa, if that's what is more familiar to the readership/audience.  If you need a reference, see the repeated measures ANOVA chapter in Biostatistics - The Bare Essentials, by Norman and Streiner.  

HTH.

Xu, Lihua wrote

Hi all,

A colleague of mine asked me how to obtain interrater reliability, Cohen's Kappa, for 7 raters over 13 items. I figured out that by using cross-tab function and it is possible to get interrater reliability between two raters. My question is what SPSS syntax i could use to get the reliability among 7 raters.

Thanks a lot for your help

Lihua

Lihua Xu, Ph.D.
Research, Evaluation, Measurement and Statistics
College of Education
Oklahoma State University
Email: [e-mail]<mailto:[e-mail]>
Phone: [phone]