inter rater reliability multiple attributes

Posted by susan.richardson on Aug 07, 2014; 7:32pm
URL: http://spssx-discussion.165.s1.nabble.com/inter-rater-reliability-multiple-attributes-tp5726913.html

Hello All,

I have 2 raters (S & J) who rate schools on 14 attributes. Each attribute is rated on a dichotomous scale.

I know that I can look at the kappa for each attribute (S1 and J1, S2 and J2, and so forth) but is there any way to account for the fact that I am doing this 14 times? Or a way to summarize overall agreement?

I found the Kupper and Hafner (1989) paper describing what to do with more than one nominal variable being rated, and I am hoping there is an SPSS solution that may be similar(?)

If there isn't a answer, what would you suggest for reporting the range of kappas across attributes?

Thanks,
Susan