I can send you a macro for Fleiss' kappa, which will give you the overall as well as the category kappa's, based on Fleiss' 1971 Psych. Bulletin article. It uses, however, the updated 1979 formula for variance. How many items will the raters categorize? You have other options, e.g., Krippendorf's alpha and Gwet's AC1 statistic. Although Fleiss is the most used, there's some use of these as well in the literature. If you're interested in Cohen's kappa, there are some multiple rater models, most notably Light (averages all the two-rater kappas) and Hubert (averages the two-rater Pe's). Let me know which you want, and I'll get them off to you.
Brian
-----Original Message-----
From: SPSSX(r) Discussion [mailto:
[hidden email]] On Behalf Of BAM86
Sent: Wednesday, January 09, 2013 3:25 PM
To:
[hidden email]Subject: Fleiss Kappa (Inter-rater reliability questions)
Hi all,
I am conducting research and need some help with doing the Fleiss Kappa
analysis.
First and foremost let me give a bit of a layout of the study. We have a
sample of 75 students in the social sciences who were asked to categorize
potential items for a new measure based on descriptions of 2 theoretical
constructs. This was meant to be a preliminary check to help with construct
validity and take it a step beyond just the research team validating items
initially.
The items are supposed to characterize coping behaviours following romantic
loss. The constructs are loss-oriented coping and restoration-oriented
coping, however, we also gave the option for participants to say potential
items were both types of coping behaviour or something else altogether
(basically an 'other' option).
Therefore we have 75 participants categorizing items into one of 4
categories. We want to have some measure of the inter-rater reliability of
all these participants though. My understanding is that Fleiss Kappa would
be an appropriate tool for this purpose, however, if anyone is aware of
other statistics which may be a better fit for our model then please let me
know.
I also need to the syntax for the Fleiss Kappa statistic if anyone has it
available. Thanks in advance for any help people can provide.
--
View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/Fleiss-Kappa-Inter-rater-reliability-questions-tp5717299.htmlSent from the SPSSX Discussion mailing list archive at Nabble.com.
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD