I would use intraclass correlations and examine the variances
contributions of each facet in the model. The variance components for
the rater by candidate, item by candidate, and rater by item by
candidate would constitute the relative error in this model.
Paul R. Swank, Ph.D.
Professor and Director of Research
Children's Learning Institute
University of Texas Health Science Center - Houston
-----Original Message-----
From: SPSSX(r) Discussion [mailto:
[hidden email]] On Behalf Of
Dwight Harley
Sent: Saturday, March 15, 2008 10:24 PM
To:
[hidden email]
Subject: Inter rater reliability
Does anyone have a suggestion on how to address the following problem?
Two raters, rate n candidates on 19 different items. The rating scale
is binary. I am interested in estimating inter-rater reliability. I
don't think that Kappa will work as there are 19 ratings per individual.
Any sugestions would be appreciated.
D. Harley
==========To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Locked
