Posted by
bdates on
Jan 22, 2018; 2:50pm
URL: http://spssx-discussion.165.s1.nabble.com/interrater-reliabillity-but-for-any-pair-of-raters-tp5735428p5735438.html
Great syntax, David! Thanks. One caution to Talma...the average of all two-rater kappa's is not equivalent to Fleiss' kappa calculated by Fleiss' formula.
Brian
________________________________________
From: SPSSX(r) Discussion [
[hidden email]] on behalf of David Marso [
[hidden email]]
Sent: Monday, January 22, 2018 1:15 AM
To:
[hidden email]
Subject: Re: interrater reliabillity - but for any pair of raters?
Here is a tested version with some revision for efficiency
-Just passes the data once per main iteration rather than each pair-
along with OMS and AGGREGATE.
---SIMULATED DATA--- EXCLUDE FROM PRODUCTION RUN.
NEW FILE.
DATASET CLOSE ALL.
MATRIX.
SAVE TRUNC(UNIFORM(200,30)*4+1) / OUTFILE= * /VARIABLES j1 TO j30.
END MATRIX.
DATASET NAME raw.
DEFINE !AllPairsKappa( raters !CMDEND)
!LET !CPY = !raters
!DO !j1 !IN ( !raters)
!LET !CPY= !TAIL(!CPY)
!IF ( !CPY !NE !NULL) !THEN
CROSSTABS TABLES !j1 BY !CPY /STATISTICS KAPPA.
!IFEND
!DOEND
!ENDDEFINE.
* OMS.
DATASET DECLARE Kappa.
OMS
/SELECT TABLES /IF COMMANDS=['Crosstabs'] /DESTINATION VIEWER=NO.
OMS /SELECT TABLES
/IF COMMANDS=['Crosstabs'] SUBTYPES=['Symmetric Measures']
/DESTINATION FORMAT=SAV NUMBERED=TableNumber_
OUTFILE='Kappa' VIEWER=NO.
!AllPairsKappa raters = j1 j2 j3 j4 j5 j6 j7 j8 j9 j10 j11 j12 j13 j14 j15
j16 j17 j18 j19 j20 j21 j22 j23 j24 j25 j26 j27 j28
j29 j30.
OMSEND.
DATASET ACTIVATE Kappa.
DELETE VARIABLES TableNumber_ Command_ Subtype_ Label_ Var1.
RENAME VARIABLES Var2 =Kappa.
SELECT IF Kappa EQ 'Kappa'.
DATASET DECLARE agg.
AGGREGATE OUTFILE agg /BREAK Kappa /AvgKappa=MEAN( Value).
bdates wrote
> Talma,
>
> Before you abandon David's code, he inserted the OMS section to provide a
> file to average the pair-wise kappa's. If you want to calculate the
> overall kappa in addition to the pairwise kappa's, then pasting and
> inserting syntax for kappa from the built-in kappa analysis after !End
> Define will do the trick. This will be a whole lot easier than for me to
> adapt my syntax.
>
> Brian
> ________________________________________
> From: Talma Claviger [
> talma.claviger@
> ]
> Sent: Sunday, January 21, 2018 2:55 PM
> To:
> SPSSX-L@.UGA
> ; Dates, Brian
> Subject: AW: interrater reliabillity - but for any pair of raters?
>
> Dear all,
>
>
> many thanks for your time + efforts! Because I am a bit slow I adapting
> the output management system (=SPSS novice, using it for the first time:)
> ) to David's syntax; I would also be interested in the matrix code Brian
> referred to?
>
>
> Best,
>
> Talma
> Gesendet von Outlook<
http://aka.ms/weboutlook>>
>
> ________________________________
> Von: SPSSX(r) Discussion <
> SPSSX-L@.UGA
> > im Auftrag von Dates, Brian <
> BDATES@
> >
> Gesendet: Sonntag, 21. Januar 2018 18:57
> An:
> SPSSX-L@.UGA
> Betreff: Re: interrater reliabillity - but for any pair of raters?
>
> This looks great. My syntax has all the matrix stuff to calculate Fleiss,
> so it wouldn't be as elegant as this, in which the OP just needs to figure
> out how to insert the OMS. I hope the OP from earlier this week is reading
> this thread, too. It's exactly what he wanted to do.
>
> My one concern is that the ordinal nature of the responses, or at least
> what looks ordinal, calls for the ICC rather than Fleiss. But that's not
> really our issue.
>
> Brian
> ________________________________________
> From: SPSSX(r) Discussion [
> SPSSX-L@.UGA
> ] on behalf of David Marso [
> Dmmugaspss@
> ]
> Sent: Saturday, January 20, 2018 9:04 PM
> To:
> SPSSX-L@.UGA
> Subject: Re: interrater reliabillity - but for any pair of raters?
>
> Macro with nested !DO and CROSSTABS in body. Use OMS to capture output.
> AGGREGATE to get average. Or maybe Brian will post a link to his MATRIX
> program.
>
> UNTESTED. I'm on my phone...
>
> DEFINE ! AllPairsKappa( raters !CMDEND)
> !LET !CPY = !raters
> !DO ! j1 !IN ( !raters)
> !LET ! CPY= !TAIL( !CPY)
> !DO !j2 !IN (!CPY)
> CROSSTABS TABLES !j1 BY !j2 /STATISTICS KAPPA.
> !DOEND
> !DOEND
> !ENDDEFINE.
>
> SET UP OMS..RTFM FOR THAT...
>
> !AllPairsKappa raters= j1 j2 j3 ..... j30.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> LISTSERV@.UGA
> (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> LISTSERV@.UGA
> (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> LISTSERV@.UGA
> (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
-----
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me.
---
"Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis."
Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?"
--
Sent from:
http://spssx-discussion.1045642.n5.nabble.com/=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD