weighted kappa

[SoS Statistical Services]

Is it possible to do weighted kappa in SPSS? If yes, can anyone tell me how (I can do the normal kappa).

  Also is it possible to do the bhapkar test or stuart-maxwell test?

  Thank you for your help.

  Evie


Evie Gardner
SoS Statistical Services
Email: [hidden email]
Tel:   [phone] Mobile [phone]

---------------------------------
 Yahoo! for Good helps you make a difference

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Marta Garcia-Granero]

SoS Statistical Services escribió:
> Is it possible to do weighted kappa in SPSS? If yes, can anyone tell me how (I can do the normal kappa).
>

Are you talking about linear/quadratic weights or user defined? I have a
macro for linear&quadratic weighted kappa, plus some extra statistics
(maximum attainable kappa and kappa for individual catgories).
>   Also is it possible to do the bhapkar test or stuart-maxwell test?
>

I don't have anything written in SPSS, but there is a webpage with a
freeware program for that:

http://ourworld.compuserve.com/homepages/jsuebersax/mcnemar.htm#bhapkar

>   Thank you for your help.
>
>   Evie
>
>
> Evie Gardner
> SoS Statistical Services
> Email: [hidden email]
> Tel:   [phone] Mobile [phone]
>
> ---------------------------------
>  Yahoo! for Good helps you make a difference
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Marta Garcia-Granero]

SoS Statistical Services escribió:
>
> Thank you for your prompt reply. Linear/quadratic weights should be
> fine. If you are able to send me the macro that would be brilliant.
> Thanks also for the website for bhapkar.
Best regards,
Marta

 ******************************
/*          KAPPAPLUS         */
 ******************************
 * (C) M García-Granero, 2007 *
 *  [hidden email]  *
 ******************************

**********************************************************************
* For good references (both used for this MACRO), see:               *
**********************************************************************
* 1) Julius Sim, Chris C Wright "The Kappa Statistic in Reliability  *
*    Studies: Use, Interpretation, and Sample Size Requirements"     *
*    Physical Therapy 2005;85:257-268.                               *
* 2) Joseph J Fleiss "Statistical Methods for Rates and Proportions" *
*    2nd Ed. Wiley Interscience. (Chapter 13)                        *
**********************************************************************
* Accuracy of the MACRO checked with WinPepi (www.brixtonhealth.com) *
**********************************************************************

* MACRO DEFINITION *.
DEFINE KAPPAPLUS(!POS=!TOKENS(3)).
* Taking advantage of CROSSTABS & OMS to simplify tabulation *.
OMS
 /SELECT TABLES
 /IF COMMANDS = 'Crosstabs'
     SUBTYPES = 'Case Processing Summary'
 /DESTINATION VIEWER = NO.
OMS
 /SELECT TABLES
 /IF COMMANDS = 'Crosstabs'
     SUBTYPES = 'Crosstabulation'
 /DESTINATION FORMAT = SAV
             OUTFILE = 'C:\Temp\CrosstabData.sav'.
CROSSTABS /TABLES=!1.
OMSEND.
* MATRIX code *.
MATRIX.
PRINT /TITLE='      ******* AGREEMENT: KAPPA  **********'.
* Read OMS data *.
GET TabData /VAR=ALL
            /FILE='C:\Temp\CrosstabData.sav'
            /NAMES=Vnames
            /MISSING=ACCEPT /SYSMIS=0.
* Extract relevant info *.
COMPUTE CrossTab=TabData(1:(NROW(TabData)-1),7:(NCOL(Tabdata)-1)).
COMPUTE CatNames={Vnames(7:(NCOL(Tabdata)-1)),'Total'}.
* Report input data *.
PRINT {CrossTab,      RSUM(CrossTab);
       CSUM(CrossTab),MSUM(CrossTab)}
 /FORMAT='F8.0'
 /CNAME=CatNames
 /RNAME=CatNames
 /TITLE='Input data'.
* Common data calculation *.
COMPUTE n=MSUM(CrossTab).
COMPUTE Ncat=NROW(CrossTab).
COMPUTE Pij=CrossTab/n.
COMPUTE Pidot=RSUM(Pij).
COMPUTE Pdotj=CSUM(Pij).
COMPUTE AllData=MAKE(NCat+1,8,0).
* Get rid of useless data *.
RELEASE TabData,Vnames,CrossTab.
********* UNWEIGHTED KAPPA ***********.
PRINT /TITLE=' ** UNWEIGHTED KAPPA **'.
COMPUTE Wij=IDENT(Ncat).
COMPUTE Pew=T(Pidot)*Wij*T(Pdotj).
COMPUTE Pow=MSUM(Pij&*Wij).
COMPUTE K=(Pow-Pew)/(1-Pew).
* SE(0) & SE(k) *.
COMPUTE Widot=Wij*T(Pdotj)).
COMPUTE Wdotj=T(Pidot)*Wij.
COMPUTE cterm=Widot*MAKE(1,Ncat,1)+MAKE(Ncat,1,1)*Wdotj.
COMPUTE
SE0K=SQRT(T(Pidot)*((Wij-cterm)&**2)*T(Pdotj)-Pew&**2)/((1-Pew)*SQRT(n)).
COMPUTE
SE1K=SQRT(MSUM(Pij&*(Wij-cterm&*(1-K))&**2)-(K-Pew*(1-K))**2)/((1-Pew)*SQRT(n)).
* Tests & CI *.
COMPUTE ZValue=K/SE0K.
COMPUTE ZSig=1-CDFNORM(ABS(ZValue)).
COMPUTE LowCL=K-1.96*SE1K.
COMPUTE UppCL=K+1.96*SE1K.
COMPUTE ZValue4=(K-0.4)/SE1K.
COMPUTE ZSig4=1-CDFNORM(ZValue4).
COMPUTE ZValue6=(K-0.6)/SE1K.
COMPUTE ZSig6=1-CDFNORM(ZValue6).
* Store all infor for condensed report at end *.
COMPUTE Alldata((Ncat+1),:)={K,SE0K,ZSig,SE1K,LowCL,UppCL,ZSig4,ZSig6}.
* Max. attainable Kappa *.
COMPUTE PowExp=0.
LOOP i=1 TO Ncat.
- DO IF Pidot(i) LT Pdotj(i).
-  COMPUTE PowExp=PowExp+Pidot(i).
- ELSE.
-  COMPUTE PowExp=PowExp+Pdotj(i).
- END IF.
END LOOP.
COMPUTE MaxK=(PowExp-Pew)/(1-Pew).
* Reports *.
PRINT {K,SE0K,ZValue,ZSig}
 /FORMAT='F8.3'
 /CLABEL='Kappa','SE','Z value','P-value'
 /TITLE='Unweighted Overall Kappa, SE and significance'.
PRINT {K,SE1K,LowCL,UppCL}
 /FORMAT='F8.3'
 /CLABEL='Kappa','SE','Lower95%','Upper95%'
 /TITLE='Unweighted Overall Kappa & 95%CI'.
PRINT {ZValue4,ZSig4;Zvalue6,Zsig6}
 /FORMAT='F8.3'
 /CLABEL='Z value','P-value'
 /RLABEL='Ko=0.4','Ko=0.6'
 /TITLE='Significantly above 0.4 & 0.6?'.
PRINT MaxK
 /FORMAT='F8.3'
 /RLABEL='Max.Kap.'
 /TITLE='Maximum Atttainable Kappa (ceiling given observed marginal
totals)'.
********* WEIGHTED KAPPAS ***********.
PRINT /TITLE=' ** WEIGHTED KAPPAS **'.
* Linear weights *.
COMPUTE Wij=MAKE(Ncat,Ncat,0).
LOOP i=1 TO Ncat.
- LOOP j=1 TO Ncat.
-  COMPUTE Wij(i,j)=1-(ABS(i-j)/(Ncat-1)).
- END LOOP.
END LOOP.
PRINT Wij
 /FORMAT='F8.3'
 /CNAME=CatNames
 /RNAME=CatNames
 /TITLE=' --- Linear weights ---'.
* Linear Weighted Kappa *.
COMPUTE Pew=T(Pidot)*Wij*T(Pdotj).
COMPUTE Pow=MSUM(Pij&*Wij).
COMPUTE WK=(Pow-Pew)/(1-Pew).
* SE(0) & SE(k) *.
COMPUTE Widot=Wij*T(Pdotj)).
COMPUTE Wdotj=T(Pidot)*Wij.
COMPUTE cterm=Widot*MAKE(1,Ncat,1)+MAKE(Ncat,1,1)*Wdotj.
COMPUTE
SE0Kw=SQRT(T(Pidot)*((Wij-cterm)&**2)*T(Pdotj)-Pew&**2)/((1-Pew)*SQRT(n)).
COMPUTE
SE1Kw=SQRT(MSUM(Pij&*(Wij-cterm&*(1-WK))&**2)-(WK-Pew*(1-WK))**2)/((1-Pew)*SQRT(n)).
* Tests & CI *.
COMPUTE ZValue=WK/SE0Kw.
COMPUTE ZSig=1-CDFNORM(ABS(ZValue)).
COMPUTE LowCLw=WK-1.96*SE1Kw.
COMPUTE UppCLw=WK+1.96*SE1Kw.
COMPUTE ZValue4=(WK-0.4)/SE1Kw.
COMPUTE ZSig4=1-CDFNORM(ZValue4).
COMPUTE ZValue6=(WK-0.6)/SE1Kw.
COMPUTE ZSig6=1-CDFNORM(ZValue6).
* Reports *.
PRINT {WK,SE0Kw,ZValue,ZSig}
 /FORMAT='F8.3'
 /CLABEL='Kappa','SE','Z value','P-value'
 /TITLE='Linear Weighted Kappa, SE and significance'.
PRINT {WK,SE1Kw,LowCLw,UppCLw}
 /FORMAT='F8.3'
 /CLABEL='Kappa','SE','Lower95%','Upper95%'
 /TITLE='Linear Weighted Kappa & 95%CI'.
PRINT {ZValue4,ZSig4;Zvalue6,Zsig6}
 /FORMAT='F8.3'
 /CLABEL='Z value','P-value'
 /RLABEL='Ko=0.4','Ko=0.6'
 /TITLE='Significantly above 0.4 & 0.6?'.
* Quadratic weights *.
COMPUTE Wij=MAKE(Ncat,Ncat,0).
LOOP i=1 TO Ncat.
- LOOP j=1 TO Ncat.
-  COMPUTE Wij(i,j)=1-((i-j)/(Ncat-1))&**2.
- END LOOP.
END LOOP.
PRINT Wij
 /FORMAT='F8.3'
 /CNAME=CatNames
 /RNAME=CatNames
 /TITLE='--- Quadratic weights ---'.
* Quadratic Weighted Kappa *.
COMPUTE Pew=T(Pidot)*Wij*T(Pdotj).
COMPUTE Pow=MSUM(Pij&*Wij).
COMPUTE WK=(Pow-Pew)/(1-Pew).
* SE(0) & SE(k) *.
COMPUTE Widot=Wij*T(Pdotj)).
COMPUTE Wdotj=T(Pidot)*Wij.
COMPUTE cterm=Widot*MAKE(1,Ncat,1)+MAKE(Ncat,1,1)*Wdotj.
COMPUTE
SE0Kw=SQRT(T(Pidot)*((Wij-cterm)&**2)*T(Pdotj)-Pew&**2)/((1-Pew)*SQRT(n)).
COMPUTE
SE1Kw=SQRT(MSUM(Pij&*(Wij-cterm&*(1-WK))&**2)-(WK-Pew*(1-WK))**2)/((1-Pew)*SQRT(n)).
* Tests & CI *.
COMPUTE ZValue=WK/SE0Kw.
COMPUTE ZSig=1-CDFNORM(ABS(ZValue)).
COMPUTE LowCLw=WK-1.96*SE1Kw.
COMPUTE UppCLw=WK+1.96*SE1Kw.
COMPUTE ZValue4=(WK-0.4)/SE1Kw.
COMPUTE ZSig4=1-CDFNORM(ZValue4).
COMPUTE ZValue6=(WK-0.6)/SE1Kw.
COMPUTE ZSig6=1-CDFNORM(ZValue6).
* Reports *.
PRINT {WK,SE0Kw,ZValue,ZSig}
 /FORMAT='F8.3'
 /CLABEL='Kappa','SE','Z value','P-value'
 /TITLE='Quadratic Weighted Kappa, SE and significance'.
PRINT {WK,SE1Kw,LowCLw,UppCLw}
 /FORMAT='F8.3'
 /CLABEL='Kappa','SE','Lower95%','Upper95%'
 /TITLE='Quadratic Weighted Kappa & 95%CI'.
PRINT {ZValue4,ZSig4;Zvalue6,Zsig6}
 /FORMAT='F8.3'
 /CLABEL='Z value','P-value'
 /RLABEL='Ko=0.4','Ko=0.6'
 /TITLE='Significantly above 0.4 & 0.6?'.
********** KAPPA FOR INDIVIDUAL CATEGORIES **********.
PRINT /TITLE=' ** KAPPA FOR INDIVIDUAL CATEGORIES **'.
COMPUTE Wij=IDENT(2).
LOOP NKappa=1 TO Ncat.
- COMPUTE Copied=Pij.
- COMPUTE ScrapCol=Copied(:,1).
- COMPUTE Copied(:,1)=Copied(:,Nkappa).
- COMPUTE Copied(:,NKappa)=ScrapCol.
- COMPUTE ScrapRow=Copied(1,:).
- COMPUTE Copied(1,:)=Copied(Nkappa,:).
- COMPUTE Copied(NKappa,:)=ScrapRow(1,:).
- COMPUTE SPij=MAKE(2,2,0).
- COMPUTE SPij(1,1)=Copied(1,1).
- COMPUTE SPij(1,2)=MSUM(Copied(1,2:Ncat)).
- COMPUTE SPij(2,1)=MSUM(Copied(2:Ncat,1)).
- COMPUTE SPij(2,2)=MSUM(Copied(2:Ncat,2:Ncat)).
- COMPUTE SPidot=RSUM(SPij).
- COMPUTE SPdotj=CSUM(SPij).
- COMPUTE Pew=T(SPidot)*Wij*T(SPdotj).
- COMPUTE Pow=MSUM(SPij&*Wij).
- COMPUTE K=(Pow-Pew)/(1-Pew).
- COMPUTE AllData(Nkappa,1)=K.
* SE(0) & SE(k) *.
- COMPUTE Widot=Wij*T(SPdotj)).
- COMPUTE Wdotj=T(SPidot)*Wij.
- COMPUTE cterm=Widot*MAKE(1,2,1)+MAKE(2,1,1)*Wdotj.
- COMPUTE
SE0K=SQRT(T(SPidot)*((Wij-cterm)&**2)*T(SPdotj)-Pew&**2)/((1-Pew)*SQRT(n)).
- COMPUTE
SE1K=SQRT(MSUM(SPij&*(Wij-cterm&*(1-K))&**2)-(K-Pew*(1-K))**2)/((1-Pew)*SQRT(n)).
- COMPUTE AllData(Nkappa,2)=SE0K.
- COMPUTE AllData(Nkappa,4)=SE1K.
* Tests & CI *.
- COMPUTE ZValue=K/SE0K.
- COMPUTE ZSig=1-CDFNORM(ABS(ZValue)).
- COMPUTE AllData(Nkappa,3)=ZSig.
- COMPUTE LowCL=K-1.96*SE1K.
- COMPUTE UppCL=K+1.96*SE1K.
- COMPUTE AllData(Nkappa,5)=LowCL.
- COMPUTE AllData(Nkappa,6)=UppCL.
- COMPUTE ZValue4=(K-0.4)/SE1K.
- COMPUTE ZSig4=1-CDFNORM(ZValue4).
- COMPUTE ZValue6=(K-0.6)/SE1K.
- COMPUTE ZSig6=1-CDFNORM(ZValue6).
- COMPUTE AllData(Nkappa,7)=ZSig4.
- COMPUTE AllData(Nkappa,8)=ZSig6.
END LOOP.
* Condensed Report *.
PRINT AllData
 /FORMAT='F5.3'
 /RNAMES=CatNames
 /CLABELS='Kappa','SE[H0]','P(K=0)','SE[Ha]','Low95%','Upp95%','P(0.4)','P(0.6)'
 /TITLE='Summary report (individual categories and Overall unweighted
Kappa) '.
END MATRIX.
!ENDDEFINE.

* Sample dataset, from table 2 (test-retest agreement) of Sim & Wright
paper *.
DATA LIST FREE/Test1 Test2 n (3 F8).
BEGIN DATA
1 1 15 1 2  3 1 3  1 1 4  1
2 1  4 2 2 18 2 3  3 2 4  2
3 1  4 3 2  5 3 3 16 3 4  4
4 1  1 4 2  2 4 3  4 4 4 17
END DATA.
VAR LEVEL Test1 Test2 (ORDINAL).
VAL LABEL Test1 Test2
 1'None'
 2'Mild'
 3'Moderate'
 4'Severe'.
VAR LABEL Test1 'Pain at Shoulder Joint (test 1)'/
          Test2 'Pain at Shoulder Joint (test 2)'.
WEIGHT BY n.

* MACRO call *.
KAPPAPLUS Test1 BY Test2.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[SoS Statistical Services]

Marta

  Thank you so much, you've been very helpful.

  Evie

Marta García-Granero <[hidden email]> wrote:
  SoS Statistical Services escribió:
>
> Thank you for your prompt reply. Linear/quadratic weights should be
> fine. If you are able to send me the macro that would be brilliant.
> Thanks also for the website for bhapkar.
Best regards,
Marta

******************************
/* KAPPAPLUS */
******************************
* (C) M García-Granero, 2007 *
* [hidden email] *
******************************

**********************************************************************
* For good references (both used for this MACRO), see: *
**********************************************************************
* 1) Julius Sim, Chris C Wright "The Kappa Statistic in Reliability *
* Studies: Use, Interpretation, and Sample Size Requirements" *
* Physical Therapy 2005;85:257-268. *
* 2) Joseph J Fleiss "Statistical Methods for Rates and Proportions" *
* 2nd Ed. Wiley Interscience. (Chapter 13) *
**********************************************************************
* Accuracy of the MACRO checked with WinPepi (www.brixtonhealth.com) *
**********************************************************************

* MACRO DEFINITION *.
DEFINE KAPPAPLUS(!POS=!TOKENS(3)).
* Taking advantage of CROSSTABS & OMS to simplify tabulation *.
OMS
/SELECT TABLES
/IF COMMANDS = 'Crosstabs'
SUBTYPES = 'Case Processing Summary'
/DESTINATION VIEWER = NO.
OMS
/SELECT TABLES
/IF COMMANDS = 'Crosstabs'
SUBTYPES = 'Crosstabulation'
/DESTINATION FORMAT = SAV
OUTFILE = 'C:\Temp\CrosstabData.sav'.
CROSSTABS /TABLES=!1.
OMSEND.
* MATRIX code *.
MATRIX.
PRINT /TITLE=' ******* AGREEMENT: KAPPA **********'.
* Read OMS data *.
GET TabData /VAR=ALL
/FILE='C:\Temp\CrosstabData.sav'
/NAMES=Vnames
/MISSING=ACCEPT /SYSMIS=0.
* Extract relevant info *.
COMPUTE CrossTab=TabData(1:(NROW(TabData)-1),7:(NCOL(Tabdata)-1)).
COMPUTE CatNames={Vnames(7:(NCOL(Tabdata)-1)),'Total'}.
* Report input data *.
PRINT {CrossTab, RSUM(CrossTab);
CSUM(CrossTab),MSUM(CrossTab)}
/FORMAT='F8.0'
/CNAME=CatNames
/RNAME=CatNames
/TITLE='Input data'.
* Common data calculation *.
COMPUTE n=MSUM(CrossTab).
COMPUTE Ncat=NROW(CrossTab).
COMPUTE Pij=CrossTab/n.
COMPUTE Pidot=RSUM(Pij).
COMPUTE Pdotj=CSUM(Pij).
COMPUTE AllData=MAKE(NCat+1,8,0).
* Get rid of useless data *.
RELEASE TabData,Vnames,CrossTab.
********* UNWEIGHTED KAPPA ***********.
PRINT /TITLE=' ** UNWEIGHTED KAPPA **'.
COMPUTE Wij=IDENT(Ncat).
COMPUTE Pew=T(Pidot)*Wij*T(Pdotj).
COMPUTE Pow=MSUM(Pij&*Wij).
COMPUTE K=(Pow-Pew)/(1-Pew).
* SE(0) & SE(k) *.
COMPUTE Widot=Wij*T(Pdotj)).
COMPUTE Wdotj=T(Pidot)*Wij.
COMPUTE cterm=Widot*MAKE(1,Ncat,1)+MAKE(Ncat,1,1)*Wdotj.
COMPUTE
SE0K=SQRT(T(Pidot)*((Wij-cterm)&**2)*T(Pdotj)-Pew&**2)/((1-Pew)*SQRT(n)).
COMPUTE
SE1K=SQRT(MSUM(Pij&*(Wij-cterm&*(1-K))&**2)-(K-Pew*(1-K))**2)/((1-Pew)*SQRT(n)).
* Tests & CI *.
COMPUTE ZValue=K/SE0K.
COMPUTE ZSig=1-CDFNORM(ABS(ZValue)).
COMPUTE LowCL=K-1.96*SE1K.
COMPUTE UppCL=K+1.96*SE1K.
COMPUTE ZValue4=(K-0.4)/SE1K.
COMPUTE ZSig4=1-CDFNORM(ZValue4).
COMPUTE ZValue6=(K-0.6)/SE1K.
COMPUTE ZSig6=1-CDFNORM(ZValue6).
* Store all infor for condensed report at end *.
COMPUTE Alldata((Ncat+1),:)={K,SE0K,ZSig,SE1K,LowCL,UppCL,ZSig4,ZSig6}.
* Max. attainable Kappa *.
COMPUTE PowExp=0.
LOOP i=1 TO Ncat.
- DO IF Pidot(i) LT Pdotj(i).
- COMPUTE PowExp=PowExp+Pidot(i).
- ELSE.
- COMPUTE PowExp=PowExp+Pdotj(i).
- END IF.
END LOOP.
COMPUTE MaxK=(PowExp-Pew)/(1-Pew).
* Reports *.
PRINT {K,SE0K,ZValue,ZSig}
/FORMAT='F8.3'
/CLABEL='Kappa','SE','Z value','P-value'
/TITLE='Unweighted Overall Kappa, SE and significance'.
PRINT {K,SE1K,LowCL,UppCL}
/FORMAT='F8.3'
/CLABEL='Kappa','SE','Lower95%','Upper95%'
/TITLE='Unweighted Overall Kappa & 95%CI'.
PRINT {ZValue4,ZSig4;Zvalue6,Zsig6}
/FORMAT='F8.3'
/CLABEL='Z value','P-value'
/RLABEL='Ko=0.4','Ko=0.6'
/TITLE='Significantly above 0.4 & 0.6?'.
PRINT MaxK
/FORMAT='F8.3'
/RLABEL='Max.Kap.'
/TITLE='Maximum Atttainable Kappa (ceiling given observed marginal
totals)'.
********* WEIGHTED KAPPAS ***********.
PRINT /TITLE=' ** WEIGHTED KAPPAS **'.
* Linear weights *.
COMPUTE Wij=MAKE(Ncat,Ncat,0).
LOOP i=1 TO Ncat.
- LOOP j=1 TO Ncat.
- COMPUTE Wij(i,j)=1-(ABS(i-j)/(Ncat-1)).
- END LOOP.
END LOOP.
PRINT Wij
/FORMAT='F8.3'
/CNAME=CatNames
/RNAME=CatNames
/TITLE=' --- Linear weights ---'.
* Linear Weighted Kappa *.
COMPUTE Pew=T(Pidot)*Wij*T(Pdotj).
COMPUTE Pow=MSUM(Pij&*Wij).
COMPUTE WK=(Pow-Pew)/(1-Pew).
* SE(0) & SE(k) *.
COMPUTE Widot=Wij*T(Pdotj)).
COMPUTE Wdotj=T(Pidot)*Wij.
COMPUTE cterm=Widot*MAKE(1,Ncat,1)+MAKE(Ncat,1,1)*Wdotj.
COMPUTE
SE0Kw=SQRT(T(Pidot)*((Wij-cterm)&**2)*T(Pdotj)-Pew&**2)/((1-Pew)*SQRT(n)).
COMPUTE
SE1Kw=SQRT(MSUM(Pij&*(Wij-cterm&*(1-WK))&**2)-(WK-Pew*(1-WK))**2)/((1-Pew)*SQRT(n)).
* Tests & CI *.
COMPUTE ZValue=WK/SE0Kw.
COMPUTE ZSig=1-CDFNORM(ABS(ZValue)).
COMPUTE LowCLw=WK-1.96*SE1Kw.
COMPUTE UppCLw=WK+1.96*SE1Kw.
COMPUTE ZValue4=(WK-0.4)/SE1Kw.
COMPUTE ZSig4=1-CDFNORM(ZValue4).
COMPUTE ZValue6=(WK-0.6)/SE1Kw.
COMPUTE ZSig6=1-CDFNORM(ZValue6).
* Reports *.
PRINT {WK,SE0Kw,ZValue,ZSig}
/FORMAT='F8.3'
/CLABEL='Kappa','SE','Z value','P-value'
/TITLE='Linear Weighted Kappa, SE and significance'.
PRINT {WK,SE1Kw,LowCLw,UppCLw}
/FORMAT='F8.3'
/CLABEL='Kappa','SE','Lower95%','Upper95%'
/TITLE='Linear Weighted Kappa & 95%CI'.
PRINT {ZValue4,ZSig4;Zvalue6,Zsig6}
/FORMAT='F8.3'
/CLABEL='Z value','P-value'
/RLABEL='Ko=0.4','Ko=0.6'
/TITLE='Significantly above 0.4 & 0.6?'.
* Quadratic weights *.
COMPUTE Wij=MAKE(Ncat,Ncat,0).
LOOP i=1 TO Ncat.
- LOOP j=1 TO Ncat.
- COMPUTE Wij(i,j)=1-((i-j)/(Ncat-1))&**2.
- END LOOP.
END LOOP.
PRINT Wij
/FORMAT='F8.3'
/CNAME=CatNames
/RNAME=CatNames
/TITLE='--- Quadratic weights ---'.
* Quadratic Weighted Kappa *.
COMPUTE Pew=T(Pidot)*Wij*T(Pdotj).
COMPUTE Pow=MSUM(Pij&*Wij).
COMPUTE WK=(Pow-Pew)/(1-Pew).
* SE(0) & SE(k) *.
COMPUTE Widot=Wij*T(Pdotj)).
COMPUTE Wdotj=T(Pidot)*Wij.
COMPUTE cterm=Widot*MAKE(1,Ncat,1)+MAKE(Ncat,1,1)*Wdotj.
COMPUTE
SE0Kw=SQRT(T(Pidot)*((Wij-cterm)&**2)*T(Pdotj)-Pew&**2)/((1-Pew)*SQRT(n)).
COMPUTE
SE1Kw=SQRT(MSUM(Pij&*(Wij-cterm&*(1-WK))&**2)-(WK-Pew*(1-WK))**2)/((1-Pew)*SQRT(n)).
* Tests & CI *.
COMPUTE ZValue=WK/SE0Kw.
COMPUTE ZSig=1-CDFNORM(ABS(ZValue)).
COMPUTE LowCLw=WK-1.96*SE1Kw.
COMPUTE UppCLw=WK+1.96*SE1Kw.
COMPUTE ZValue4=(WK-0.4)/SE1Kw.
COMPUTE ZSig4=1-CDFNORM(ZValue4).
COMPUTE ZValue6=(WK-0.6)/SE1Kw.
COMPUTE ZSig6=1-CDFNORM(ZValue6).
* Reports *.
PRINT {WK,SE0Kw,ZValue,ZSig}
/FORMAT='F8.3'
/CLABEL='Kappa','SE','Z value','P-value'
/TITLE='Quadratic Weighted Kappa, SE and significance'.
PRINT {WK,SE1Kw,LowCLw,UppCLw}
/FORMAT='F8.3'
/CLABEL='Kappa','SE','Lower95%','Upper95%'
/TITLE='Quadratic Weighted Kappa & 95%CI'.
PRINT {ZValue4,ZSig4;Zvalue6,Zsig6}
/FORMAT='F8.3'
/CLABEL='Z value','P-value'
/RLABEL='Ko=0.4','Ko=0.6'
/TITLE='Significantly above 0.4 & 0.6?'.
********** KAPPA FOR INDIVIDUAL CATEGORIES **********.
PRINT /TITLE=' ** KAPPA FOR INDIVIDUAL CATEGORIES **'.
COMPUTE Wij=IDENT(2).
LOOP NKappa=1 TO Ncat.
- COMPUTE Copied=Pij.
- COMPUTE ScrapCol=Copied(:,1).
- COMPUTE Copied(:,1)=Copied(:,Nkappa).
- COMPUTE Copied(:,NKappa)=ScrapCol.
- COMPUTE ScrapRow=Copied(1,:).
- COMPUTE Copied(1,:)=Copied(Nkappa,:).
- COMPUTE Copied(NKappa,:)=ScrapRow(1,:).
- COMPUTE SPij=MAKE(2,2,0).
- COMPUTE SPij(1,1)=Copied(1,1).
- COMPUTE SPij(1,2)=MSUM(Copied(1,2:Ncat)).
- COMPUTE SPij(2,1)=MSUM(Copied(2:Ncat,1)).
- COMPUTE SPij(2,2)=MSUM(Copied(2:Ncat,2:Ncat)).
- COMPUTE SPidot=RSUM(SPij).
- COMPUTE SPdotj=CSUM(SPij).
- COMPUTE Pew=T(SPidot)*Wij*T(SPdotj).
- COMPUTE Pow=MSUM(SPij&*Wij).
- COMPUTE K=(Pow-Pew)/(1-Pew).
- COMPUTE AllData(Nkappa,1)=K.
* SE(0) & SE(k) *.
- COMPUTE Widot=Wij*T(SPdotj)).
- COMPUTE Wdotj=T(SPidot)*Wij.
- COMPUTE cterm=Widot*MAKE(1,2,1)+MAKE(2,1,1)*Wdotj.
- COMPUTE
SE0K=SQRT(T(SPidot)*((Wij-cterm)&**2)*T(SPdotj)-Pew&**2)/((1-Pew)*SQRT(n)).
- COMPUTE
SE1K=SQRT(MSUM(SPij&*(Wij-cterm&*(1-K))&**2)-(K-Pew*(1-K))**2)/((1-Pew)*SQRT(n)).
- COMPUTE AllData(Nkappa,2)=SE0K.
- COMPUTE AllData(Nkappa,4)=SE1K.
* Tests & CI *.
- COMPUTE ZValue=K/SE0K.
- COMPUTE ZSig=1-CDFNORM(ABS(ZValue)).
- COMPUTE AllData(Nkappa,3)=ZSig.
- COMPUTE LowCL=K-1.96*SE1K.
- COMPUTE UppCL=K+1.96*SE1K.
- COMPUTE AllData(Nkappa,5)=LowCL.
- COMPUTE AllData(Nkappa,6)=UppCL.
- COMPUTE ZValue4=(K-0.4)/SE1K.
- COMPUTE ZSig4=1-CDFNORM(ZValue4).
- COMPUTE ZValue6=(K-0.6)/SE1K.
- COMPUTE ZSig6=1-CDFNORM(ZValue6).
- COMPUTE AllData(Nkappa,7)=ZSig4.
- COMPUTE AllData(Nkappa,8)=ZSig6.
END LOOP.
* Condensed Report *.
PRINT AllData
/FORMAT='F5.3'
/RNAMES=CatNames
/CLABELS='Kappa','SE[H0]','P(K=0)','SE[Ha]','Low95%','Upp95%','P(0.4)','P(0.6)'
/TITLE='Summary report (individual categories and Overall unweighted
Kappa) '.
END MATRIX.
!ENDDEFINE.

* Sample dataset, from table 2 (test-retest agreement) of Sim & Wright
paper *.
DATA LIST FREE/Test1 Test2 n (3 F8).
BEGIN DATA
1 1 15 1 2 3 1 3 1 1 4 1
2 1 4 2 2 18 2 3 3 2 4 2
3 1 4 3 2 5 3 3 16 3 4 4
4 1 1 4 2 2 4 3 4 4 4 17
END DATA.
VAR LEVEL Test1 Test2 (ORDINAL).
VAL LABEL Test1 Test2
1'None'
2'Mild'
3'Moderate'
4'Severe'.
VAR LABEL Test1 'Pain at Shoulder Joint (test 1)'/
Test2 'Pain at Shoulder Joint (test 2)'.
WEIGHT BY n.

* MACRO call *.
KAPPAPLUS Test1 BY Test2.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD



Evie Gardner
SoS Statistical Services
Email: [hidden email]
Tel:   [phone] Mobile [phone]

---------------------------------
 Yahoo! for Good helps you make a difference

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD