Cohen's Kappa for multiple raters in versions later than SPSS 12

[Paul Mcgeoghan]

Hi,

I am trying to get Cohen's Kappa for multiple raters to work on SPSS 18
(and SPSS 16) using the syntax at
http://www.spsstools.net/Syntax/Matrix/CohensKappa.txt

It works OK in SPSS 12 but problems getting it working since SPSS changed
to multiple datasets.

Is the syntax supposed to work with current versions of SPSS?

Also, in SPSS 12, I get a negative value for Kappa -.0040 although 95% CI
is (-.0483 to .0403). What does a negative Cohen's Kappa mean?
There are 67 raters and 5 items.

The error I get in SPSS 18 begins with

Run MATRIX procedure:

>Error # 34 in column 22.  Text: ikstat1.sav
>PASW Statistics cannot access a file with the given file specification.
The
>file specification is either syntactically invalid, specifies an invalid
>drive, specifies a protected directory, specifies a protected file, or
>specifies a non-sharable file.
>Execution of this command stops.

>Error # 34 in column 75.  Text: k__tmp2.sav
>PASW Statistics cannot access a file with the given file specification.
The
>file specification is either syntactically invalid, specifies an invalid
>drive, specifies a protected directory, specifies a protected file, or
>specifies a non-sharable file.
>Execution of this command stops.

------ END MATRIX -----
Thanks,
Paul

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

Re interpretation of a negative kappa, see if the following demo helps.  In the first table, there is nearly perfect agreement between the two raters, and kappa = .96.  In the last, there is nearly perfect disagreement, and kappa = -.96.


data list list / table rater1 rater2 kount (4f5.0).
begin data
1 1 1 49
1 1 2 1
1 2 1 1
1 2 2 49
2 1 1 40
2 1 2 10
2 2 1 10
2 2 2 40
3 1 1 30
3 1 2 20
3 2 1 20
3 2 2 30
4 1 1 20
4 1 2 30
4 2 1 30
4 2 2 20
5 1 1 10
5 1 2 40
5 2 1 40
5 2 2 10
6 1 1 1
6 1 2 49
6 2 1 49
6 2 2 1
end data.

weight by kount.
crosstabs rater1 by rater2 by table / stat = kappa.

Paul Mcgeoghan wrote

Hi,

I am trying to get Cohen's Kappa for multiple raters to work on SPSS 18
(and SPSS 16) using the syntax at
http://www.spsstools.net/Syntax/Matrix/CohensKappa.txt

It works OK in SPSS 12 but problems getting it working since SPSS changed
to multiple datasets.

Is the syntax supposed to work with current versions of SPSS?

Also, in SPSS 12, I get a negative value for Kappa -.0040 although 95% CI
is (-.0483 to .0403). What does a negative Cohen's Kappa mean?
There are 67 raters and 5 items.

The error I get in SPSS 18 begins with

Run MATRIX procedure:

>Error # 34 in column 22.  Text: ikstat1.sav
>PASW Statistics cannot access a file with the given file specification.
The
>file specification is either syntactically invalid, specifies an invalid
>drive, specifies a protected directory, specifies a protected file, or
>specifies a non-sharable file.
>Execution of this command stops.

>Error # 34 in column 75.  Text: k__tmp2.sav
>PASW Statistics cannot access a file with the given file specification.
The
>file specification is either syntactically invalid, specifies an invalid
>drive, specifies a protected directory, specifies a protected file, or
>specifies a non-sharable file.
>Execution of this command stops.

------ END MATRIX -----
Thanks,
Paul

=====================
To manage your subscription to SPSSX-L, send a message to
[e-mail] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD