Interrater reliability Multiple Items

[Pete-94]

To those who may be able to assist!

I am trying to establish the interrater reliability of a new evaluation form
that uses a total of 23, seven point likert-type scales. This system has
been used by three raters on seven participants. I have attempted to
establish the interrater reliability using the intraclass correlation
coefficient, however, I have been unable to ascertain whether this is the
correct procedure.

This method involves the three raters as the variables, and each of the
participants 23 ratings are stacked above each other as though each
represented a different person, (i.e., each row in SPSS data editor
represented a rating). For example, rows 1-23 contained the ratings for
participant one, rows 24-47 contained the ratings for participant two etc.
This gives me a total of 161 rows of data (7 participans each with 23 scores).

I have been unable to find any examples of multiple ratings for each
individual and how best to establish the reliability of the raters. Anyone
who can point out my failings or confirm that this may be acceptable, your
comments would be greatly appreciated.

Many thanks.

Pete

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[bdates]

Pete,

When you say "...23 likert-type scales..." are you referring to 23
items?  If so, are there subscales and/or a total summed score?  You can
use the ICC with Likert items; however, interrater reliability is
typically done with each subscale and then a summed scaled score, not
each item.

Brian

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Pete
Sent: Thursday, December 06, 2007 11:58 AM
To: [hidden email]
Subject: Interrater reliability Multiple Items

To those who may be able to assist!

I am trying to establish the interrater reliability of a new evaluation
form
that uses a total of 23, seven point likert-type scales. This system has
been used by three raters on seven participants. I have attempted to
establish the interrater reliability using the intraclass correlation
coefficient, however, I have been unable to ascertain whether this is
the
correct procedure.

This method involves the three raters as the variables, and each of the
participants 23 ratings are stacked above each other as though each
represented a different person, (i.e., each row in SPSS data editor
represented a rating). For example, rows 1-23 contained the ratings for
participant one, rows 24-47 contained the ratings for participant two
etc.
This gives me a total of 161 rows of data (7 participans each with 23
scores).

I have been unable to find any examples of multiple ratings for each
individual and how best to establish the reliability of the raters.
Anyone
who can point out my failings or confirm that this may be acceptable,
your
comments would be greatly appreciated.

Many thanks.

Pete

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Zdaniuk, Bozena-2]

Hello, how can I make logistic regression enter a predetermined subset of variables (by me) to enter in step 1 and then enter all of them in step 2 and see the difference in Nagelkerke R Square?
In other words, I want the syntax below for linear regression to adapt for logistic regression. Would someone be willing to help? Thanks a lot.
Bozena

REGRESSION VARIABLES=x1 x2 x3 x4 x5
  /MISSING LISTWISE
  /STATISTICS COEFF OUTS R ANOVA CHA
  /CRITERIA=PIN(.05) POUT(.10)
  /NOORIGIN
  /DEPENDENT x1
  / ENTER x2 x3 x4
  /enter.

Bozena Zdaniuk, Ph.D.
University of Pittsburgh

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Parry, James]

Hi Bozena-

In the dialog box, within the covariates box that says 'block 1 of 1', just put the group of variables that you want in the model first, then hit the 'next' button. Then do the same for the next group of predictors until you are finished.

-hth

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Zdaniuk, Bozena
Sent: Thursday, December 06, 2007 4:37 PM
To: [hidden email]
Subject: stepwise in logistic regression

Hello, how can I make logistic regression enter a predetermined subset of variables (by me) to enter in step 1 and then enter all of them in step 2 and see the difference in Nagelkerke R Square?
In other words, I want the syntax below for linear regression to adapt for logistic regression. Would someone be willing to help? Thanks a lot.
Bozena

REGRESSION VARIABLES=x1 x2 x3 x4 x5
  /MISSING LISTWISE
  /STATISTICS COEFF OUTS R ANOVA CHA
  /CRITERIA=PIN(.05) POUT(.10)
  /NOORIGIN
  /DEPENDENT x1
  / ENTER x2 x3 x4
  /enter.

Bozena Zdaniuk, Ph.D.
University of Pittsburgh

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD