SPSSX Discussion

judges discrepancy from median

Classic

List

Threaded

12 messages Options

Hall, Mark

judges discrepancy from median

Hello

I am looking for some help. I am validating a survey instrument using a panel of expert judges. I have a number of items and ratings from 1-5 from each judge for each item.

I am looking for an SPSS command/syntax that will allow me to calculate the sum of the Judges discrepancy from the Median (JDM) for each judge for all items.

Thanks very much

Mark

Bruce Weaver

Re: judges discrepancy from median

Administrator

Hall, Mark wrote

Hello

I am looking for some help. I am validating a survey instrument using a
panel of expert judges. I have a number of items and ratings from 1-5
from each judge for each item.

I am looking for an SPSS command/syntax that will allow me to calculate
the sum of the Judges discrepancy from the Median (JDM) for each judge
for all items.

Thanks very much

Mark

Mark, it's not clear to me which median you want. Do you want one grand median (across all items & judges)? Or do you want one median per item (across judges)? Perhaps you could provide a small sample data set that illustrates exactly what you want.

Thanks,
Bruce

--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

PLEASE NOTE THE FOLLOWING:
1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above.
2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/).

Marta Garcia-Granero

Re: judges discrepancy from median

In reply to this post by Hall, Mark

Hi Mark:

Is the formula you want to use this one?

http://library2.usask.ca/theses/available/etd-04022009-131700/unrestricted/charles_duncan_phd_dissertation_march_2009_revised.pdf

(page 88 of PDF file)

It should be easy to compute something using MATRIX, but I'll wait until you confirm if that's indeed the formula you want. Also, as Bruce suggested, it would be a good idea if you sent some real data (just a tiny sample, 5 cases or so).

Best regards,
Marta GG

El 28/09/2010 18:18, Hall, Mark escribió:

Hello

I am looking for some help. I am validating a survey instrument using a panel of expert judges. I have a number of items and ratings from 1-5 from each judge for each item.

I am looking for an SPSS command/syntax that will allow me to calculate the sum of the Judges discrepancy from the Median (JDM) for each judge for all items.

Thanks very much

Mark

--
For miscellaneous SPSS related statistical stuff, visit:
http://gjyp.nl/marta/

Art Kendall

Re: judges discrepancy from median

In reply to this post by Hall, Mark

Is there a reason you want to use this approach rather than more traditional estimates of inter-coder, inter-judge, inter-rater reliability?
What questions are you trying to answer with this approach?

Art Kendall
Social Research Consultants

On 9/28/2010 12:18 PM, Hall, Mark wrote:

Hello

I am looking for some help. I am validating a survey instrument using a panel of expert judges. I have a number of items and ratings from 1-5 from each judge for each item.

I am looking for an SPSS command/syntax that will allow me to calculate the sum of the Judges discrepancy from the Median (JDM) for each judge for all items.

Thanks very much

Mark

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Art Kendall
Social Research Consultants

James Whanger

Re: judges discrepancy from median

In reply to this post by Hall, Mark

Mark,

Manipulating data based on the median is a heck of lot easier using R
because it has a built in median function unlike SPSS. So, if you have R
you might want to take that route.

Best,

Jim

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

zstatman

Re: judges discrepancy from median

In reply to this post by Hall, Mark

So Jim, perhaps you might create a reply with generated data and the R
routine to do this.

W

On 9/29/2010 9:25:51 AM, [hidden email] wrote:

> Mark,
>
> Manipulating data based on the median is a heck of lot easier using R
> because it has a built in median function unlike SPSS. So, if you have R
> you might want to take that route.
>
> Best,
>
> Jim
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Will
Statistical Services

============
info.statman@earthlink.net
http://home.earthlink.net/~z_statman/
============

Art Kendall

Re: judges discrepancy from median

In reply to this post by James Whanger

SPSS has a within case (row) median function.
compute xmedian = median(x1 to x 30).

Across case medians are also available in MEANS, FREQUENCIES, RATIO
(also its confidence interval), SUMMARIZE, EXAMINE, ETC.

Art Kendall
Social Research Consultants

On 9/29/2010 9:25 AM, James Whanger wrote:

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Art Kendall
Social Research Consultants

Marta Garcia-Granero

Re: judges discrepancy from median

In reply to this post by Hall, Mark

28/09/2010 18:18, Hall, Mark wrote:

I am looking for some help. I am validating a survey instrument using a panel of expert judges. I have a number of items and ratings from 1-5 from each judge for each item.
I am looking for an SPSS command/syntax that will allow me to calculate the sum of the Judges discrepancy from the Median (JDM) for each judge for all items.

Hi Mark:

Here it is. An old fashioned (obsolete?) solution that does not use the programmability extension, nor Python, nor R..., just a simple macro and matrix:

* Sample dataset (provided by Mark) *.
DATA LIST LIST/I2a I2b I2c I2d I2e I2f I2g I2h I2i I2j I2k (11 F8.0).
BEGIN DATA
5 5 5 5 5 5 4 4 5 5 4
5 5 5 5 5 5 5 5 5 5 5
3 5 5 5 5 5 5 3 5 4 5
5 5 1 4 5 5 5 5 5 5 5
5 5 5 5 5 5 5 5 5 5 5
3 5 3 5 4 3 5 3 5 5 4
5 5 5 5 5 5 5 5 5 5 5
4 4 5 3 5 5 3 4 5 2 5
5 5 5 5 5 5 5 1 5 5 5
5 4 4 5 4 3 5 5 5 5 5
5 5 5 5 5 5 5 5 5 5 5
4 5 3 5 2 2 1 2 5 4 5
END DATA.

* Macro definition *.
DEFINE GETJDM(Items=!CMDEND).
DATASET NAME OriginalData.
DATASET DECLARE JDMResults.
* Warning, if more than 1000 judges are studied, increase MXLOOPS *.
TEMPORARY.
SET MXLOOPS=1000.
MATRIX.
* Replace variable names as required *.
GET Data /VAR=!Items
         /NAMES=VNames
         /MISSING=OMIT.
COMPUTE NJ=NROW(Data).
COMPUTE NItems= NCOL(Data).
COMPUTE Median=MAKE(NItems,1,0).
COMPUTE pair=(TRUNC(NJ/2) EQ NJ/2).                   /* Check if NJ is odd (0) or even (1) *.
LOOP I=1 TO NItems.
- COMPUTE IData=DATA(:,i).
- COMPUTE Sorted=IData.
- COMPUTE Sorted(GRADE(IData))=IData.
- DO IF pair EQ 0.                                    /* Median formula for odd samples.
- COMPUTE Median(i)=Sorted((NJ+1)/2).
- ELSE.                                               /* Median formula for even samples *.
- COMPUTE Median(i)=(Sorted(NJ/2)+Sorted(1+(NJ/2)))/2.
- END IF.
END LOOP.
PRINT Median
/FORMAT='F8.1'
/RNAMES=VNames
/TITLE='Median of each item'.
COMPUTE MedMat=MAKE(NJ,1,1)*T(Median).
COMPUTE AbsDiff=ABS(Data-MedMat).
COMPUTE JDM=RSUM(AbsDiff).
PRINT JDM
/FORMAT='F8.1'
/TITLE='Judges discrepancy from the Median'.
COMPUTE VName={'JDM'}.
SAVE JDM /OUTFILE=JDMResults /NAMES=Vname.
PRINT /TITLE='JDM saved to active file JDMResults'.
END MATRIX.
DATASET ACTIVATE JDMResults .
LIST.
!ENDDEFINE.

* Macro call *.
GETJDM Items=I2A TO I2K.

Best regards,
Marta GG

--
For miscellaneous SPSS related statistical stuff, visit:
http://gjyp.nl/marta/

Art Kendall

Re: judges discrepancy from median

Have you compared the results of this approach to running RELIABILITY with items as cases and judges as variables? Did you compare the results to the Krippendorf alpha approach which was on this list some years ago?

Did you come to substantively different conclusions?

Art Kendall
Social Research Consultants

On 9/30/2010 4:05 AM, Marta García-Granero wrote:

28/09/2010 18:18, Hall, Mark wrote:

I am looking for some help. I am validating a survey instrument using a panel of expert judges. I have a number of items and ratings from 1-5 from each judge for each item.
I am looking for an SPSS command/syntax that will allow me to calculate the sum of the Judges discrepancy from the Median (JDM) for each judge for all items.

Hi Mark:

Here it is. An old fashioned (obsolete?) solution that does not use the programmability extension, nor Python, nor R..., just a simple macro and matrix:

* Sample dataset (provided by Mark) *.
DATA LIST LIST/I2a I2b I2c I2d I2e I2f I2g I2h I2i I2j I2k (11 F8.0).
BEGIN DATA
5 5 5 5 5 5 4 4 5 5 4
5 5 5 5 5 5 5 5 5 5 5
3 5 5 5 5 5 5 3 5 4 5
5 5 1 4 5 5 5 5 5 5 5
5 5 5 5 5 5 5 5 5 5 5
3 5 3 5 4 3 5 3 5 5 4
5 5 5 5 5 5 5 5 5 5 5
4 4 5 3 5 5 3 4 5 2 5
5 5 5 5 5 5 5 1 5 5 5
5 4 4 5 4 3 5 5 5 5 5
5 5 5 5 5 5 5 5 5 5 5
4 5 3 5 2 2 1 2 5 4 5
END DATA.

* Macro definition *.
DEFINE GETJDM(Items=!CMDEND).
DATASET NAME OriginalData.
DATASET DECLARE JDMResults.
* Warning, if more than 1000 judges are studied, increase MXLOOPS *.
TEMPORARY.
SET MXLOOPS=1000.
MATRIX.
* Replace variable names as required *.
GET Data /VAR=!Items
         /NAMES=VNames
         /MISSING=OMIT.
COMPUTE NJ=NROW(Data).
COMPUTE NItems= NCOL(Data).
COMPUTE Median=MAKE(NItems,1,0).
COMPUTE pair=(TRUNC(NJ/2) EQ NJ/2).                   /* Check if NJ is odd (0) or even (1) *.
LOOP I=1 TO NItems.
- COMPUTE IData=DATA(:,i).
- COMPUTE Sorted=IData.
- COMPUTE Sorted(GRADE(IData))=IData.
- DO IF pair EQ 0.                                    /* Median formula for odd samples.
- COMPUTE Median(i)=Sorted((NJ+1)/2).
- ELSE.                                               /* Median formula for even samples *.
- COMPUTE Median(i)=(Sorted(NJ/2)+Sorted(1+(NJ/2)))/2.
- END IF.
END LOOP.
PRINT Median
/FORMAT='F8.1'
/RNAMES=VNames
/TITLE='Median of each item'.
COMPUTE MedMat=MAKE(NJ,1,1)*T(Median).
COMPUTE AbsDiff=ABS(Data-MedMat).
COMPUTE JDM=RSUM(AbsDiff).
PRINT JDM
/FORMAT='F8.1'
/TITLE='Judges discrepancy from the Median'.
COMPUTE VName={'JDM'}.
SAVE JDM /OUTFILE=JDMResults /NAMES=Vname.
PRINT /TITLE='JDM saved to active file JDMResults'.
END MATRIX.
DATASET ACTIVATE JDMResults .
LIST.
!ENDDEFINE.

* Macro call *.
GETJDM Items=I2A TO I2K.

Best regards,
Marta GG
--
For miscellaneous SPSS related statistical stuff, visit:
http://gjyp.nl/marta/

Art Kendall
Social Research Consultants

Hall, Mark

Re: judges discrepancy from median

In reply to this post by Marta Garcia-Granero

Thank you to everyone for your advice and assistance.

Mark

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Marta García-Granero
Sent: September 30, 2010 2:05 AM
To: [hidden email]
Subject: Re: judges discrepancy from median

28/09/2010 18:18, Hall, Mark wrote:

I am looking for some help. I am validating a survey instrument using a panel of expert judges. I have a number of items and ratings from 1-5 from each judge for each item.

I am looking for an SPSS command/syntax that will allow me to calculate the sum of the Judges discrepancy from the Median (JDM) for each judge for all items.

--

For miscellaneous SPSS related statistical stuff, visit:

http://gjyp.nl/marta/

Bruce Weaver

Re: judges discrepancy from median

Administrator

In reply to this post by Marta Garcia-Granero

Marta García-Granero-2 wrote

28/09/2010 18:18, Hall, Mark wrote:

> I am looking for some help. I am validating a survey instrument using
> a panel of expert judges. I have a number of items and ratings from
> 1-5 from each judge for each item.
>
> I am looking for an SPSS command/syntax that will allow me to
> calculate the sum of the Judges discrepancy from the Median (JDM) for
> each judge for all items.
>
>
Hi Mark:

Here it is. An old fashioned (obsolete?) solution that does not use the
programmability extension, nor Python, nor R..., just a simple macro and
matrix:

* Sample dataset (provided by Mark) *.
DATA LIST LIST/I2a I2b I2c I2d I2e I2f I2g I2h I2i I2j I2k (11 F8.0).
BEGIN DATA
5 5 5 5 5 5 4 4 5 5 4
5 5 5 5 5 5 5 5 5 5 5
3 5 5 5 5 5 5 3 5 4 5
5 5 1 4 5 5 5 5 5 5 5
5 5 5 5 5 5 5 5 5 5 5
3 5 3 5 4 3 5 3 5 5 4
5 5 5 5 5 5 5 5 5 5 5
4 4 5 3 5 5 3 4 5 2 5
5 5 5 5 5 5 5 1 5 5 5
5 4 4 5 4 3 5 5 5 5 5
5 5 5 5 5 5 5 5 5 5 5
4 5 3 5 2 2 1 2 5 4 5
END DATA.

--- snip ---

Alternatively, one could do this after reading in the data in that format:

* Each row is a judge, currently. FLIP the file
* so that each row is an ITEM.

flip.
rename var( var001 to var012 = j1 to j12).
compute item_median = median(j1 to j12).

do repeat j = j1 to j12 / jdm = jdm1 to jdm12 .
- compute jdm = abs(j - item_median).
end repeat.
format item_median jdm1 to jdm12 (f5.2).

descrip jdm1 to jdm12 / stat = sum.

The SUMS of variables jdm1 to jdm12 give what you want, I think.

Marta Garcia-Granero

Re: judges discrepancy from median

El 30/09/2010 15:51, Bruce Weaver escribió:

> Marta García-Granero-2 wrote:
>
> Alternatively, one could do this after reading in the data in that format:
>
> * Each row is a judge, currently. FLIP the file
> * so that each row is an ITEM.
>
> flip.
> rename var( var001 to var012 = j1 to j12).
> compute item_median = median(j1 to j12).

Problem is: SPSS 15 does NOT not have the "median(j1 TO j12)" function,
and I did not know which version was Mark using. Besides, I wanted a
fully automatic method, that did not require having to modify the DO
REPEAT. I started by writing something quite similar (bur using
AGGREGATE to compute the medians)
> do repeat j = j1 to j12 / jdm = jdm1 to jdm12 .
> - compute jdm = abs(j - item_median).
> end repeat.
> format item_median jdm1 to jdm12 (f5.2).
>
> descrip jdm1 to jdm12 / stat = sum.
>
>
> The SUMS of variables jdm1 to jdm12 give what you want, I think.

Best regards,
Marta GG

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD