RV: Advice, please?

[Hector Maletta]

 

Interesting question. First thoughts:

1.       Why does you need to standardize the responses? Why don’t use the raw scores?

2.       Z-scores may be used in order to abstract from the mean and variance of a particular sample, to make inferences about a population that may have a different mean and sample. In this case, z-scores are computed relative to the sample mean and SD. On the other hand, z-scores may be calculated relative to a standard or normative mean and SD, in order to place a particular sample with respect to the reference distribution (e.g. when one uses the z-scores of children weight to measure malnutrition, where the z-scores is based on the reference growth curve of well nourished healthy children). Which is your case?

3.       If you measure the same sample (or a partially rotating sample) several times, you may standardize relative to the mean and SD of each occasion,  or relative to the mean and SD of the entire set of observations irrespective of the occasion. In the first case, the mean for each wave of observations will be zero, and the SD=1, and therefore those z-scores would not enable you to see trends from one wave to the next. In the second case, individual measurements are measured relative to the overall mean for all observations, the mean of each wave is not zero, and therefore you are able to ascertain whether the mean or SD has changed over time.

4.       Concerning the difference between individuals and “panels”, one may want to consider each panel as one unit, irrespective of composition, and therefore each group of 4 panel members would generate one (collective) response. On the other hand, one may be interested in individual responses of people when they are participating in a panel, perhaps contrasted with the responses of people acting alone. In the first case, the z-score of each panel group is standardized relative to the mean of all mean panel group scores. In the second case, the z-score of each individual panel member is standardized relative to the mean of all individual panel members (again, this may be done for each occasion or for all occasions, see point 3 above).

5.       Since the responses are apparently scores in a qualitative categorical scale with 4 categories, standardization implies treating such scale as an interval variable (i.e. assuming that the subjective difference between successive categories is constant, i.e. that the subjective difference between responses A and B equals the subjective difference between responses B and C, or C and D, for all individuals, just as differences in age or weight would be). Treating ordinal scales as if they were interval-level measurement is common enough to be forgivable, but one should at least consider the question.

Hector

 

From: SPSSX(r) Discussion [[hidden email]] En nombre de Doyle, Jennifer
Date: Tuesday, January 10, 2012 11:53
To: [hidden email]
Subject: Advice, please?

 

Good Morning~

I have a question regarding z-scores:  I have a database consisting of 121 subjects, each of whom was interviewed by 4 different interviewers:

3 individuals and 1 panel (consisting of a variety of 4-8 individuals) on 4 different dates -- the 3 individuals remained the same on the 4 dates, but the panels varied in terms of who sat on them.  Each interviewer (and panel) graded each interaction on a 4-point scale (A-D).

My question is this:  is it valid to simply calculate a z score on the average of the 4 raw scores / grades each subject received? --

OR
Do I need to calculate individual z-scores for each interviewer -- and how do I then calculate a valid z-score for panels--and then average those to rank subjects? Any guidance would be HUGELY appreciated! 

Best, Jennifer

Jennifer Doyle, M.A.
Lecturer on Surgery, Harvard Medical School
Director of Surgical Education
Massachusetts General Hospital
55 Parkman Street - WAC 455Q
Boston, MA 02114
Phone:  [phone]
Fax:       [phone]
[hidden email]

 

I'm not an outlier; I just haven't found my distribution yet! -- Ronan M. Conroy, Lecturer in Biostatistics, Royal College of Surgeons of Ireland

Believe those who are seeking the truth, Doubt those who find it. -- Andre Gide

 The information transmitted in this email is intended only for the person or entity to which it is addressed. It may contain privileged or confidential material. Any review, retransmission, dissemination, or other use of this information by other than the intended recipient is prohibited. If you receive this email in error, please contact the sender and delete the material from any computer.