Question

[Doyle, Jennifer-2]

I have a huge dataset of faculty evaluations of students – at different grade levels

6 evaluative items;

I want to calculate z-scores--- within grade levels

But wonder – should I calculate individual z-scores for each of the 6 items?

Calculate an average of the 6 items and then calculate a z-score for that?

Any insights would be greatly appreciated.....thanks!

 

 

Jenny

 

Jennifer Doyle, M.A.
Director of Surgical Education
Massachusetts General Hospital
Lecturer on Surgery, Harvard Medical School
3 Hawthorne Place, Suite 105
Boston, MA 02114
Phone:  [phone]
Fax:       [phone]
E-mails: 
[hidden email]   or
[hidden email]

 

I'm not an outlier; I just haven't found my distribution yet! -- Ronan M. Conroy, Lecturer in Biostatistics, Royal College of Surgeons of Ireland

Believe those who are seeking the truth, Doubt those who find it. -- Andre Gide

 The information transmitted in this email is intended only for the person or entity to which it is addressed. It may contain privileged or confidential material. Any review, retransmission, dissemination, or other use of this information by other than the intended recipient is prohibited. If you receive this email in error, please contact the sender and delete the material from any computer.

 

 

 

The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.

[Rich Ulrich]

First:  I would create so-called T-scores instead of z-scores.

There are two protocols for T-scores, both of which are used
with some regularity.  The point for either is that you wish to
estimate how extreme a value is, like knowing that a z-score
of 1.64  marks the 95th percentile.  You can either do that by
approximation, from mean and SD, or you can count the number
of cases in your "huge dataset" and find the percentiles exactly.
The latter form is shaky when there is a tiny dataset.

 1) Translate a z-score so that it has mean=50, SD=10;
 2) Use inverse-normal from the percentile-rank, and express the
result for a mean=50, SD=10 -- 95th percentile becomes 66 (66.4).
[For percentiles, see the options in Rank.]

Second:  Of course you *can* do it either way, by individual item or
by pooling them.  There are additional things you can say, or else that
you unfortunately may have to say, when you have pooled them.   If you
pool:  Will you be looking at artifacts that you have to explain where
there are differences between items? - or do you simply improve the
narrative available?

--
Rich Ulrich

---

Date: Mon, 5 May 2014 16:11:22 +0000
From: [hidden email]
Subject: Question
To: [hidden email]

I have a huge dataset of faculty evaluations of students – at different grade levels

6 evaluative items;

I want to calculate z-scores--- within grade levels

But wonder – should I calculate individual z-scores for each of the 6 items?

Calculate an average of the 6 items and then calculate a z-score for that?

Any insights would be greatly appreciated.....thanks!

 

 

Jenny

 

Jennifer Doyle, M.A.
Director of Surgical Education
Massachusetts General Hospital
Lecturer on Surgery, Harvard Medical School
3 Hawthorne Place, Suite 105
Boston, MA 02114
Phone:  [phone]
Fax:       [phone]
E-mails: 
[hidden email]   or
[hidden email]

 

I'm not an outlier; I just haven't found my distribution yet! -- Ronan M. Conroy, Lecturer in Biostatistics, Royal College of Surgeons of Ireland

Believe those who are seeking the truth, Doubt those who find it. -- Andre Gide

 The information transmitted in this email is intended only for the person or entity to which it is addressed. It may contain privileged or confidential material. Any review, retransmission, dissemination, or other use of this information by other than the intended recipient is prohibited. If you receive this email in error, please contact the sender and delete the material from any computer.

 

 

 

The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.

[Andy W]

From a measurement perspective simply summing the items (ignoring the final linear transformation into Z-scores based on the summed items) presumes a parallel measurement model with equal error variances for each instrument. Z-scoring each item individually and then summing them presumes a tau-equivalent model, where the error variances differ between items.

On its face if the instruments have very different variances it would suggest the parallel measurement model is unlikely (unless you have some other insider info.). Typically you would have prior knowledge if the parallel measurement model (or tau equivalent) is reasonable anyway for your given purposes.

It may be the case that neither procedure is appropriate, that is the motivation for confirmatory factor analysis essentially.