Advice
Locked
 
|
Good afternoon-- I have a database of test scores on 10 different clinical scenarios/cases; with an unequal number of test takers in each arm (50 faculty answers; 247 student answers. Which test would best determine whether the differences in scores between faculty/students was significant for each of the scenarios? Thanks for your advice-- Jennifer Jennifer Doyle, M.A.
I'm not an outlier; I just haven't found my distribution yet! -- Ronan M. Conroy, Lecturer in Biostatistics, Royal College of Surgeons of Ireland Believe those who are seeking the truth, Doubt those who find it. -- Andre Gide The information transmitted in this email is intended only for the person or entity to which it is addressed. It may contain privileged or confidential material. Any review, retransmission, dissemination, or other use of this information by other than the intended recipient is prohibited. If you receive this email in error, please contact the sender and delete the material from any computer. The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail. |
|
|
This is what an exploration of the data shows -- I'd thought I
could do ANOVA to test differences between faculty &
learners on each scenario - but my sense is that the numbers are too small to do
anything more than a global t-test? Am I on the mark? Thanks!
Jennifer
Jennifer Doyle, M.A. I'm not an outlier; I just haven't found my distribution yet! -- Ronan M. Conroy, Lecturer in Biostatistics, Royal College of Surgeons of Ireland Believe those who are seeking the truth, Doubt those who find it. -- Andre Gide The information transmitted in this email is intended only for the person or entity to which it is addressed. It may contain privileged or confidential material. Any review, retransmission, dissemination, or other use of this information by other than the intended recipient is prohibited. If you receive this email in error, please contact the sender and delete the material from any computer. From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Doyle, Jennifer Sent: Wednesday, January 25, 2012 12:40 PM To: [hidden email] Subject: Advice Good afternoon-- I have a database of test scores on 10 different clinical scenarios/cases; with an unequal number of test takers in each arm (50 faculty answers; 247 student answers. Which test would best determine whether the differences in scores between faculty/students was significant for each of the scenarios? Thanks for your advice-- Jennifer Jennifer Doyle,
M.A. I'm not an outlier; I just haven't found my distribution yet! -- Ronan M. Conroy, Lecturer in Biostatistics, Royal College of Surgeons of Ireland Believe those who are seeking the truth, Doubt those who find it. -- Andre Gide The information transmitted in this email is intended only for the person or entity to which it is addressed. It may contain privileged or confidential material. Any review, retransmission, dissemination, or other use of this information by other than the intended recipient is prohibited. If you receive this email in error, please contact the sender and delete the material from any computer. The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
In reply to this post by Doyle, Jennifer
Does each respondent have test scores on each of the
10 scenarios? Or are there different groups for each
scenario?
Are the students matched to the faculty members? What questions are you trying to answer? Art Kendall Social Research Consultants On 1/25/2012 12:40 PM, Doyle, Jennifer wrote: ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Art Kendall
Social Research Consultants |
|
|
How fine grained are your dependent variables?
Is the case with missing data the same one for each scenario? Art Kendall Social Research Consultants On 1/25/2012 1:52 PM, Doyle, Jennifer wrote: ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Art Kendall
Social Research Consultants |
|
|
Does this help?
Jennifer Doyle,
M.A. I'm not an outlier; I just haven't found my distribution yet! -- Ronan M. Conroy, Lecturer in Biostatistics, Royal College of Surgeons of Ireland Believe those who are seeking the truth, Doubt those who find it. -- Andre Gide The information transmitted in this email is intended only for the person or entity to which it is addressed. It may contain privileged or confidential material. Any review, retransmission, dissemination, or other use of this information by other than the intended recipient is prohibited. If you receive this email in error, please contact the sender and delete the material from any computer. From: Art Kendall [mailto:[hidden email]] Sent: Wednesday, January 25, 2012 2:02 PM To: Doyle, Jennifer; SPSSX-L post Subject: Re: [SPSSX-L] Advice Is the case with missing data the same one for each scenario? Art Kendall Social Research Consultants On 1/25/2012 1:52 PM, Doyle, Jennifer wrote:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
In reply to this post by Doyle, Jennifer
More clarification, please.
If I understand the table of data, you have about 5 faculty ratings (4 to 6) and 25 (24 or 25) student ratings on each of 10 scenarios. Is that correct? Unless you have (what seems unlikely) 50 *different* faculty members, and 247 *different* students, your suggestion of an overall t-test is wrong. Proper testing will have to account for each set of ratings done by each person. An unbalanced ANOVA across scenes, identifying IDs, would test whether the Attendings regularly scored higher or lower than the Residents. That would not examine whether Residents might be more varying in their responses. Art also asked, "What hypotheses are you interested in?" -- Rich Ulrich Date: Wed, 25 Jan 2012 13:43:04 -0500 From: [hidden email] Subject: Re: Advice - Follow-up To: [hidden email] This is what an exploration of the data shows -- I'd thought I
could do ANOVA to test differences between faculty &
learners on each scenario - but my sense is that the numbers are too small to do
anything more than a global t-test? Am I on the mark? Thanks!
Jennifer [snip, table; previous] |
|
|
Yes -- correct -- our hypothesis is that the faculty will
consistently be significantly different (across all scenarios) than
learners.....
Can I legitimately do an "unbalanced" ANOVA with such few
faculty? Thanks --jennifer
Jennifer
Doyle, M.A. I'm not an outlier; I just haven't found my distribution yet! -- Ronan M. Conroy, Lecturer in Biostatistics, Royal College of Surgeons of Ireland Believe those who are seeking the truth, Doubt those who find it. -- Andre Gide The information transmitted in this email is intended only for the person or entity to which it is addressed. It may contain privileged or confidential material. Any review, retransmission, dissemination, or other use of this information by other than the intended recipient is prohibited. If you receive this email in error, please contact the sender and delete the material from any computer. From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Rich Ulrich Sent: Wednesday, January 25, 2012 2:18 PM To: [hidden email] Subject: Re: Advice - Follow-up More clarification, please. If I understand the table of data, you have about 5 faculty ratings (4 to 6) and 25 (24 or 25) student ratings on each of 10 scenarios. Is that correct? Unless you have (what seems unlikely) 50 *different* faculty members, and 247 *different* students, your suggestion of an overall t-test is wrong. Proper testing will have to account for each set of ratings done by each person. An unbalanced ANOVA across scenes, identifying IDs, would test whether the Attendings regularly scored higher or lower than the Residents. That would not examine whether Residents might be more varying in their responses. Art also asked, "What hypotheses are you interested in?" -- Rich Ulrich Date: Wed, 25 Jan 2012 13:43:04 -0500 From: [hidden email] Subject: Re: Advice - Follow-up To: [hidden email] This is what an exploration of the data shows -- I'd thought I could
do ANOVA to test differences between faculty & learners
on each scenario - but my sense is that the numbers are too small to do anything
more than a global t-test? Am I on the mark? Thanks!
Jennifer [snip, table;
previous]The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail. |
|
|
To be more clear than I was -- I was referring to a design
that is basically repeated measures, Scenes (10) by Group (2); and it needs to be analyzed with IDs specified, since IDs are not balanced. (Presumably, each ID had several ratings.) If your table does not reveal some coding errors, there are at least 6 different faculty members, though there are usually only 5 ratings. You did not confirm that the same 25 students did all 25 Resident ratings. If there were a lot more raters than that, the analysis could have difficulties from sparseness. I think you would set this up using MIXED, specifying that IDs are collected within Group; but I don't have the syntax. It is certainly "legitimate" to do an analysis with small N, even when that analysis lacks power. But 50 ratings across 5 or 6 raters is not especially "few", in the general universe of studies. -- Rich Ulrich Date: Wed, 25 Jan 2012 14:25:14 -0500 From: [hidden email] Subject: Re: Advice - Follow-up To: [hidden email] Yes -- correct -- our hypothesis is that the faculty will
consistently be significantly different (across all scenarios) than
learners.....
Can I legitimately do an "unbalanced" ANOVA with such few
faculty? Thanks --jennifer [snip, previous] |
|
|
I'm going to go out on a limb here, as I'm still not terribly clear about the design. Assuming I understand, here are some preliminary thoughts... First and foremost, in order to employ the MIXED procedure, the dataset needs to be structured in vertical format as follows:
ID Group Scenario Rating --------------------------- 1 1 1 missing**
1 1 2 score** . . . . . . . . 1 1 10 score 2 1 1 score 2 1 2 score
. . . . . . . . 2 1 10 score 25 1 1 missing 25 1 2 score . . . .
. . . . 25 1 10 score 1 2 1 score 1 2 2 score . . . . . . . .
1 2 10 score
2 2 1 score 2 2 2 score . . . . . . . . 2 2 10 score 6 2 1 missing
6 2 2 score . . . . 6 2 10 score --------------------------- where ID = Subject identification variable which starts at 1 for each Group Group = Grouping indicator variable (1=Student, 2=Faculty member) Scenario = Scenario indicator variable (1 through 10 Scenarios)
Rating = Scenario Ratings **missing = missing response data **score = valid response data Although I have never tried the code below, I think it should test for group differences in mean ratings while estimating group-specific variance components. In other words, the model assumes within-subject correlation, which is permitted to differ across groups. Hope that makes sense.
mixed Rating by Group /fixed=Group /method=reml /print=solution /random=Group | subject(ID) covtype(diag). It should be noted that the estimated variance components will be biased given the small sample sizes, especially the variance component for the faculty group. As I think about this study, I question whether my proposed code is the optimal approach. Anyway, no time to think about it further right now.
If someone thinks I've misunderstood a fundamental issue, please write back and I will try to adjust the MIXED code accordingly to help the OP.
Ryan
On Wed, Jan 25, 2012 at 2:49 PM, Rich Ulrich <[hidden email]> wrote:
|
| Free forum by Nabble | Edit this page |