SPSSX Discussion

Factorial ANOVA with Ranked Data

Classic

List

Threaded

7 messages Options

Evan Harrington, Ph.D.

Factorial ANOVA with Ranked Data

I'd like to solicit advice on conducting a factorial ANOVA (between subjects) using ranked data.

A student devised the design and I'm looking for advice to give her on how to appropriately analyze the data. Instead of using interval scales she used a ranking system where each subject had to choose from 8 options, and each option had to be ranked relative to each other, resulting in a set of ranks that are yoked (to some extent; some options are forced toward the end of the spectrum of ranks).

The ranking system lends itself well to the Friedman test (provided you ignore the independent variables and analyze the data as a whole). I assume the student had the Friedman test in mind when creating the scoring system. However, I am looking for a way to capitalize on the 2x2 factorial design. I've done a fair amount of digging and I've come up with some interesting work by Conover & Iman (1981), Larry Toothaker, and by Puri & Sen (1985) -- the consensus seems to be that you can analyze ranked data (if the ranking is done as you would for the Kruskal-Wallis, for instance) provided that you analyze the ANOVA main effects without interaction (or vice versa) but if you attempt to analyze the whole model there is a strong chance of anomalous results.

I have some ideas about testing contrasts on the data, or keeping it based on Kruskal-Wallis, but I wanted to check and see if anyone had thoughts on how to adjust the ANOVA to deal with ranked data of this type.

Thanks

Evan Harrington, Ph.D.
Department of Forensic Psychology
The Chicago School of Professional Psychology
325 N. Wells Street
Chicago, IL 60654

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Bruce Weaver

Re: Factorial ANOVA with Ranked Data

Administrator

Hi Evan. Do you have repeated measures on both factors, or is it a between-within design? Thanks for clarifying.

Bruce

Evan Harrington, Ph.D. wrote

I'd like to solicit advice on conducting a factorial ANOVA (between subjects) using ranked data.

A student devised the design and I'm looking for advice to give her on how to appropriately analyze the data. Instead of using interval scales she used a ranking system where each subject had to choose from 8 options, and each option had to be ranked relative to each other, resulting in a set of ranks that are yoked (to some extent; some options are forced toward the end of the spectrum of ranks).

The ranking system lends itself well to the Friedman test (provided you ignore the independent variables and analyze the data as a whole). I assume the student had the Friedman test in mind when creating the scoring system. However, I am looking for a way to capitalize on the 2x2 factorial design. I've done a fair amount of digging and I've come up with some interesting work by Conover & Iman (1981), Larry Toothaker, and by Puri & Sen (1985) -- the consensus seems to be that you can analyze ranked data (if the ranking is done as you would for the Kruskal-Wallis, for instance) provided that you analyze the ANOVA main effects without interaction (or vice versa) but if you attempt to analyze the whole model there is a strong chance of anomalous results.

I have some ideas about testing contrasts on the data, or keeping it based on Kruskal-Wallis, but I wanted to check and see if anyone had thoughts on how to adjust the ANOVA to deal with ranked data of this type.

Thanks

Evan Harrington, Ph.D.
Department of Forensic Psychology
The Chicago School of Professional Psychology
325 N. Wells Street
Chicago, IL 60654

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

PLEASE NOTE THE FOLLOWING:
1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above.
2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/).

Evan Harrington, Ph.D.

Re: Factorial ANOVA with Ranked Data

There is a 2x2 factorial between subjects design, resulting in 4 cells with random assignment. Each subject is asked to make the rankings on each of 8 response options (which essentially ruins the ANOVA design).

I'm sort of intrigued by the problem of trying to force a parametric approach on nonparametric data, and whether it can be done.

Evan Harrington, Ph.D.
Department of Forensic Psychology
The Chicago School of Professional Psychology
325 N. Wells Street
Chicago, IL 60654
________________________________________
From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Bruce Weaver [[hidden email]]
Sent: Tuesday, April 12, 2011 4:09 PM
To: [hidden email]
Subject: Re: Factorial ANOVA with Ranked Data

Hi Evan. Do you have repeated measures on both factors, or is it a
between-within design? Thanks for clarifying.

Bruce

Evan Harrington, Ph.D. wrote:

>
> I'd like to solicit advice on conducting a factorial ANOVA (between
> subjects) using ranked data.
>
> A student devised the design and I'm looking for advice to give her on how
> to appropriately analyze the data. Instead of using interval scales she
> used a ranking system where each subject had to choose from 8 options, and
> each option had to be ranked relative to each other, resulting in a set of
> ranks that are yoked (to some extent; some options are forced toward the
> end of the spectrum of ranks).
>
> The ranking system lends itself well to the Friedman test (provided you
> ignore the independent variables and analyze the data as a whole). I
> assume the student had the Friedman test in mind when creating the scoring
> system. However, I am looking for a way to capitalize on the 2x2 factorial
> design. I've done a fair amount of digging and I've come up with some
> interesting work by Conover & Iman (1981), Larry Toothaker, and by Puri &
> Sen (1985) -- the consensus seems to be that you can analyze ranked data
> (if the ranking is done as you would for the Kruskal-Wallis, for instance)
> provided that you analyze the ANOVA main effects without interaction (or
> vice versa) but if you attempt to analyze the whole model there is a
> strong chance of anomalous results.
>
> I have some ideas about testing contrasts on the data, or keeping it based
> on Kruskal-Wallis, but I wanted to check and see if anyone had thoughts on
> how to adjust the ANOVA to deal with ranked data of this type.
>
> Thanks
>
>
> Evan Harrington, Ph.D.
> Department of Forensic Psychology
> The Chicago School of Professional Psychology
> 325 N. Wells Street
> Chicago, IL 60654
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>

-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Factorial-ANOVA-with-Ranked-Data-tp4299119p4299141.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Kelly Vander Ley

Automatic reply: Factorial ANOVA with Ranked Data

I will be out of the office from Tuesday April 12th -April 18th, returning to the office on April 19th. I will not have regular access to email. If you need immediate assistance please call the main office number 503/223-8248 or 800/788-1887 and the receptionist will ensure that I get the message. Thank you. Kelly

Swank, Paul R

Re: Factorial ANOVA with Ranked Data

In reply to this post by Evan Harrington, Ph.D.

Could you bootstrap the ANOVA?

Dr. Paul R. Swank,
Professor and Director of Research
Children's Learning Institute
University of Texas Health Science Center-Houston

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Evan Harrington
Sent: Tuesday, April 12, 2011 4:21 PM
To: [hidden email]
Subject: Re: Factorial ANOVA with Ranked Data

There is a 2x2 factorial between subjects design, resulting in 4 cells with random assignment. Each subject is asked to make the rankings on each of 8 response options (which essentially ruins the ANOVA design).

I'm sort of intrigued by the problem of trying to force a parametric approach on nonparametric data, and whether it can be done.

Evan Harrington, Ph.D.
Department of Forensic Psychology
The Chicago School of Professional Psychology
325 N. Wells Street
Chicago, IL 60654
________________________________________
From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Bruce Weaver [[hidden email]]
Sent: Tuesday, April 12, 2011 4:09 PM
To: [hidden email]
Subject: Re: Factorial ANOVA with Ranked Data

Hi Evan. Do you have repeated measures on both factors, or is it a
between-within design? Thanks for clarifying.

Bruce

Evan Harrington, Ph.D. wrote:

Rich Ulrich

Re: Factorial ANOVA with Ranked Data

In reply to this post by Evan Harrington, Ph.D.

> Date: Tue, 12 Apr 2011 16:21:12 -0500
> From: [hidden email]
> Subject: Re: Factorial ANOVA with Ranked Data
> To: [hidden email]
>
> There is a 2x2 factorial between subjects design, resulting in 4 cells with random assignment. Each subject is asked to make the rankings on each of 8 response options (which essentially ruins the ANOVA design).

Eight responses?
The main analysis should probably be performed on a composite
score, or a couple of scores. If you average even two scores
that are 1-8, you suppress the potential problem of interactions
which can be introduced by the natural bad scaling of ranks.

I could suggest doing a test, first, to see if anything shows
up. Why borrow trouble? If there are no main effects, there
is nothing to induce an artifactual interaction.

>
> I'm sort of intrigued by the problem of trying to force a parametric approach on nonparametric data, and whether it can be done.
>

One artificial way is to score the ranks as percentiles; then
score the percentiles to logit or normal. I am curious as to
whether anyone has references on doing that, since it has long
seemed pretty obvious to me but I have never seen it discussed.

--
Rich Ulrich

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Mike

Re: Factorial ANOVA with Ranked Data

In reply to this post by Evan Harrington, Ph.D.

Take a look at this article by Mewhort et al. Not exactly what you
want but it may point to another solution for your situation.
http://www.springerlink.com/content/h07x273h3nw21872/

-Mike Palij
New York University
[hidden email]

----- Original Message -----
From: "Evan Harrington" <[hidden email]>
To: <[hidden email]>
Sent: Tuesday, April 12, 2011 5:21 PM
Subject: Re: Factorial ANOVA with Ranked Data

> There is a 2x2 factorial between subjects design, resulting in 4 cells with random assignment. Each subject is asked to make the rankings on each of 8 response options (which essentially ruins the ANOVA design).
>
> I'm sort of intrigued by the problem of trying to force a parametric approach on nonparametric data, and whether it can be done.
>
>
> Evan Harrington, Ph.D.
> Department of Forensic Psychology
> The Chicago School of Professional Psychology
> 325 N. Wells Street
> Chicago, IL 60654
> ________________________________________
> From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Bruce Weaver [[hidden email]]
> Sent: Tuesday, April 12, 2011 4:09 PM
> To: [hidden email]
> Subject: Re: Factorial ANOVA with Ranked Data
>
> Hi Evan. Do you have repeated measures on both factors, or is it a
> between-within design? Thanks for clarifying.
>
> Bruce
>
>
>
> Evan Harrington, Ph.D. wrote:
>>
>> I'd like to solicit advice on conducting a factorial ANOVA (between
>> subjects) using ranked data.
>>
>> A student devised the design and I'm looking for advice to give her on how
>> to appropriately analyze the data. Instead of using interval scales she
>> used a ranking system where each subject had to choose from 8 options, and
>> each option had to be ranked relative to each other, resulting in a set of
>> ranks that are yoked (to some extent; some options are forced toward the
>> end of the spectrum of ranks).
>>
>> The ranking system lends itself well to the Friedman test (provided you
>> ignore the independent variables and analyze the data as a whole). I
>> assume the student had the Friedman test in mind when creating the scoring
>> system. However, I am looking for a way to capitalize on the 2x2 factorial
>> design. I've done a fair amount of digging and I've come up with some
>> interesting work by Conover & Iman (1981), Larry Toothaker, and by Puri &
>> Sen (1985) -- the consensus seems to be that you can analyze ranked data
>> (if the ranking is done as you would for the Kruskal-Wallis, for instance)
>> provided that you analyze the ANOVA main effects without interaction (or
>> vice versa) but if you attempt to analyze the whole model there is a
>> strong chance of anomalous results.
>>
>> I have some ideas about testing contrasts on the data, or keeping it based
>> on Kruskal-Wallis, but I wanted to check and see if anyone had thoughts on
>> how to adjust the ANOVA to deal with ranked data of this type.
>>
>> Thanks
>>
>>
>> Evan Harrington, Ph.D.
>> Department of Forensic Psychology
>> The Chicago School of Professional Psychology
>> 325 N. Wells Street
>> Chicago, IL 60654
>>
>> =====================
>> To manage your subscription to SPSSX-L, send a message to
>> [hidden email] (not to SPSSX-L), with no body text except the
>> command. To leave the list, send the command
>> SIGNOFF SPSSX-L
>> For a list of commands to manage subscriptions, send the command
>> INFO REFCARD
>>
>
>
> -----
> --
> Bruce Weaver
> [hidden email]
> http://sites.google.com/a/lakeheadu.ca/bweaver/
>
> "When all else fails, RTFM."
>
> NOTE: My Hotmail account is not monitored regularly.
> To send me an e-mail, please use the address shown above.
>
> --
> View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Factorial-ANOVA-with-Ranked-Data-tp4299119p4299141.html
> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD