SPSSX Discussion

Fw: Separate One-Way vs Two-Way Repeated Measures

classic Classic list List threaded Threaded
♦
♦
 5 messages Options
Mike
Reply | Threaded
Open this post in threaded view
♦
♦
|

Fw: Separate One-Way vs Two-Way Repeated Measures

Mike

I had sent the message below to Joao off-list but then
thought that it might elicit some additional useful comments
on the SPSSX-L list.
 
-Mike Palij
New York University
 
----- Original Message -----
Sent: Wednesday, April 29, 2015 9:40 AM

Let me rephrase what you say below:
 
(1) You have a 3 by 6 within-subjects design with voice
(3 levels) factorially crossed by time condition (6 levels;
not sure what this variable actually is) which can be
represented by the following table:
 

 

Time Condition

Voice

1

2

3

4

5

6

1

V1T1

V1T2

V1T3

V1T4

 

 

2

V2T1

V2T2

V2T3

V2T4

V2T5

V2T6

3

V3T1

V3T2

V3T3

V3T3

V3T4

 

 
Blank cells indicate combinations of voice and
time condition that you either did not collect
data in (why?) or were structurally blank (i.e.,
the combination of conditions do not exist
even though they appear to the factorial design).
 
(2) You are not interested in the main effects
of either voice or time conditions.
 
(3) You are interested in comparing components
of the 2-way interaction between voice and time.
 
(4) Since traditional repeated measures ANOVA
assumes complete data, you cannot use it to analyze
this design.  One alternative is to change the design
to a 3 (voice) by 4 (time 1-4) but though this simplifies
things, it raises questions about whether it is appropriate
and whether it answers the research questions you
really want answered. However, such a design would
tell you if you have a significant 2-way interaction; if
you do not, analysis of components of the interaction
would need justification.
 
(5) Instead of taking the ANOVA route, you could
specific the analysis you want as planned comparisons
and calculate the appropriate error terms for such
comparisons.  But you would have to think such an
analysis through.  Additional considerations have to do
with whether you sphericity or not -- in which case
what is the nature of your variance-covariance matrix.
 
(6) I could be wrong but I think most statistical consultants
would suggest that you use a multilevel model to analyze
the design above, one reason being that it would allow
you to make use of all of the data (potentially).  There
are people on SPSSX-L who are familiar with multilevel
analysis for these types of designs (though a 2-way
incomplete within-subject design is a little odd, at least
in the present form -- can the empty cells be treated as
"missing at random"?).
 
(7) If you want to take the multilevel route, these is some
literature on incomplete within-subject or incomplete
repeated measures ANOVA (you can do a search on
scholar.google.com for the literature).  One source you
might want to look at is the work by Lesa Hoffman,
for example, see:
 
Hoffman L, Rovine MJ. Multilevel models for experimental
psychologists: Foundations and illustrative examples. Behavior
Research Methods. 2007;39:101–117.
 
This article is on the University of Nebraska-Lincoln's UNL)
digital commons and can be accessed here:
 
At the time of the writing of this article, Hoffman was faculty
at UNL but she has apparently changed affiliations and here is
a link to her webpage:
 
You can examine these sources and the literature in working out
what analysis you want to do.  Some on SPSS may be able to
help with SPSS syntax for the analysis (Hoffman provided some
examples of SPSS analysis but they are no longer on the UNL
website though her article provides an address there; it may be
on her new site).
 
(8) Remember that you have a completely within-subject design
but many examples will be "mixed designs" (i.e., have between-subject
and within-subject ind vars; another definition of "mixed design" is
that some variables are "fixed effects" and other are "random effects",
a distinction you will have to make if you go the multilevel analysis
route).
 
HTH.
 
-Mike Palij
New York University
 
 
---- Original Message -----
Sent: Wednesday, April 29, 2015 6:44 AM
Subject: Separate One-Way vs Two-Way Repeated Measures

Hello everyone,

I'm struggling with a Within-Subjects analysis and I haven't found the answer to my doubts, so here it goes:

I have two IV (Time and Voice Conditions) and one DV (continuous variable).

Time conditions depend on the Time to Complete the task
Voice conditions refer to different voices used

                                                           Time Condition

                                         1         2          3          4          5         6
                             1          x          x          x          x

Voice Conditions     2          x          x          x          x          x          x

                             3                      x          x          x          x          x

This table shows all combinations Time X Voice that were tested, by all participants (30). For example, Time condition 1 with Voice condition 3 was not executed by any participant.

What I want with this test is to obtain the following comparisons:
- Within each Time Condition, check for differences in Voice conditions
- For each voice condition, check for differences in time condition (in particular, I want to know when does performance start to deteriorate significantly).

My first thoughts were to perform one-way repeated measures ANOVA (or Friedman due to normality violation) within each time condition; and then within each voice condition (Followed by post-hocs).

However, I found that I could analyze these two variables together with the two-way repeated measures ANOVA. I performed this analysis using the General Linear Model -> Repeated Measures menu. Yet, I found the following problems:

- I have a few combinations that don't exist (for example Time 1 X Voice 3); If I use empty values, I get no results for all the analysis... but I have to insert one variable in order to proceed with the analysis. What should I do?
- When checking for normality, almost all TimeXVoice conditions violate the assumption. I read that normality is not that strict in this test, but some of the conditions show Sig results of ,000 and almost all of them are below ,050. Should I go for a non-parametric alternative?

Is it OK to use the one-way version and analyze each of the variables separately or should I do the two-way analysis? How should I overcome the problems referred above? Should I consider a different test?

I hope I have provided enough details.
Thanks in advance,

--
João Guerreiro
- Researcher @  INESC-ID / Visualization and Intelligent MultiModal Interfaces Group
- PhD Student @ IST / Technical University of Lisbon
- Tel: +351.21.4233532
- web: http://web.ist.utl.pt/joao.p.guerreiro/
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
bdates
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Separate One-Way vs Two-Way Repeated Measures

bdates

Mike,

 

Thanks for this. Having done a lot of work on missing data, I would not recommend imputing data in this instance. First of all, there are only 30 subjects, and so the mechanism of missingness is probably not random. Even then, however, of three possible cells at Time 6 there is only one voice category. I don’t believe that is enough to use any kind of maximum likelihood model, e.g., MI or EM. I would also be hesitant to try correlational approaches as well, although Joost van Ginkel has been very active on this listserve, providing lots of imputation syntax. Perhaps he might weigh in.

 

Brian Dates, M.A.
Director of Evaluation and Research | Evaluation & Research | Southwest Counseling Solutions
Southwest Solutions
1700 Waterman, Detroit, MI 48209
313-841-8900 (x7442) office | 313-849-2702 fax
[hidden email] | www.swsol.org

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Mike Palij
Sent: Wednesday, April 29, 2015 11:51 AM
To: [hidden email]
Subject: Fw: Separate One-Way vs Two-Way Repeated Measures

 

I had sent the message below to Joao off-list but then

thought that it might elicit some additional useful comments

on the SPSSX-L list.

 

-Mike Palij

New York University

 

----- Original Message -----

Sent: Wednesday, April 29, 2015 9:40 AM

 

Let me rephrase what you say below:

 

(1) You have a 3 by 6 within-subjects design with voice

(3 levels) factorially crossed by time condition (6 levels;

not sure what this variable actually is) which can be

represented by the following table:

 

 

Time Condition

Voice

1

2

3

4

5

6

1

V1T1

V1T2

V1T3

V1T4

 

 

2

V2T1

V2T2

V2T3

V2T4

V2T5

V2T6

3

V3T1

V3T2

V3T3

V3T3

V3T4

 

 

Blank cells indicate combinations of voice and

time condition that you either did not collect

data in (why?) or were structurally blank (i.e.,

the combination of conditions do not exist

even though they appear to the factorial design).

 

(2) You are not interested in the main effects

of either voice or time conditions.

 

(3) You are interested in comparing components

of the 2-way interaction between voice and time.

 

(4) Since traditional repeated measures ANOVA

assumes complete data, you cannot use it to analyze

this design.  One alternative is to change the design

to a 3 (voice) by 4 (time 1-4) but though this simplifies

things, it raises questions about whether it is appropriate

and whether it answers the research questions you

really want answered. However, such a design would

tell you if you have a significant 2-way interaction; if

you do not, analysis of components of the interaction

would need justification.

 

(5) Instead of taking the ANOVA route, you could

specific the analysis you want as planned comparisons

and calculate the appropriate error terms for such

comparisons.  But you would have to think such an

analysis through.  Additional considerations have to do

with whether you sphericity or not -- in which case

what is the nature of your variance-covariance matrix.

 

(6) I could be wrong but I think most statistical consultants

would suggest that you use a multilevel model to analyze

the design above, one reason being that it would allow

you to make use of all of the data (potentially).  There

are people on SPSSX-L who are familiar with multilevel

analysis for these types of designs (though a 2-way

incomplete within-subject design is a little odd, at least

in the present form -- can the empty cells be treated as

"missing at random"?).

 

(7) If you want to take the multilevel route, these is some

literature on incomplete within-subject or incomplete

repeated measures ANOVA (you can do a search on

scholar.google.com for the literature).  One source you

might want to look at is the work by Lesa Hoffman,

for example, see:

 

Hoffman L, Rovine MJ. Multilevel models for experimental

psychologists: Foundations and illustrative examples. Behavior

Research Methods. 2007;39:101–117.

 

This article is on the University of Nebraska-Lincoln's UNL)

digital commons and can be accessed here:

 

At the time of the writing of this article, Hoffman was faculty

at UNL but she has apparently changed affiliations and here is

a link to her webpage:

 

You can examine these sources and the literature in working out

what analysis you want to do.  Some on SPSS may be able to

help with SPSS syntax for the analysis (Hoffman provided some

examples of SPSS analysis but they are no longer on the UNL

website though her article provides an address there; it may be

on her new site).

 

(8) Remember that you have a completely within-subject design

but many examples will be "mixed designs" (i.e., have between-subject

and within-subject ind vars; another definition of "mixed design" is

that some variables are "fixed effects" and other are "random effects",

a distinction you will have to make if you go the multilevel analysis

route).

 

HTH.

 

-Mike Palij

New York University

 

 

---- Original Message -----

Sent: Wednesday, April 29, 2015 6:44 AM

Subject: Separate One-Way vs Two-Way Repeated Measures

 

Hello everyone,

I'm struggling with a Within-Subjects analysis and I haven't found the answer to my doubts, so here it goes:

I have two IV (Time and Voice Conditions) and one DV (continuous variable).

Time conditions depend on the Time to Complete the task
Voice conditions refer to different voices used

                                                           Time Condition

                                         1         2          3          4          5         6
                             1          x          x          x          x

Voice Conditions     2          x          x          x          x          x          x

                             3                      x          x          x          x          x

This table shows all combinations Time X Voice that were tested, by all participants (30). For example, Time condition 1 with Voice condition 3 was not executed by any participant.

What I want with this test is to obtain the following comparisons:
- Within each Time Condition, check for differences in Voice conditions
- For each voice condition, check for differences in time condition (in particular, I want to know when does performance start to deteriorate significantly).

My first thoughts were to perform one-way repeated measures ANOVA (or Friedman due to normality violation) within each time condition; and then within each voice condition (Followed by post-hocs).

However, I found that I could analyze these two variables together with the two-way repeated measures ANOVA. I performed this analysis using the General Linear Model -> Repeated Measures menu. Yet, I found the following problems:

- I have a few combinations that don't exist (for example Time 1 X Voice 3); If I use empty values, I get no results for all the analysis... but I have to insert one variable in order to proceed with the analysis. What should I do?
- When checking for normality, almost all TimeXVoice conditions violate the assumption. I read that normality is not that strict in this test, but some of the conditions show Sig results of ,000 and almost all of them are below ,050. Should I go for a non-parametric alternative?

Is it OK to use the one-way version and analyze each of the variables separately or should I do the two-way analysis? How should I overcome the problems referred above? Should I consider a different test?

I hope I have provided enough details.
Thanks in advance,

 

--

João Guerreiro
- Researcher @  INESC-ID / Visualization and Intelligent MultiModal Interfaces Group
- PhD Student @ IST / Technical University of Lisbon
- Tel: +351.21.4233532
- web: http://web.ist.utl.pt/joao.p.guerreiro/

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
João Guerreiro
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Separate One-Way vs Two-Way Repeated Measures

João Guerreiro
Thank you all for your replies.

I don't know if this comment changes something, but I just want to make a correction to Mike Palik's table:
- Time 6 X Voice 3 condition exists. The other condition that does not exist is Time 1 X Voice 3.

In my case, these missing conditions are design related:
- Time 1 X Voice 3 would be a step below a baseline condition and would not make sense in the context of my experiment;
- Time 5 X Voice 1 and Time 6 X Voice 1 were extremely difficult (seen in pilot studies) and therefore we wanted to avoid to frustrate/exhaust the participants.

For what I have been reading about mixed effects model (it is the first time I'm reading about it), it seemed a good fit: using time and voice as fixed effects (should also consider time*voice?) and the participant as random effect. But I believe I am missing something here...
Moreover, both normality and sphericity are violated in most cases. Does this also influence mixed effects model as it does with ANOVA tests (requiring non-parametric testing)?

Would it be statistically invalid/wrong to perform, for example, Friedman Tests for each Time condition (to compare the 3 voice conditions in that Time condition)?

Thank you all!


2015-04-29 17:04 GMT+01:00 Dates, Brian <[hidden email]>:

Mike,

 

Thanks for this. Having done a lot of work on missing data, I would not recommend imputing data in this instance. First of all, there are only 30 subjects, and so the mechanism of missingness is probably not random. Even then, however, of three possible cells at Time 6 there is only one voice category. I don’t believe that is enough to use any kind of maximum likelihood model, e.g., MI or EM. I would also be hesitant to try correlational approaches as well, although Joost van Ginkel has been very active on this listserve, providing lots of imputation syntax. Perhaps he might weigh in.

 

Brian Dates, M.A.
Director of Evaluation and Research | Evaluation & Research | Southwest Counseling Solutions
Southwest Solutions
1700 Waterman, Detroit, MI 48209
<a href="tel:313-841-8900" value="+13138418900" target="_blank">313-841-8900 (x7442) office | <a href="tel:313-849-2702" value="+13138492702" target="_blank">313-849-2702 fax
[hidden email] | www.swsol.org

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Mike Palij
Sent: Wednesday, April 29, 2015 11:51 AM
To: [hidden email]
Subject: Fw: Separate One-Way vs Two-Way Repeated Measures

 

I had sent the message below to Joao off-list but then

thought that it might elicit some additional useful comments

on the SPSSX-L list.

 

-Mike Palij

New York University

 

----- Original Message -----

Sent: Wednesday, April 29, 2015 9:40 AM

 

Let me rephrase what you say below:

 

(1) You have a 3 by 6 within-subjects design with voice

(3 levels) factorially crossed by time condition (6 levels;

not sure what this variable actually is) which can be

represented by the following table:

 

 

Time Condition

Voice

1

2

3

4

5

6

1

V1T1

V1T2

V1T3

V1T4

 

 

2

V2T1

V2T2

V2T3

V2T4

V2T5

V2T6

3

V3T1

V3T2

V3T3

V3T3

V3T4

 

 

Blank cells indicate combinations of voice and

time condition that you either did not collect

data in (why?) or were structurally blank (i.e.,

the combination of conditions do not exist

even though they appear to the factorial design).

 

(2) You are not interested in the main effects

of either voice or time conditions.

 

(3) You are interested in comparing components

of the 2-way interaction between voice and time.

 

(4) Since traditional repeated measures ANOVA

assumes complete data, you cannot use it to analyze

this design.  One alternative is to change the design

to a 3 (voice) by 4 (time 1-4) but though this simplifies

things, it raises questions about whether it is appropriate

and whether it answers the research questions you

really want answered. However, such a design would

tell you if you have a significant 2-way interaction; if

you do not, analysis of components of the interaction

would need justification.

 

(5) Instead of taking the ANOVA route, you could

specific the analysis you want as planned comparisons

and calculate the appropriate error terms for such

comparisons.  But you would have to think such an

analysis through.  Additional considerations have to do

with whether you sphericity or not -- in which case

what is the nature of your variance-covariance matrix.

 

(6) I could be wrong but I think most statistical consultants

would suggest that you use a multilevel model to analyze

the design above, one reason being that it would allow

you to make use of all of the data (potentially).  There

are people on SPSSX-L who are familiar with multilevel

analysis for these types of designs (though a 2-way

incomplete within-subject design is a little odd, at least

in the present form -- can the empty cells be treated as

"missing at random"?).

 

(7) If you want to take the multilevel route, these is some

literature on incomplete within-subject or incomplete

repeated measures ANOVA (you can do a search on

scholar.google.com for the literature).  One source you

might want to look at is the work by Lesa Hoffman,

for example, see:

 

Hoffman L, Rovine MJ. Multilevel models for experimental

psychologists: Foundations and illustrative examples. Behavior

Research Methods. 2007;39:101–117.

 

This article is on the University of Nebraska-Lincoln's UNL)

digital commons and can be accessed here:

 

At the time of the writing of this article, Hoffman was faculty

at UNL but she has apparently changed affiliations and here is

a link to her webpage:

 

You can examine these sources and the literature in working out

what analysis you want to do.  Some on SPSS may be able to

help with SPSS syntax for the analysis (Hoffman provided some

examples of SPSS analysis but they are no longer on the UNL

website though her article provides an address there; it may be

on her new site).

 

(8) Remember that you have a completely within-subject design

but many examples will be "mixed designs" (i.e., have between-subject

and within-subject ind vars; another definition of "mixed design" is

that some variables are "fixed effects" and other are "random effects",

a distinction you will have to make if you go the multilevel analysis

route).

 

HTH.

 

-Mike Palij

New York University

 

 

---- Original Message -----

Sent: Wednesday, April 29, 2015 6:44 AM

Subject: Separate One-Way vs Two-Way Repeated Measures

 

Hello everyone,



I'm struggling with a Within-Subjects analysis and I haven't found the answer to my doubts, so here it goes:

I have two IV (Time and Voice Conditions) and one DV (continuous variable).

Time conditions depend on the Time to Complete the task
Voice conditions refer to different voices used

                                                           Time Condition

                                         1         2          3          4          5         6
                             1          x          x          x          x

Voice Conditions     2          x          x          x          x          x          x

                             3                      x          x          x          x          x

This table shows all combinations Time X Voice that were tested, by all participants (30). For example, Time condition 1 with Voice condition 3 was not executed by any participant.

What I want with this test is to obtain the following comparisons:
- Within each Time Condition, check for differences in Voice conditions
- For each voice condition, check for differences in time condition (in particular, I want to know when does performance start to deteriorate significantly).

My first thoughts were to perform one-way repeated measures ANOVA (or Friedman due to normality violation) within each time condition; and then within each voice condition (Followed by post-hocs).

However, I found that I could analyze these two variables together with the two-way repeated measures ANOVA. I performed this analysis using the General Linear Model -> Repeated Measures menu. Yet, I found the following problems:

- I have a few combinations that don't exist (for example Time 1 X Voice 3); If I use empty values, I get no results for all the analysis... but I have to insert one variable in order to proceed with the analysis. What should I do?
- When checking for normality, almost all TimeXVoice conditions violate the assumption. I read that normality is not that strict in this test, but some of the conditions show Sig results of ,000 and almost all of them are below ,050. Should I go for a non-parametric alternative?

Is it OK to use the one-way version and analyze each of the variables separately or should I do the two-way analysis? How should I overcome the problems referred above? Should I consider a different test?

I hope I have provided enough details.
Thanks in advance,

 

--

João Guerreiro
- Researcher @  INESC-ID / Visualization and Intelligent MultiModal Interfaces Group
- PhD Student @ IST / Technical University of Lisbon
- Tel: <a href="tel:%2B351.21.4233532" value="+351214233532" target="_blank">+351.21.4233532
- web: http://web.ist.utl.pt/joao.p.guerreiro/

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



--
João Guerreiro
- Researcher @  INESC-ID / Visualization and Intelligent MultiModal Interfaces Group
- PhD Student @ IST / Technical University of Lisbon
- Tel: +351.21.4233532
- web: http://web.ist.utl.pt/joao.p.guerreiro/
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Rich Ulrich
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Separate One-Way vs Two-Way Repeated Measures

Rich Ulrich
In your original post, you also stated an aim that is not well-thought-out:
 > For each voice condition, check for differences in time condition (in particular, I want to know when does performance start to deteriorate significantly).

If you assume a linear trend, which is the most common sort when you
have a good measurement in use, "start to deteriorate" mainly implies
"at the start"; and "significantly", to be meaningful, has to mean, "When
does it drop below a certain, specified value?"  If you do not assume a
linear trend, are you looking to evaluate a "hockey stick" function?  To
measure one of those, you almost certainly want to have a good measure
where you do know that you have normality and sphericity.

Even without the hockey-stick, I do not get past your "normality and sphericity
are violated", when looking for good tests in the two-way, repeated design.
Heterogeneity raises terrible problems for a two-way design, including the
chance that rank-transforms will not work well when the extreme condition
ends up with too little variance. 

WHY do you have heterogeneity/ non-normality?  What are you measuring?
Is there a natural transformation that will fix it? - square roots or logits for
counts;  - logs or reciprocals for times.

The best starting point for transformations is to consider what generates the
numbers.  What, indeed, are you measuring?

If you can't get homogeneity across these time points by transforming, then
you might want to consider constructing your specific set of contrasts, and
using the *contrasts* (not the original scores) in t-tests or linear models.

--
Rich Ulrich



Date: Wed, 29 Apr 2015 17:47:08 +0100
From: [hidden email]
Subject: Re: Separate One-Way vs Two-Way Repeated Measures
To: [hidden email]

Thank you all for your replies.

I don't know if this comment changes something, but I just want to make a correction to Mike Palik's table:
- Time 6 X Voice 3 condition exists. The other condition that does not exist is Time 1 X Voice 3.

In my case, these missing conditions are design related:
- Time 1 X Voice 3 would be a step below a baseline condition and would not make sense in the context of my experiment;
- Time 5 X Voice 1 and Time 6 X Voice 1 were extremely difficult (seen in pilot studies) and therefore we wanted to avoid to frustrate/exhaust the participants.

For what I have been reading about mixed effects model (it is the first time I'm reading about it), it seemed a good fit: using time and voice as fixed effects (should also consider time*voice?) and the participant as random effect. But I believe I am missing something here...
Moreover, both normality and sphericity are violated in most cases. Does this also influence mixed effects model as it does with ANOVA tests (requiring non-parametric testing)?

Would it be statistically invalid/wrong to perform, for example, Friedman Tests for each Time condition (to compare the 3 voice conditions in that Time condition)?

Thank you all!

[... snip, previous]
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Mike
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Separate One-Way vs Two-Way Repeated Measures

Mike
In reply to this post by João Guerreiro

Perhaps I am confused but allow me to enumerate the tests
that I think you might want to do:
 
(1) Time 1: Voice 1 vs Voice 2 (correlated groups t-test
or non parametric test, depending upon distribution of data
and assumptions about populations samples were drawn
from).
 
(2) Time 2: Voice 1 vs Voice 2 vs Voice 3 (repeated
measures ANOVA plus post hoc or nonparametric ANOVA)
 
(3) Time 3: Voice 1 vs Voice 2 vs Voice 3 (same as (2))
 
(4) Time 4: Voice 1 vs Voice 2 vs Voice 3 (same as (2))
 
(5) Time 5: Voice 2 vs Voice 3 (same as (1))
 
(6) Voice 1: Time 1 vs Time 2 vs Time 3 vs Time 4
(one-way repeated measures ANOVA or nonparametric
analog along with appropriate post hoc).
 
(7) Voice 2: Time 1 vs Time 2 vs Time 3 Time 4 vs Time 5 vs Time 6
(one-way repeated-measures ANOVA or nonparametric
analog along with appropriate post hoc).
 
(8) Voice 3: Time 2 vs Time 3 vs Time 4 vs Time 5 (one-way
repeated measures ANOVA or nonparametric analog along
with appropriate post hoc).
 
If you had the full 3 x 6 completely within-subject design,
assuming you have a significant 2-way interaction, you
are essentially asking for simple effects analysis at each level of
voice and at each level of time. Because you do not have a full
design, you have the patchwork of comparisons represented
by (1) - (8) above.  If each test is done at alpha per comparison = .05,
then the alpha you have after doing the 8 test is alpha overall = .34,
or a 34% chance of committing a Type I error (for any ANOVA
that is significant, add the alphas for each post hoc comparisons).
In other words, following this strategy is likely to produce at least
one Type 1 error (i.e., false statistically significant result).
Perhaps I am wrong and someone can suggest an alternative
analysis plan that will keep the Type I error rate down.
 
Sidenote:  if you have not done so, I think it might be a good
idea to graph the means and their standard errors.  Time
would be on the X-axis, the dependent variable on the Y-axis,
and you would have a line for each Voice (3 lines).  At the
very least, this would show whether the lines are parallel or
not. If not, do the line fan out (implying an ordinal interaction)
or crossover (implying a disordinal interaction).  This might
suggest which of the comparison above make sense.
 
HTH.
 
-Mike Palij
New York University
 
 
 
 
 
----- Original Message -----
Sent: Wednesday, April 29, 2015 12:47 PM
Subject: Re: Separate One-Way vs Two-Way Repeated Measures

Thank you all for your replies.

I don't know if this comment changes something, but I just want to make a correction to Mike Palik's table:
- Time 6 X Voice 3 condition exists. The other condition that does not exist is Time 1 X Voice 3.

In my case, these missing conditions are design related:
- Time 1 X Voice 3 would be a step below a baseline condition and would not make sense in the context of my experiment;
- Time 5 X Voice 1 and Time 6 X Voice 1 were extremely difficult (seen in pilot studies) and therefore we wanted to avoid to frustrate/exhaust the participants.

For what I have been reading about mixed effects model (it is the first time I'm reading about it), it seemed a good fit: using time and voice as fixed effects (should also consider time*voice?) and the participant as random effect. But I believe I am missing something here...
Moreover, both normality and sphericity are violated in most cases. Does this also influence mixed effects model as it does with ANOVA tests (requiring non-parametric testing)?

Would it be statistically invalid/wrong to perform, for example, Friedman Tests for each Time condition (to compare the 3 voice conditions in that Time condition)?

Thank you all!


2015-04-29 17:04 GMT+01:00 Dates, Brian <[hidden email]>:

Mike,

 

Thanks for this. Having done a lot of work on missing data, I would not recommend imputing data in this instance. First of all, there are only 30 subjects, and so the mechanism of missingness is probably not random. Even then, however, of three possible cells at Time 6 there is only one voice category. I don’t believe that is enough to use any kind of maximum likelihood model, e.g., MI or EM. I would also be hesitant to try correlational approaches as well, although Joost van Ginkel has been very active on this listserve, providing lots of imputation syntax. Perhaps he might weigh in.

 

Brian Dates, M.A.
Director of Evaluation and Research | Evaluation & Research | Southwest Counseling Solutions
Southwest Solutions
1700 Waterman, Detroit, MI 48209
<A href="tel:313-841-8900" target=_blank value="+13138418900">313-841-8900 (x7442) office | <A href="tel:313-849-2702" target=_blank value="+13138492702">313-849-2702 fax
[hidden email] | www.swsol.org

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Mike Palij
Sent: Wednesday, April 29, 2015 11:51 AM
To: [hidden email]
Subject: Fw: Separate One-Way vs Two-Way Repeated Measures

 

I had sent the message below to Joao off-list but then

thought that it might elicit some additional useful comments

on the SPSSX-L list.

 

-Mike Palij

New York University

 

----- Original Message -----

Sent: Wednesday, April 29, 2015 9:40 AM

 

Let me rephrase what you say below:

 

(1) You have a 3 by 6 within-subjects design with voice

(3 levels) factorially crossed by time condition (6 levels;

not sure what this variable actually is) which can be

represented by the following table:

 

 

Time Condition

Voice

1

2

3

4

5

6

1

V1T1

V1T2

V1T3

V1T4

 

 

2

V2T1

V2T2

V2T3

V2T4

V2T5

V2T6

3

V3T1

V3T2

V3T3

V3T3

V3T4

 

 

Blank cells indicate combinations of voice and

time condition that you either did not collect

data in (why?) or were structurally blank (i.e.,

the combination of conditions do not exist

even though they appear to the factorial design).

 

(2) You are not interested in the main effects

of either voice or time conditions.

 

(3) You are interested in comparing components

of the 2-way interaction between voice and time.

 

(4) Since traditional repeated measures ANOVA

assumes complete data, you cannot use it to analyze

this design.  One alternative is to change the design

to a 3 (voice) by 4 (time 1-4) but though this simplifies

things, it raises questions about whether it is appropriate

and whether it answers the research questions you

really want answered. However, such a design would

tell you if you have a significant 2-way interaction; if

you do not, analysis of components of the interaction

would need justification.

 

(5) Instead of taking the ANOVA route, you could

specific the analysis you want as planned comparisons

and calculate the appropriate error terms for such

comparisons.  But you would have to think such an

analysis through.  Additional considerations have to do

with whether you sphericity or not -- in which case

what is the nature of your variance-covariance matrix.

 

(6) I could be wrong but I think most statistical consultants

would suggest that you use a multilevel model to analyze

the design above, one reason being that it would allow

you to make use of all of the data (potentially).  There

are people on SPSSX-L who are familiar with multilevel

analysis for these types of designs (though a 2-way

incomplete within-subject design is a little odd, at least

in the present form -- can the empty cells be treated as

"missing at random"?).

 

(7) If you want to take the multilevel route, these is some

literature on incomplete within-subject or incomplete

repeated measures ANOVA (you can do a search on

scholar.google.com for the literature).  One source you

might want to look at is the work by Lesa Hoffman,

for example, see:

 

Hoffman L, Rovine MJ. Multilevel models for experimental

psychologists: Foundations and illustrative examples. Behavior

Research Methods. 2007;39:101–117.

 

This article is on the University of Nebraska-Lincoln's UNL)

digital commons and can be accessed here:

 

At the time of the writing of this article, Hoffman was faculty

at UNL but she has apparently changed affiliations and here is

a link to her webpage:

 

You can examine these sources and the literature in working out

what analysis you want to do.  Some on SPSS may be able to

help with SPSS syntax for the analysis (Hoffman provided some

examples of SPSS analysis but they are no longer on the UNL

website though her article provides an address there; it may be

on her new site).

 

(8) Remember that you have a completely within-subject design

but many examples will be "mixed designs" (i.e., have between-subject

and within-subject ind vars; another definition of "mixed design" is

that some variables are "fixed effects" and other are "random effects",

a distinction you will have to make if you go the multilevel analysis

route).

 

HTH.

 

-Mike Palij

New York University

 

 

---- Original Message -----

Sent: Wednesday, April 29, 2015 6:44 AM

Subject: Separate One-Way vs Two-Way Repeated Measures

 

Hello everyone,



I'm struggling with a Within-Subjects analysis and I haven't found the answer to my doubts, so here it goes:

I have two IV (Time and Voice Conditions) and one DV (continuous variable).

Time conditions depend on the Time to Complete the task
Voice conditions refer to different voices used

                                                           Time Condition

                                         1         2          3          4          5         6
                             1          x          x          x          x

Voice Conditions     2          x          x          x          x          x          x

                             3                      x          x          x          x          x

This table shows all combinations Time X Voice that were tested, by all participants (30). For example, Time condition 1 with Voice condition 3 was not executed by any participant.

What I want with this test is to obtain the following comparisons:
- Within each Time Condition, check for differences in Voice conditions
- For each voice condition, check for differences in time condition (in particular, I want to know when does performance start to deteriorate significantly).

My first thoughts were to perform one-way repeated measures ANOVA (or Friedman due to normality violation) within each time condition; and then within each voice condition (Followed by post-hocs).

However, I found that I could analyze these two variables together with the two-way repeated measures ANOVA. I performed this analysis using the General Linear Model -> Repeated Measures menu. Yet, I found the following problems:

- I have a few combinations that don't exist (for example Time 1 X Voice 3); If I use empty values, I get no results for all the analysis... but I have to insert one variable in order to proceed with the analysis. What should I do?
- When checking for normality, almost all TimeXVoice conditions violate the assumption. I read that normality is not that strict in this test, but some of the conditions show Sig results of ,000 and almost all of them are below ,050. Should I go for a non-parametric alternative?

Is it OK to use the one-way version and analyze each of the variables separately or should I do the two-way analysis? How should I overcome the problems referred above? Should I consider a different test?

I hope I have provided enough details.
Thanks in advance,

 

--

João Guerreiro
- Researcher @  INESC-ID / Visualization and Intelligent MultiModal Interfaces Group
- PhD Student @ IST / Technical University of Lisbon
- Tel: <A href="tel:%2B351.21.4233532" target=_blank value="+351214233532">+351.21.4233532
- web: http://web.ist.utl.pt/joao.p.guerreiro/

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



--
João Guerreiro
- Researcher @  INESC-ID / Visualization and Intelligent MultiModal Interfaces Group
- PhD Student @ IST / Technical University of Lisbon
- Tel: +351.21.4233532
- web: http://web.ist.utl.pt/joao.p.guerreiro/
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Free forum by Nabble Edit this page