SPSSX Discussion

Repeated measures analysis

Classic

List

Threaded

5 messages Options

S Elgie

Repeated measures analysis

My clients have carried out a careful treatment evaluation study. One of
the groups received a battery of assessment tests at time 1, then were on a
waiting list for about 16 weeks. At time 2 at the beginning of treatment
they received the identical battery of tests and then at time 3 after 16
weeks of treatment they received the battery of tests again.

I have been asked to provide an analysis which has supporting statistical
tests which would prove that the change from time 2 to time 3 (pre- to post-
treatment) was greater than the change from time 1 to time 2 (pre- to
post-wait list). The usual analysis of repeated measures would not answer
this exact question and so with some doubt in my mind I formed change scores
by subtracting time 2 scores from time 1 and time 3 scores from time 2. The
results of a repeated measures analysis on these scores were quite
significant and reasonable and I wrote them up. I might mention that this
is a sub-analysis rather than the principal analysis.

A reviewer has criticized, suggesting that I divide the time 1 and 2 scores
by the time 2 SD and the time 2 and 3 scores by the time 3 SD prior to
forming the difference scores. This suggestion is not sitting well with me,
and I thought to ask your advice as to other possible analyses which would
answer the question posed.

With thanks,

Susan

Susan Elgie
QQ Consulting
Toronto Canada
[hidden email]

statisticsdoc

Re: Repeated measures analysis

Stephen Brand
www.statisticsdoc.com

Susan,

I think that this is a question of carrying out a trend analysis across
time. The issue is whether the difference between time points is linear, or
whether a curvilinear effect is also needed to fit the data (a particular
type of curvilinear trend in which the change from T2 to T3 is greater than
the linear slope between time points). I imagine that you have plotted the
means of the measures at each time point, and that what you are looking for
is a formal statistical procedure for estimating the strength and
significance of the curvilinear trend).

SPSS provides the tools you need to test the within-subject trends. GLM
provides the tools to test the significance of the quadratic trend. You
might also want to consider using the TEST option within the MIXED command
in SPSS to examine the shape of the within-subject trends over time.

HTH,

Stephen Brand

For personalized and professional consultation in statistics and research
design, visit
www.statisticsdoc.com

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
S Elgie
Sent: Friday, November 17, 2006 12:52 PM
To: [hidden email]
Subject: Repeated measures analysis

My clients have carried out a careful treatment evaluation study. One of
the groups received a battery of assessment tests at time 1, then were on a
waiting list for about 16 weeks. At time 2 at the beginning of treatment
they received the identical battery of tests and then at time 3 after 16
weeks of treatment they received the battery of tests again.

I have been asked to provide an analysis which has supporting statistical
tests which would prove that the change from time 2 to time 3 (pre- to post-
treatment) was greater than the change from time 1 to time 2 (pre- to
post-wait list). The usual analysis of repeated measures would not answer
this exact question and so with some doubt in my mind I formed change scores
by subtracting time 2 scores from time 1 and time 3 scores from time 2. The
results of a repeated measures analysis on these scores were quite
significant and reasonable and I wrote them up. I might mention that this
is a sub-analysis rather than the principal analysis.

A reviewer has criticized, suggesting that I divide the time 1 and 2 scores
by the time 2 SD and the time 2 and 3 scores by the time 3 SD prior to
forming the difference scores. This suggestion is not sitting well with me,
and I thought to ask your advice as to other possible analyses which would
answer the question posed.

With thanks,

Susan

Susan Elgie
QQ Consulting
Toronto Canada
[hidden email]

S Elgie

Re: Repeated measures analysis

Thank you Stephen,
A trend analysis is a very good idea. The publication is aimed at a
nontechnical audience however, and I do find that the simple word
'quadratic' may cause considerable anxiety with some people (or they simply
skip over that part.) I had another idea, on which I thought to ask your
opinion. What if I do a repeated measures ANOVA with repeated contrasts (t1
vs t2 and t2 vs t3) accompanied by Cohen d statistics for EACH contrast? In
fact there is significant change during the waitlist on several measures due
to people's natural improvement and perhaps regression to the mean.
However, it is trivial compared to the change during the treatment period.
The results of the contrasts and the associated d statistics would show the
relative magnitudes of the change during the two periods although there
would be no direct statistical test of the difference.

Thanks again for your input on this,
Susan

>From: Statisticsdoc <[hidden email]>
>Reply-To: Statisticsdoc <[hidden email]>
>To: [hidden email]
>Subject: Re: Repeated measures analysis
>Date: Fri, 17 Nov 2006 22:12:05 -0500
>
>Stephen Brand
>www.statisticsdoc.com
>
>Susan,
>
>I think that this is a question of carrying out a trend analysis across
>time. The issue is whether the difference between time points is linear,
>or
>whether a curvilinear effect is also needed to fit the data (a particular
>type of curvilinear trend in which the change from T2 to T3 is greater than
>the linear slope between time points). I imagine that you have plotted the
>means of the measures at each time point, and that what you are looking for
>is a formal statistical procedure for estimating the strength and
>significance of the curvilinear trend).
>
>SPSS provides the tools you need to test the within-subject trends. GLM
>provides the tools to test the significance of the quadratic trend. You
>might also want to consider using the TEST option within the MIXED command
>in SPSS to examine the shape of the within-subject trends over time.
>
>HTH,
>
>Stephen Brand
>
>For personalized and professional consultation in statistics and research
>design, visit
>www.statisticsdoc.com
>
>
>-----Original Message-----
>From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
>S Elgie
>Sent: Friday, November 17, 2006 12:52 PM
>To: [hidden email]
>Subject: Repeated measures analysis
>
>
>My clients have carried out a careful treatment evaluation study. One of
>the groups received a battery of assessment tests at time 1, then were on a
>waiting list for about 16 weeks. At time 2 at the beginning of treatment
>they received the identical battery of tests and then at time 3 after 16
>weeks of treatment they received the battery of tests again.
>
>I have been asked to provide an analysis which has supporting statistical
>tests which would prove that the change from time 2 to time 3 (pre- to
>post-
>treatment) was greater than the change from time 1 to time 2 (pre- to
>post-wait list). The usual analysis of repeated measures would not answer
>this exact question and so with some doubt in my mind I formed change
>scores
>by subtracting time 2 scores from time 1 and time 3 scores from time 2.
>The
>results of a repeated measures analysis on these scores were quite
>significant and reasonable and I wrote them up. I might mention that this
>is a sub-analysis rather than the principal analysis.
>
>A reviewer has criticized, suggesting that I divide the time 1 and 2 scores
>by the time 2 SD and the time 2 and 3 scores by the time 3 SD prior to
>forming the difference scores. This suggestion is not sitting well with
>me,
>and I thought to ask your advice as to other possible analyses which would
>answer the question posed.
>
>With thanks,
>
>Susan
>
>Susan Elgie
>QQ Consulting
>Toronto Canada
>[hidden email]

statisticsdoc

Re: Repeated measures analysis

Stephen Brand
www.statisticsdoc.com

Susan,

For expository purposes, giving the reader Cohen's d for each contrast would
do very well, I think. I would also suggest including a graph of the means
for each point. Sometimes trends are easier to see and to show than they
are to write about.

You may want to include a footnote stating that the quadratic trend was
significant. The reader can skip the footnote without losing the point, and
the more statistically-minded among us will note that you tested the effect.

HTH,

Stephen Brand

For personalized and professional consultation in statistics and research
design, visit
www.statisticsdoc.com

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
S Elgie
Sent: Sunday, November 19, 2006 2:58 PM
To: [hidden email]
Subject: Re: Repeated measures analysis

Thank you Stephen,
A trend analysis is a very good idea. The publication is aimed at a
nontechnical audience however, and I do find that the simple word
'quadratic' may cause considerable anxiety with some people (or they simply
skip over that part.) I had another idea, on which I thought to ask your
opinion. What if I do a repeated measures ANOVA with repeated contrasts (t1
vs t2 and t2 vs t3) accompanied by Cohen d statistics for EACH contrast? In
fact there is significant change during the waitlist on several measures due
to people's natural improvement and perhaps regression to the mean.
However, it is trivial compared to the change during the treatment period.
The results of the contrasts and the associated d statistics would show the
relative magnitudes of the change during the two periods although there
would be no direct statistical test of the difference.

Thanks again for your input on this,
Susan

paulandpen

Re: Repeated measures analysis

In reply to this post by S Elgie

Susan,

The recommendations of the examiner do not sit well with me either. I cannot see the rationale for using the SDs of one period only, as opposed to using combined variance measures over time, it makes no sense.

At the university of texas, from memory they have keppels methodology applied using spss manova and GLM repeated measures for breaking down time and change information in the form of contrasts. I think your idea is better, and for a non-technical audies, putting them in a ratio format would provide a nice visual display of the differences.

Regards Paul

> S Elgie <[hidden email]> wrote:
>
> Thank you Stephen,
> A trend analysis is a very good idea. The publication is aimed at a
> nontechnical audience however, and I do find that the simple word
> 'quadratic' may cause considerable anxiety with some people (or they
> simply
> skip over that part.) I had another idea, on which I thought to ask
> your
> opinion. What if I do a repeated measures ANOVA with repeated contrasts
> (t1
> vs t2 and t2 vs t3) accompanied by Cohen d statistics for EACH contrast?
> In
> fact there is significant change during the waitlist on several measures
> due
> to people's natural improvement and perhaps regression to the mean.
> However, it is trivial compared to the change during the treatment
> period.
> The results of the contrasts and the associated d statistics would show
> the
> relative magnitudes of the change during the two periods although there
> would be no direct statistical test of the difference.
>
> Thanks again for your input on this,
> Susan
>
> >From: Statisticsdoc <[hidden email]>
> >Reply-To: Statisticsdoc <[hidden email]>
> >To: [hidden email]
> >Subject: Re: Repeated measures analysis
> >Date: Fri, 17 Nov 2006 22:12:05 -0500
> >
> >Stephen Brand
> >www.statisticsdoc.com
> >
> >Susan,
> >
> >I think that this is a question of carrying out a trend analysis across
> >time. The issue is whether the difference between time points is
> linear,
> >or
> >whether a curvilinear effect is also needed to fit the data (a
> particular
> >type of curvilinear trend in which the change from T2 to T3 is greater
> than
> >the linear slope between time points). I imagine that you have plotted
> the
> >means of the measures at each time point, and that what you are looking
> for
> >is a formal statistical procedure for estimating the strength and
> >significance of the curvilinear trend).
> >
> >SPSS provides the tools you need to test the within-subject trends.
> GLM
> >provides the tools to test the significance of the quadratic trend.
> You
> >might also want to consider using the TEST option within the MIXED
> command
> >in SPSS to examine the shape of the within-subject trends over time.
> >
> >HTH,
> >
> >Stephen Brand
> >
> >For personalized and professional consultation in statistics and
> research
> >design, visit
> >www.statisticsdoc.com
> >
> >
> >-----Original Message-----
> >From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
> >S Elgie
> >Sent: Friday, November 17, 2006 12:52 PM
> >To: [hidden email]
> >Subject: Repeated measures analysis
> >
> >
> >My clients have carried out a careful treatment evaluation study. One
> of
> >the groups received a battery of assessment tests at time 1, then were
> on a
> >waiting list for about 16 weeks. At time 2 at the beginning of
> treatment
> >they received the identical battery of tests and then at time 3 after
> 16
> >weeks of treatment they received the battery of tests again.
> >
> >I have been asked to provide an analysis which has supporting
> statistical
> >tests which would prove that the change from time 2 to time 3 (pre- to
> >post-
> >treatment) was greater than the change from time 1 to time 2 (pre- to
> >post-wait list). The usual analysis of repeated measures would not
> answer
> >this exact question and so with some doubt in my mind I formed change
> >scores
> >by subtracting time 2 scores from time 1 and time 3 scores from time 2.
> >The
> >results of a repeated measures analysis on these scores were quite
> >significant and reasonable and I wrote them up. I might mention that
> this
> >is a sub-analysis rather than the principal analysis.
> >
> >A reviewer has criticized, suggesting that I divide the time 1 and 2
> scores
> >by the time 2 SD and the time 2 and 3 scores by the time 3 SD prior to
> >forming the difference scores. This suggestion is not sitting well
> with
> >me,
> >and I thought to ask your advice as to other possible analyses which
> would
> >answer the question posed.
> >
> >With thanks,
> >
> >Susan
> >
> >Susan Elgie
> >QQ Consulting
> >Toronto Canada
> >[hidden email]