MANOVA and multiple comp correction

[Zdaniuk, Bozena-2]

Hello, could someone check my thinking?
I have five measures which are considered related indicators of well-being. I have 2 groups.
I run MANOVA comparing 2 groups on all 5 measures.
If I find the omnibus F significant, I can then report Univariate F-tests at .05 level of significance without having to correct for multiple comparisons.
Am I correct?
Thanks so much in advance.
Bozena Zdaniuk

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Swank, Paul R]

Technically, no. You still need to control for the individual tests
alpha level or you will inflate the type I error rate for those tests.
Of course, most people go ahead and do it any way.

Paul R. Swank, Ph.D
Professor and Director of Research
Children's Learning Institute
University of Texas Health Science Center
Houston, TX 77038


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Zdaniuk, Bozena
Sent: Friday, October 10, 2008 8:38 AM
To: [hidden email]
Subject: MANOVA and multiple comp correction

Hello, could someone check my thinking?
I have five measures which are considered related indicators of
well-being. I have 2 groups.
I run MANOVA comparing 2 groups on all 5 measures.
If I find the omnibus F significant, I can then report Univariate
F-tests at .05 level of significance without having to correct for
multiple comparisons.
Am I correct?
Thanks so much in advance.
Bozena Zdaniuk

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[SR Millis-3]

It is common for many investigators to begin an analysis with a MANOVA and then follow up with a series of ANOVAs if the MANOVA is statistically significant – in the mistaken belief that this procedure controls for Type I error.  Actually, the alpha level for each subsequent ANOVA is less than the alpha for the MANOVA only when the MANOVA null hypothesis is true – which is precisely when investigators do not adopt this strategy (Huberty & Morris, 1989, Psych Bull, 105, 302-308).  Following up a significant MANOVA with a series of univariate tests results in a loss of power and an increase in Type I error.

The primary reason to perform a MANOVA is when the investigator is interested in determining whether groups differ on a linear composite set of measures – typically “…a collection of conceptually interrelated variables that, at least potentially, determines one of more underlying variates or constructs” (Huberty & Morris, 1989, p. 304). If so, canonical variates are created and group differences are interpreted in terms of these canonical variates.  Multicollinearity is addressed and complex relationships among dependent variables can be examined.  One doesn’t need to perform any univariate ANOVAs – that would defeat the purpose of doing the MANOVA in the first place.  Hence, if you are interested in examining group differences in terms of a multivariate composite(s) composed of the dependent variables, you need to examine and describe the canonical variates derived from your MANOVA – and avoid doing the univariate analyses.

If, on the other, you're interested in simply examining on which dependent variables that their groups differ, univariate t-tests are appropriate.  In this case, there is no need to perform any MANOVA.  Type I error can be addressed by apportioning the alpha at the level of families of hypotheses (Dar et al., 1994, J Consult Clin Psych, 62, 75-82).

Scott R Millis, PhD, MEd, ABPP (CN,CL,RP), CStat
Professor & Director of Research
Dept of Physical Medicine & Rehabilitation
Wayne State University School of Medicine
261 Mack Blvd
Detroit, MI 48201
Email:  [hidden email]
Tel: [phone]
Fax: [phone]


--- On Fri, 10/10/08, Zdaniuk, Bozena <[hidden email]> wrote:

> From: Zdaniuk, Bozena <[hidden email]>
> Subject: MANOVA and multiple comp correction
> To: [hidden email]
> Date: Friday, October 10, 2008, 9:38 AM
> Hello, could someone check my thinking?
> I have five measures which are considered related
> indicators of well-being. I have 2 groups.
> I run MANOVA comparing 2 groups on all 5 measures.
> If I find the omnibus F significant, I can then report
> Univariate F-tests at .05 level of significance without
> having to correct for multiple comparisons.
> Am I correct?
> Thanks so much in advance.
> Bozena Zdaniuk
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body
> text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the
> command
> INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD