Whether to use post hok tests, or discriminant function analysis after a one-way significant MANOVA. (Non-SPSS question).

[Faiz Rasool]

Hi all,

 

My apologies for a Non-SPSS question.

 

My research question is whether females rate three environmental problems more serious as compare to males. Survey participants were given a list of three environmental problems and they were requested to rate each of the problem on a 4 points scale (not a problem, small problem, somewhat serious problem, very serious problem).

 

I performed a one-way MANOVA using gender as an IV and the responses   to the three questions as three DVs. MANOVA gives a significant result (P < .000).

 

I did consider to use  these three questions as a composite scale, but the cronbach alpha statistic was .46, so I desided to use these three questions as three DVS.

 

I’m slightly confused on what is the best approach to follow after a significant MANOVA. My research question can be answered by the ANOVA results that SPSS provides in a MANOVA output. However, I’m confused after reading discovering statistics using SPSS by Field who recommends that MANOVA should always be followed by discriminant function analysis.  I’ve searched the internet using Google, and have tried to read few published studies that have used MANOVA. Unfortunately, I cannot decide what is the best approach using one-way ANOVAs or post hok tests, or discriminant function analysis after a significant MANOVA.

 

I thank you for your help.

 

I recognize that this is a non-SPSS question   so if list members can direct  me to any list which is a more suitable forum for  this types of questions, I shall be very thankful.

 

Regards,

Faiz.

[Bruce Weaver]

I suggest you look at the 1989 article by Huberty & Morris.  A PDF can be downloaded here:

https://bitbucket.org/eyecat/readinglists/src/d8e8010f0b0d2dbb0863af3050411695254cc6b1/ReadingList_NotreDame/HubertyMorris1989MultivariateVsUnivariate.pdf

See also this thread from a few years ago.

http://spssx-discussion.1045642.n5.nabble.com/Using-gender-as-an-outcome-variable-in-logistic-regression-tp1086376p1086390.html

HTH.

Faiz Rasool wrote

Hi all,



My apologies for a Non-SPSS question.



My research question is whether females rate three environmental problems
more serious as compare to males. Survey participants were given a list of
three environmental problems and they were requested to rate each of the
problem on a 4 points scale (not a problem, small problem, somewhat serious
problem, very serious problem).



I performed a one-way MANOVA using gender as an IV and the responses   to
the three questions as three DVs. MANOVA gives a significant result (P <
.000).



I did consider to use  these three questions as a composite scale, but the
cronbach alpha statistic was .46, so I desided to use these three questions
as three DVS.



I'm slightly confused on what is the best approach to follow after a
significant MANOVA. My research question can be answered by the ANOVA
results that SPSS provides in a MANOVA output. However, I'm confused after
reading discovering statistics using SPSS by Field who recommends that
MANOVA should always be followed by discriminant function analysis.  I've
searched the internet using Google, and have tried to read few published
studies that have used MANOVA. Unfortunately, I cannot decide what is the
best approach using one-way ANOVAs or post hok tests, or discriminant
function analysis after a significant MANOVA.



I thank you for your help.



I recognize that this is a non-SPSS question   so if list members can direct
me to any list which is a more suitable forum for  this types of questions,
I shall be very thankful.



Regards,

Faiz.

[Rich Ulrich]

MANOVA gives an overall test for patterns of differences.  One-way Manova
differs from "discriminant function" procedures only in the choice of ancillary
statistics that are displayed.  Two-group d.f.  gives exactly the same tests
as ordinary multiple regression with a 0/1 criterion (though the coefficients
are standardized differently).  Two-group d.f.  is sometimes replaced by logistic
regression, which is mainly a bad idea when the sample sizes are too small.

And here is how you apply Huberty and Morris to the present problem.

The stated hypothesis is not about a pattern difference, which is what is
tested by MANOVA  (or discriminant function, for the one-way case) and
by logistic regression.  "Do females rate [problems as] more serious?"

That calls, directly, for testing the simple composite score: the average
of the three problems.  If you want a test of whether the effects differ by
problems, you could set up the three parallel variables as Repeated Measures,
and look at the interaction test as a modifier of the conclusion from the main-
effects test. 

Cronbach's alpha coefficient is not appropriate as a rule-out.  Ideally, in
fact, at times like this, you might *prefer*  for the parts of a composite score
to be rather independent and distinct.  A high "internal reliability" for Cronbach's
alpha indicates that you have a narrow subject; you do not have a "broad validity"
that comes from testing independent elements.  Guilford's old texts include a
discussion between the trade-off (in this sense) between reliability (narrow subject)
and validity (generalization).

--
Rich Ulrich

> Date: Sun, 26 May 2013 07:34:04 -0700

> From: [hidden email]
> Subject: Re: Whether to use post hok tests, or discriminant function analysis after a one-way significant MANOVA. (Non-SPSS question).
> To: [hidden email]
>
> I suggest you look at the 1989 article by Huberty & Morris. A PDF can be
> downloaded here:
>
> https://bitbucket.org/eyecat/readinglists/src/d8e8010f0b0d2dbb0863af3050411695254cc6b1/ReadingList_NotreDame/HubertyMorris1989MultivariateVsUnivariate.pdf
>
> See also this thread from a few years ago.
>
> http://spssx-discussion.1045642.n5.nabble.com/Using-gender-as-an-outcome-variable-in-logistic-regression-tp1086376p1086390.html
>
> HTH.
>
>
>
> Faiz Rasool wrote
> > Hi all,
> >
> >
> > My apologies for a Non-SPSS question.
> >
> > My research question is whether females rate three environmental problems
> > more serious as compare to males. Survey participants were given a list of
> > three environmental problems and they were requested to rate each of the
> > problem on a 4 points scale (not a problem, small problem, somewhat
> > serious
> > problem, very serious problem).
> >
> > I performed a one-way MANOVA using gender as an IV and the responses to
> > the three questions as three DVs. MANOVA gives a significant result (P <
> > .000).
> >
> > I did consider to use these three questions as a composite scale, but the
> > cronbach alpha statistic was .46, so I desided to use these three
> > questions
> > as three DVS.
> >
> >
> > I'm slightly confused on what is the best approach to follow after a
> > significant MANOVA. My research question can be answered by the ANOVA
> > results that SPSS provides in a MANOVA output. However, I'm confused after
> > reading discovering statistics using SPSS by Field who recommends that
> > MANOVA should always be followed by discriminant function analysis. I've
> > searched the internet using Google, and have tried to read few published
> > studies that have used MANOVA. Unfortunately, I cannot decide what is the
> > best approach using one-way ANOVAs or post hok tests, or discriminant
> > function analysis after a significant MANOVA.
> >
> >
> >
> >[snip closing; tags]...

[Florio Arguillas]

I will be out of the office from May 27 until June 8, so e-mail access will be intermittent.  Apologies for any delay in reply.

[Bruce Weaver]

Rich raises a good point about sample size & logistic regression.  I meant to ask about the sample sizes for the two groups.  To avoid over-fitting, you should have 15-20 in the smaller of the two groups for each model parameter.

Rich Ulrich-2 wrote

MANOVA gives an overall test for patterns of differences.  One-way Manova
differs from "discriminant function" procedures only in the choice of ancillary
statistics that are displayed.  Two-group d.f.  gives exactly the same tests
as ordinary multiple regression with a 0/1 criterion (though the coefficients
are standardized differently).  Two-group d.f.  is sometimes replaced by logistic
regression, which is mainly a bad idea when the sample sizes are too small.

And here is how you apply Huberty and Morris to the present problem.

The stated hypothesis is not about a pattern difference, which is what is
tested by MANOVA  (or discriminant function, for the one-way case) and
by logistic regression.  "Do females rate [problems as] more serious?"

That calls, directly, for testing the simple composite score: the average
of the three problems.  If you want a test of whether the effects differ by
problems, you could set up the three parallel variables as Repeated Measures,
and look at the interaction test as a modifier of the conclusion from the main-
effects test.  

Cronbach's alpha coefficient is not appropriate as a rule-out.  Ideally, in
fact, at times like this, you might *prefer*  for the parts of a composite score
to be rather independent and distinct.  A high "internal reliability" for Cronbach's
alpha indicates that you have a narrow subject; you do not have a "broad validity"
that comes from testing independent elements.  Guilford's old texts include a
discussion between the trade-off (in this sense) between reliability (narrow subject)
and validity (generalization).

--
Rich Ulrich


> Date: Sun, 26 May 2013 07:34:04 -0700
> From: [hidden email]
> Subject: Re: Whether to use post  hok tests,              or discriminant  function analysis after a one-way significant              MANOVA. (Non-SPSS question).
> To: [hidden email]
>
> I suggest you look at the 1989 article by Huberty & Morris.  A PDF can be
> downloaded here:
>
> https://bitbucket.org/eyecat/readinglists/src/d8e8010f0b0d2dbb0863af3050411695254cc6b1/ReadingList_NotreDame/HubertyMorris1989MultivariateVsUnivariate.pdf
>
> See also this thread from a few years ago.
>
> http://spssx-discussion.1045642.n5.nabble.com/Using-gender-as-an-outcome-variable-in-logistic-regression-tp1086376p1086390.html
>
> HTH.
>
>
>
> Faiz Rasool wrote
> > Hi all,
> >
> >
> > My apologies for a Non-SPSS question.
> >
> > My research question is whether females rate three environmental problems
> > more serious as compare to males. Survey participants were given a list of
> > three environmental problems and they were requested to rate each of the
> > problem on a 4 points scale (not a problem, small problem, somewhat
> > serious
> > problem, very serious problem).
> >
> > I performed a one-way MANOVA using gender as an IV and the responses   to
> > the three questions as three DVs. MANOVA gives a significant result (P <
> > .000).
> >
> > I did consider to use  these three questions as a composite scale, but the
> > cronbach alpha statistic was .46, so I desided to use these three
> > questions
> > as three DVS.
> >
> >
> > I'm slightly confused on what is the best approach to follow after a
> > significant MANOVA. My research question can be answered by the ANOVA
> > results that SPSS provides in a MANOVA output. However, I'm confused after
> > reading discovering statistics using SPSS by Field who recommends that
> > MANOVA should always be followed by discriminant function analysis.  I've
> > searched the internet using Google, and have tried to read few published
> > studies that have used MANOVA. Unfortunately, I cannot decide what is the
> > best approach using one-way ANOVAs or post hok tests, or discriminant
> > function analysis after a significant MANOVA.
> >
> >
> >
> >[snip closing; tags]...

[Faiz Rasool]

Thank you, Bruce and Rich, for such helpful replies, I appreciate it.

The sample size for the variables I'm trying to analyze is 1469.  The group
sizes are unequal. The smallest   group has 53 cases, few groups have around
100 and other groups have around 200 cases.

What I've been able to understand from the responses is (1) MANOVA may not
be the best choice considering the research questions I'm trying to answer.
(2) creating a composite score of the ratings of three questions and using
the scores as the dependent variable in any subsequent analysis can be a
better approach.  (3) Cronbach alpha of .40 can be acceptable under some
circumstances.

Please feel free to correct me if my understanding is incorrect.

I've the following questions in my mind.

The concern about these problems in my country has not been studied before
(as far as I know).  So would MANOVA be appropriate based on the fact that I
cannot draw on the literature pertaining to my country to hypothesize how
the variables are likely to be related. I should clarify that literature
about concern for these problems in other countries is available, and my
research question mentioned in the earlier email is based on that
literature.

Any comments on what can be the appropriate analytical strategy will be most
appreciated.

Thanks again,
Faiz.

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Bruce Weaver
Sent: Monday, May 27, 2013 4:43 AM
To: [hidden email]
Subject: Re: Whether to use post hok tests, or discriminant function
analysis after a one-way significant MANOVA. (Non-SPSS question).

Rich raises a good point about sample size & logistic regression.  I meant
to ask about the sample sizes for the two groups.  To avoid over-fitting,
you should have 15-20 in the smaller of the two groups for each model
parameter.




Rich Ulrich-2 wrote

> MANOVA gives an overall test for patterns of differences.  One-way
> Manova differs from "discriminant function" procedures only in the
> choice of ancillary statistics that are displayed.  Two-group d.f.
> gives exactly the same tests as ordinary multiple regression with a
> 0/1 criterion (though the coefficients are standardized differently).
> Two-group d.f.  is sometimes replaced by logistic regression, which is
> mainly a bad idea when the sample sizes are too small.
>
> And here is how you apply Huberty and Morris to the present problem.
>
> The stated hypothesis is not about a pattern difference, which is what
> is tested by MANOVA  (or discriminant function, for the one-way case)
> and by logistic regression.  "Do females rate [problems as] more serious?"
>
> That calls, directly, for testing the simple composite score: the
> average of the three problems.  If you want a test of whether the
> effects differ by problems, you could set up the three parallel
> variables as Repeated Measures, and look at the interaction test as a
> modifier of the conclusion from the
> main-
> effects test.
>
> Cronbach's alpha coefficient is not appropriate as a rule-out.
> Ideally, in fact, at times like this, you might *prefer*  for the
> parts of a composite score to be rather independent and distinct.  A
> high "internal reliability" for Cronbach's alpha indicates that you
> have a narrow subject; you do not have a "broad validity"
> that comes from testing independent elements.  Guilford's old texts
> include a discussion between the trade-off (in this sense) between
> reliability (narrow subject) and validity (generalization).
>
> --
> Rich Ulrich
>
>
>> Date: Sun, 26 May 2013 07:34:04 -0700
>> From:

> bruce.weaver@

>> Subject: Re: Whether to use post  hok tests,              or discriminant
>> function analysis after a one-way significant              MANOVA.
>> (Non-SPSS question).
>> To:

> SPSSX-L@.UGA

>>
>> I suggest you look at the 1989 article by Huberty & Morris.  A PDF
>> can be downloaded here:
>>
>> https://bitbucket.org/eyecat/readinglists/src/d8e8010f0b0d2dbb0863af3
>> 050411695254cc6b1/ReadingList_NotreDame/HubertyMorris1989Multivariate
>> VsUnivariate.pdf
>>
>> See also this thread from a few years ago.
>>
>> http://spssx-discussion.1045642.n5.nabble.com/Using-gender-as-an-outc
>> ome-variable-in-logistic-regression-tp1086376p1086390.html
>>
>> HTH.
>>
>>
>>
>> Faiz Rasool wrote
>> > Hi all,
>> >
>> >
>> > My apologies for a Non-SPSS question.
>> >
>> > My research question is whether females rate three environmental
>> problems
>> > more serious as compare to males. Survey participants were given a
>> > list
>> of
>> > three environmental problems and they were requested to rate each
>> > of
>> the
>> > problem on a 4 points scale (not a problem, small problem, somewhat
>> > serious problem, very serious problem).
>> >
>> > I performed a one-way MANOVA using gender as an IV and the
>> > responses
>> to
>> > the three questions as three DVs. MANOVA gives a significant result
>> > (P
>> <
>> > .000).
>> >
>> > I did consider to use  these three questions as a composite scale,
>> > but
>> the
>> > cronbach alpha statistic was .46, so I desided to use these three
>> > questions as three DVS.
>> >
>> >
>> > I'm slightly confused on what is the best approach to follow after
>> > a significant MANOVA. My research question can be answered by the
>> > ANOVA results that SPSS provides in a MANOVA output. However, I'm
>> > confused
>> after
>> > reading discovering statistics using SPSS by Field who recommends
>> > that MANOVA should always be followed by discriminant function

analysis.

>> I've
>> > searched the internet using Google, and have tried to read few
>> published
>> > studies that have used MANOVA. Unfortunately, I cannot decide what
>> > is
>> the
>> > best approach using one-way ANOVAs or post hok tests, or
>> > discriminant function analysis after a significant MANOVA.
>> >
>> >
>> >
>> >[snip closing; tags]...

-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/Whether-to-use-post-hok-tests-
or-discriminant-function-analysis-after-a-one-way-significant-MANOVA-N-tp572
0417p5720421.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command SIGNOFF SPSSX-L For a list of
commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

Don't forget Rich's suggestion about having Question as a repeated measures factor with 3 levels.

"If you want a test of whether the effects differ by problems, you could set up the three parallel variables as Repeated Measures, and look at the interaction test as a modifier of the conclusion from the main-
effects test."

I.e., Sex is a between Ss factor, Question a repeated measures factor, and the model includes the Sex * Question interaction term.  An appropriate EMMEANS sub-command with COMPARE will give you the Male v Female contrasts for each questions separately.  (You may want to look at those in the event of a significant interaction.)

Faiz Rasool wrote

Thank you, Bruce and Rich, for such helpful replies, I appreciate it.

The sample size for the variables I'm trying to analyze is 1469.  The group
sizes are unequal. The smallest   group has 53 cases, few groups have around
100 and other groups have around 200 cases.

What I've been able to understand from the responses is (1) MANOVA may not
be the best choice considering the research questions I'm trying to answer.
(2) creating a composite score of the ratings of three questions and using
the scores as the dependent variable in any subsequent analysis can be a
better approach.  (3) Cronbach alpha of .40 can be acceptable under some
circumstances.

Please feel free to correct me if my understanding is incorrect.

I've the following questions in my mind.

The concern about these problems in my country has not been studied before
(as far as I know).  So would MANOVA be appropriate based on the fact that I
cannot draw on the literature pertaining to my country to hypothesize how
the variables are likely to be related. I should clarify that literature
about concern for these problems in other countries is available, and my
research question mentioned in the earlier email is based on that
literature.

Any comments on what can be the appropriate analytical strategy will be most
appreciated.

Thanks again,
Faiz.

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Bruce Weaver
Sent: Monday, May 27, 2013 4:43 AM
To: [hidden email]
Subject: Re: Whether to use post hok tests, or discriminant function
analysis after a one-way significant MANOVA. (Non-SPSS question).

Rich raises a good point about sample size & logistic regression.  I meant
to ask about the sample sizes for the two groups.  To avoid over-fitting,
you should have 15-20 in the smaller of the two groups for each model
parameter.




Rich Ulrich-2 wrote
> MANOVA gives an overall test for patterns of differences.  One-way
> Manova differs from "discriminant function" procedures only in the
> choice of ancillary statistics that are displayed.  Two-group d.f.
> gives exactly the same tests as ordinary multiple regression with a
> 0/1 criterion (though the coefficients are standardized differently).
> Two-group d.f.  is sometimes replaced by logistic regression, which is
> mainly a bad idea when the sample sizes are too small.
>
> And here is how you apply Huberty and Morris to the present problem.
>
> The stated hypothesis is not about a pattern difference, which is what
> is tested by MANOVA  (or discriminant function, for the one-way case)
> and by logistic regression.  "Do females rate [problems as] more serious?"
>
> That calls, directly, for testing the simple composite score: the
> average of the three problems.  If you want a test of whether the
> effects differ by problems, you could set up the three parallel
> variables as Repeated Measures, and look at the interaction test as a
> modifier of the conclusion from the
> main-
> effects test.
>
> Cronbach's alpha coefficient is not appropriate as a rule-out.
> Ideally, in fact, at times like this, you might *prefer*  for the
> parts of a composite score to be rather independent and distinct.  A
> high "internal reliability" for Cronbach's alpha indicates that you
> have a narrow subject; you do not have a "broad validity"
> that comes from testing independent elements.  Guilford's old texts
> include a discussion between the trade-off (in this sense) between
> reliability (narrow subject) and validity (generalization).
>
> --
> Rich Ulrich
>
>
>> Date: Sun, 26 May 2013 07:34:04 -0700
>> From:

> bruce.weaver@

>> Subject: Re: Whether to use post  hok tests,              or discriminant
>> function analysis after a one-way significant              MANOVA.
>> (Non-SPSS question).
>> To:

> SPSSX-L@.UGA

>>
>> I suggest you look at the 1989 article by Huberty & Morris.  A PDF
>> can be downloaded here:
>>
>> https://bitbucket.org/eyecat/readinglists/src/d8e8010f0b0d2dbb0863af3
>> 050411695254cc6b1/ReadingList_NotreDame/HubertyMorris1989Multivariate
>> VsUnivariate.pdf
>>
>> See also this thread from a few years ago.
>>
>> http://spssx-discussion.1045642.n5.nabble.com/Using-gender-as-an-outc
>> ome-variable-in-logistic-regression-tp1086376p1086390.html
>>
>> HTH.
>>
>>
>>
>> Faiz Rasool wrote
>> > Hi all,
>> >
>> >
>> > My apologies for a Non-SPSS question.
>> >
>> > My research question is whether females rate three environmental
>> problems
>> > more serious as compare to males. Survey participants were given a
>> > list
>> of
>> > three environmental problems and they were requested to rate each
>> > of
>> the
>> > problem on a 4 points scale (not a problem, small problem, somewhat
>> > serious problem, very serious problem).
>> >
>> > I performed a one-way MANOVA using gender as an IV and the
>> > responses
>> to
>> > the three questions as three DVs. MANOVA gives a significant result
>> > (P
>> <
>> > .000).
>> >
>> > I did consider to use  these three questions as a composite scale,
>> > but
>> the
>> > cronbach alpha statistic was .46, so I desided to use these three
>> > questions as three DVS.
>> >
>> >
>> > I'm slightly confused on what is the best approach to follow after
>> > a significant MANOVA. My research question can be answered by the
>> > ANOVA results that SPSS provides in a MANOVA output. However, I'm
>> > confused
>> after
>> > reading discovering statistics using SPSS by Field who recommends
>> > that MANOVA should always be followed by discriminant function
analysis.
>> I've
>> > searched the internet using Google, and have tried to read few
>> published
>> > studies that have used MANOVA. Unfortunately, I cannot decide what
>> > is
>> the
>> > best approach using one-way ANOVAs or post hok tests, or
>> > discriminant function analysis after a significant MANOVA.
>> >
>> >
>> >
>> >[snip closing; tags]...





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/Whether-to-use-post-hok-tests-
or-discriminant-function-analysis-after-a-one-way-significant-MANOVA-N-tp572
0417p5720421.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command SIGNOFF SPSSX-L For a list of
commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD