Is ANCOVA being used correctly?

[mithrauko]

Hi! I am having a serious horrible time with an ANCOVA issue, and I would be immensely grateful for any help.

I am writing an evaluation of a research paper, which can be found here: http://w3.psychology.su.se/staff/mar...h_Med_2004.pdf

Basically, they are comparing a) a group of depressed individuals with a group of non-depressed individuals and b) different types of depressed individuals (eg, dysthymia, minor depression etc) with each other an with the non-depressed individuals. They use ANCOVAs. They use gender as a covariate in all these.

Ok, so, problem: Basically, in Field (Discovering statistics... p397-9) and also Miller and Chapman (2001) as cited in Field (http://homepages.gold.ac.uk/aphome/ancova.pdf), it says 'analysis of covariance cannot tell us how groups would differ if they did not differ on the covariate' and that 'mistakenly, investigators frequently turn to ANCOVA in hopes of 'controlling for' group differences on the covariate' etc etc (both quotes from M&C).

Therefore, they both seem to suggest that you can't use an ancova if there are differences between groups as to the covariate.

Questions:

1) In my Paper 2, it says that gender differs across groups (according to a chi-squared test), but they still entered gender as the covariate. Is this the wrong analysis method to use then??

2) To complicate matters, it goes on about how the covariate must be independent of the treatment effect etc, but I don't exactly have a treatment effect as it's just measuring scores on tests, and so I got confused. Field puts as an example: anxiety is correlated with depression (anxious people tend to be more depressed), so if comparing anxious with non anxious people, the anxious group might be more depressed. Might want to enter depression as covariate to find 'pure' effect of anxiety, but you can't as anxiety and depression share variance. I can understand this example, but unfortunately I can't relate it to my Paper 2 as I am confused. Gender is correlated with depression (or at least, females more likely to be depressed, is that the same thing?)

- does this mean 'gender shares some of its variance with depression'?

Moreover it seems that if gender affects test scores, which it might, then it also can't be used as a covariate..?

Lastly, some papers say you can only use a continuous variable as a covariate, but gender is obviously categorical.

Essentially, have they used ANCOVAs incorrectly, then? What should they have done instead, if so?

REALLY CONFUSED!! Any help would be much much much appreciated.

Isabel

[Rich Ulrich]

I found the paper at
http://w3.psychology.su.se/staff/marlar/Psych_Med_2004.pdf .
(Does Google-groups automatically screw up *every* URL with
an ellipsis, even in private e-mail?)

Dichotomous covariate:  This works out exactly the same as
having the dichotomy present as a factor while ignoring the
interactions with that factor.  Calling the analysis an ANCOVA
says that the covariate is evaluated before the factors, as is
appropriate.

The advice about unequal covariates applies with the same
(questionable) force to having unbalanced groups.   That is,
you do want to be a bit careful in drawing conclusions.  But
the problem is one that exists when the unequal covariate
(thus, correlated: Yes, gender has potential to account for some
of the variance of diagnoses of depression) is also correlated
with the outcome (not, apparently, the case here).  The proper
conclusion, in that sort of instance, is not that any such
analysis is dis-allowed; rather, you do need to limit your
conclusions by carefully considering things like group means
and *how* their pattern might be fairly interpreted.

In this study, Gender is being treated as a nuisance parameter,
"controlled for" but ignored.  Presumably, there were no notable
correlations of Gender with the outcomes.  The authors (so far
as I see) don't mention any.  Given the nature of the outcomes,
I don't particularly expect any.  If there aren't any at all, even a
hint, then there was never any problem. 

If I were reviewing the study (and thought of it), I would have asked
for a firm statement that Gender was irrelevant to outcome.

However, you are correct to this extent, that there *potentially*
could be confounding by gender, if they did not explicitly rule it out.

--
Rich Ulrich

> Date: Thu, 26 Sep 2013 11:29:14 -0700

> From: [hidden email]
> Subject: Is ANCOVA being used correctly?
> To: [hidden email]
>
> Hi! I am having a serious horrible time with an ANCOVA issue, and I would be
> immensely grateful for any help.
>
> I am writing an evaluation of a research paper, which can be found here:
> http://w3.psychology.su.se/staff/mar...h_Med_2004.pdf
>
> Basically, they are comparing a) a group of depressed individuals with a
> group of non-depressed individuals and b) different types of depressed
> individuals (eg, dysthymia, minor depression etc) with each other an with
> the non-depressed individuals. They use ANCOVAs. They use gender as a
> covariate in all these.
>
> Ok, so, problem: Basically, in Field (Discovering statistics... p397-9) and
> also Miller and Chapman (2001) as cited in Field
> (http://homepages.gold.ac.uk/aphome/ancova.pdf), it says 'analysis of
> covariance cannot tell us how groups would differ if they did not differ on
> the covariate' and that 'mistakenly, investigators frequently turn to ANCOVA
> in hopes of 'controlling for' group differences on the covariate' etc etc
> (both quotes from M&C).
>
> Therefore, they both seem to suggest that you can't use an ancova if there
> are differences between groups as to the covariate.
>
> Questions:
>
> 1) In my Paper 2, it says that gender differs across groups (according to a
> chi-squared test), but they still entered gender as the covariate. Is this
> the wrong analysis method to use then??
>
> 2) To complicate matters, it goes on about how the covariate must be
> independent of the treatment effect etc, but I don't exactly have a
> treatment effect as it's just measuring scores on tests, and so I got
> confused. Field puts as an example: anxiety is correlated with depression
> (anxious people tend to be more depressed), so if comparing anxious with non
> anxious people, the anxious group might be more depressed. Might want to
> enter depression as covariate to find 'pure' effect of anxiety, but you
> can't as anxiety and depression share variance. I can understand this
> example, but unfortunately I can't relate it to my Paper 2 as I am confused.
> Gender is correlated with depression (or at least, females more likely to be
> depressed, is that the same thing?)
>
> - does this mean 'gender shares some of its variance with depression'?
>
> Moreover it seems that if gender affects test scores, which it might, then
> it also can't be used as a covariate..?
>
> Lastly, some papers say you can only use a continuous variable as a
> covariate, but gender is obviously categorical.
>
> Essentially, have they used ANCOVAs incorrectly, then? What should they have
> done instead, if so?
>
> REALLY CONFUSED!! Any help would be much much much appreciated.
>
> Isabel

[Bruce Weaver]

This post reminded me of some comments made in an older thread that started off being about the Chow test, but ended up being more about ANCOVA-like models in general.  Here are some comments I made followed by comments from Richard Ristow.

http://spssx-discussion.1045642.n5.nabble.com/Chow-test-to-compare-regression-on-two-groups-tp2839448p2839846.html

http://spssx-discussion.1045642.n5.nabble.com/Chow-test-to-compare-regression-on-two-groups-tp2839448p2839868.html


HTH.

mithrauko wrote

Hi! I am having a serious horrible time with an ANCOVA issue, and I would be immensely grateful for any help.

I am writing an evaluation of a research paper, which can be found here: http://w3.psychology.su.se/staff/mar...h_Med_2004.pdf

Basically, they are comparing a) a group of depressed individuals with a group of non-depressed individuals and b) different types of depressed individuals (eg, dysthymia, minor depression etc) with each other an with the non-depressed individuals. They use ANCOVAs. They use gender as a covariate in all these.

Ok, so, problem: Basically, in Field (Discovering statistics... p397-9) and also Miller and Chapman (2001) as cited in Field (http://homepages.gold.ac.uk/aphome/ancova.pdf), it says 'analysis of covariance cannot tell us how groups would differ if they did not differ on the covariate' and that 'mistakenly, investigators frequently turn to ANCOVA in hopes of 'controlling for' group differences on the covariate' etc etc (both quotes from M&C).

Therefore, they both seem to suggest that you can't use an ancova if there are differences between groups as to the covariate.

Questions:

1) In my Paper 2, it says that gender differs across groups (according to a chi-squared test), but they still entered gender as the covariate. Is this the wrong analysis method to use then??

2) To complicate matters, it goes on about how the covariate must be independent of the treatment effect etc, but I don't exactly have a treatment effect as it's just measuring scores on tests, and so I got confused. Field puts as an example: anxiety is correlated with depression (anxious people tend to be more depressed), so if comparing anxious with non anxious people, the anxious group might be more depressed. Might want to enter depression as covariate to find 'pure' effect of anxiety, but you can't as anxiety and depression share variance. I can understand this example, but unfortunately I can't relate it to my Paper 2 as I am confused. Gender is correlated with depression (or at least, females more likely to be depressed, is that the same thing?)

- does this mean 'gender shares some of its variance with depression'?

Moreover it seems that if gender affects test scores, which it might, then it also can't be used as a covariate..?

Lastly, some papers say you can only use a continuous variable as a covariate, but gender is obviously categorical.

Essentially, have they used ANCOVAs incorrectly, then? What should they have done instead, if so?

REALLY CONFUSED!! Any help would be much much much appreciated.

Isabel

[Ismail Noor]

When I click on this link>>>>>>>>>>>>>>>>>>>
"I am writing an evaluation of a research paper, which can be found here:
> http://w3.psychology.su.se/staff/mar...h_Med_2004.pdf"................................

I get this:  

Not Found

The requested URL /staff/mar...h_Med_2004.pdf was not found on this server.

 

 

Ismail K. Noor, Ph.D.,
Data Group Consultant, LLC
1335 Culver Avenue
Dearborn, Michigan   48124
[hidden email]
[phone]
[phone]

---

From: Bruce Weaver <[hidden email]>
To: [hidden email]
Sent: Monday, September 30, 2013 6:29 PM
Subject: Re: Is ANCOVA being used correctly?

This post reminded me of some comments made in an older thread that started
off being about the Chow test, but ended up being more about ANCOVA-like
models in general.  Here are some comments I made followed by comments from
Richard Ristow.

http://spssx-discussion.1045642.n5.nabble.com/Chow-test-to-compare-regression-on-two-groups-tp2839448p2839846.html

http://spssx-discussion.1045642.n5.nabble.com/Chow-test-to-compare-regression-on-two-groups-tp2839448p2839868.html


HTH.



mithrauko wrote

> Hi! I am having a serious horrible time with an ANCOVA issue, and I would
> be immensely grateful for any help.
>
> I am writing an evaluation of a research paper, which can be found here:
> http://w3.psychology.su.se/staff/mar...h_Med_2004.pdf
>
> Basically, they are comparing a) a group of depressed individuals with a
> group of non-depressed individuals and b) different types of depressed
> individuals (eg, dysthymia, minor depression etc) with each other an with
> the non-depressed individuals. They use ANCOVAs. They use gender as a
> covariate in all these.
>
> Ok, so, problem: Basically, in Field (Discovering statistics... p397-9)
> and also Miller and Chapman (2001) as cited in Field
> (http://homepages.gold.ac.uk/aphome/ancova.pdf), it says 'analysis of
> covariance cannot tell us how groups would differ if they did not differ
> on the covariate' and that 'mistakenly, investigators frequently turn to
> ANCOVA in hopes of 'controlling for' group differences on the covariate'
> etc etc (both quotes from M&C).
>
> Therefore, they both seem to suggest that you can't use an ancova if there
> are differences between groups as to the covariate.
>
> Questions:
>
> 1) In my Paper 2, it says that gender differs across groups (according to
> a chi-squared test), but they still entered gender as the covariate. Is
> this the wrong analysis method to use then??
>
> 2) To complicate matters, it goes on about how the covariate must be
> independent of the treatment effect etc, but I don't exactly have a
> treatment effect as it's just measuring scores on tests, and so I got
> confused. Field puts as an example: anxiety is correlated with depression
> (anxious people tend to be more depressed), so if comparing anxious with
> non anxious people, the anxious group might be more depressed. Might want
> to enter depression as covariate to find 'pure' effect of anxiety, but you
> can't as anxiety and depression share variance. I can understand this
> example, but unfortunately I can't relate it to my Paper 2 as I am
> confused. Gender is correlated with depression (or at least, females more
> likely to be depressed, is that the same thing?)
>
> - does this mean 'gender shares some of its variance with depression'?
>
> Moreover it seems that if gender affects test scores, which it might, then
> it also can't be used as a covariate..?
>
> Lastly, some papers say you can only use a continuous variable as a
> covariate, but gender is obviously categorical.
>
> Essentially, have they used ANCOVAs incorrectly, then? What should they
> have done instead, if so?
>
> REALLY CONFUSED!! Any help would be much much much appreciated.
>
> Isabel

-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Is-ANCOVA-being-used-correctly-tp5722258p5722337.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

Try the link given in Rich Ulrich's post:

   http://w3.psychology.su.se/staff/marlar/Psych_Med_2004.pdf

Ismail Noor wrote

When I click on this link>>>>>>>>>>>>>>>>>>>
"I am writing an evaluation of a research paper, which can be found here:
> http://w3.psychology.su.se/staff/mar...h_Med_2004.pdf"................................

I get this:  

Not Found
The requested URL /staff/mar...h_Med_2004.pdf was not found on this server.

 
 
Ismail K. Noor, Ph.D.,
Data Group Consultant, LLC
1335 Culver Avenue
Dearborn, Michigan   48124
[hidden email]
[phone]
[phone]



________________________________
 From: Bruce Weaver <[hidden email]>
To: [hidden email] 
Sent: Monday, September 30, 2013 6:29 PM
Subject: Re: Is ANCOVA being used correctly?
 

This post reminded me of some comments made in an older thread that started
off being about the Chow test, but ended up being more about ANCOVA-like
models in general.  Here are some comments I made followed by comments from
Richard Ristow.

http://spssx-discussion.1045642.n5.nabble.com/Chow-test-to-compare-regression-on-two-groups-tp2839448p2839846.html

http://spssx-discussion.1045642.n5.nabble.com/Chow-test-to-compare-regression-on-two-groups-tp2839448p2839868.html


HTH.



mithrauko wrote
> Hi! I am having a serious horrible time with an ANCOVA issue, and I would
> be immensely grateful for any help.
>
> I am writing an evaluation of a research paper, which can be found here:
> http://w3.psychology.su.se/staff/mar...h_Med_2004.pdf
>
> Basically, they are comparing a) a group of depressed individuals with a
> group of non-depressed individuals and b) different types of depressed
> individuals (eg, dysthymia, minor depression etc) with each other an with
> the non-depressed individuals. They use ANCOVAs. They use gender as a
> covariate in all these.
>
> Ok, so, problem: Basically, in Field (Discovering statistics... p397-9)
> and also Miller and Chapman (2001) as cited in Field
> (http://homepages.gold.ac.uk/aphome/ancova.pdf), it says 'analysis of
> covariance cannot tell us how groups would differ if they did not differ
> on the covariate' and that 'mistakenly, investigators frequently turn to
> ANCOVA in hopes of 'controlling for' group differences on the covariate'
> etc etc (both quotes from M&C).
>
> Therefore, they both seem to suggest that you can't use an ancova if there
> are differences between groups as to the covariate.
>
> Questions:
>
> 1) In my Paper 2, it says that gender differs across groups (according to
> a chi-squared test), but they still entered gender as the covariate. Is
> this the wrong analysis method to use then??
>
> 2) To complicate matters, it goes on about how the covariate must be
> independent of the treatment effect etc, but I don't exactly have a
> treatment effect as it's just measuring scores on tests, and so I got
> confused. Field puts as an example: anxiety is correlated with depression
> (anxious people tend to be more depressed), so if comparing anxious with
> non anxious people, the anxious group might be more depressed. Might want
> to enter depression as covariate to find 'pure' effect of anxiety, but you
> can't as anxiety and depression share variance. I can understand this
> example, but unfortunately I can't relate it to my Paper 2 as I am
> confused. Gender is correlated with depression (or at least, females more
> likely to be depressed, is that the same thing?)
>
> - does this mean 'gender shares some of its variance with depression'?
>
> Moreover it seems that if gender affects test scores, which it might, then
> it also can't be used as a covariate..?
>
> Lastly, some papers say you can only use a continuous variable as a
> covariate, but gender is obviously categorical.
>
> Essentially, have they used ANCOVAs incorrectly, then? What should they
> have done instead, if so?
>
> REALLY CONFUSED!! Any help would be much much much appreciated.
>
> Isabel





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Is-ANCOVA-being-used-correctly-tp5722258p5722337.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD