SPSSX Discussion

Sign on regression coef?

Classic

List

Threaded

10 messages Options

lts1

Sign on regression coef?

Hi SPSSers,

First off, Happy New Year to all. Next, my apologies for posting a stats question & not an SPSS question, but I'm hoping someone can help me. I have 3 variables: 2 independent & 1 dependent. When I look at the correlation table, they all correlate positively with each other. However, when I put them in a regression, the regression coefficient on one of the independent variables becomes negative. Can anyone tell me when/why this might occur? Thanks in advance.

Best,
Lisa

Lisa T. Stickney
Ph.D. Candidate
The Fox School of Business
and Management
Temple University
[hidden email]

William B. Ware

Re: Sign on regression coef?

Lisa,

Look at the correlation between the two IVs. I'm going to guess that it
is pretty large... I think that what you have is a case of "suppression."
Check out one of the older editions of Cohen and Cohen or Cohen, Cohen,
Aiken, and West for an explanation...

wbw

__________________________________________________________________________
William B. Ware, Professor Educational Psychology,
CB# 3500 Measurement, and Evaluation
University of North Carolina PHONE (919)-962-7848
Chapel Hill, NC 27599-3500 FAX: (919)-962-1533
Office: 118 Peabody Hall EMAIL: [hidden email]
Adjunct Professor School of Social Work
__________________________________________________________________________

On Tue, 2 Jan 2007, Lisa Stickney wrote:

> Hi SPSSers,
>
> First off, Happy New Year to all. Next, my apologies for posting a stats question & not an SPSS question, but I'm hoping someone can help me. I have 3 variables: 2 independent & 1 dependent. When I look at the correlation table, they all correlate positively with each other. However, when I put them in a regression, the regression coefficient on one of the independent variables becomes negative. Can anyone tell me when/why this might occur? Thanks in advance.
>
> Best,
> Lisa
>
> Lisa T. Stickney
> Ph.D. Candidate
> The Fox School of Business
> and Management
> Temple University
> [hidden email]
>

lts1

Re: Sign on regression coef?

Hi William,

Thanks for the reply. I considered supression but the correlation
between the two IVs is .385 (p < .001). Is that high enough for
suppression? Also, I thought that a condition of suppression was that the
suppressor variable be uncorrelated with the DV (in my case r = .176, p <
.05). Is this incorrect? Thanks in advance.

Best,
Lisa

----- Original Message -----
From: "William B. Ware" <[hidden email]>
To: <[hidden email]>
Sent: Tuesday, January 02, 2007 9:30 PM
Subject: Re: Sign on regression coef?

> Lisa,
>
> Look at the correlation between the two IVs. I'm going to guess that it
> is pretty large... I think that what you have is a case of "suppression."
> Check out one of the older editions of Cohen and Cohen or Cohen, Cohen,
> Aiken, and West for an explanation...
>
> wbw
>
> __________________________________________________________________________
> William B. Ware, Professor Educational Psychology,
> CB# 3500 Measurement, and Evaluation
> University of North Carolina PHONE (919)-962-7848
> Chapel Hill, NC 27599-3500 FAX: (919)-962-1533
> Office: 118 Peabody Hall EMAIL: [hidden email]
> Adjunct Professor School of Social Work
> __________________________________________________________________________
>
>
> On Tue, 2 Jan 2007, Lisa Stickney wrote:
>
>> Hi SPSSers,
>>
>> First off, Happy New Year to all. Next, my apologies for posting a
>> stats question & not an SPSS question, but I'm hoping someone can help
>> me. I have 3 variables: 2 independent & 1 dependent. When I look at the
>> correlation table, they all correlate positively with each other.
>> However, when I put them in a regression, the regression coefficient on
>> one of the independent variables becomes negative. Can anyone tell me
>> when/why this might occur? Thanks in advance.
>>
>> Best,
>> Lisa
>>
>> Lisa T. Stickney
>> Ph.D. Candidate
>> The Fox School of Business
>> and Management
>> Temple University
>> [hidden email]
>>
>

statisticsdoc

Re: Sign on regression coef?

In reply to this post by lts1

Stephen Brand
www.statisticsdoc.com

Hi Lisa,

This is an instance of supression of one IV by another. X2 may have a
positive correlation with Y, but when X2 is entered into a regression
equation that already contains X1, the relationship between the residual
variance in X2 (after removing its shared variance with X1) and Y is
negative. This problem arises quite often when there is a high degree of
collinearity between the predictors, and is covered well by Cohen & Cohen
(1975), Pedhauzur, and others.

HTH,

Stephen Brand

For personalized and professional consultation in statistics and research
design, visit
www.statisticsdoc.com

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Lisa Stickney
Sent: Tuesday, January 02, 2007 8:51 PM
To: [hidden email]
Subject: Sign on regression coef?

Hi SPSSers,

First off, Happy New Year to all. Next, my apologies for posting a
stats question & not an SPSS question, but I'm hoping someone can help me.
I have 3 variables: 2 independent & 1 dependent. When I look at the
correlation table, they all correlate positively with each other. However,
when I put them in a regression, the regression coefficient on one of the
independent variables becomes negative. Can anyone tell me when/why this
might occur? Thanks in advance.

Best,
Lisa

Lisa T. Stickney
Ph.D. Candidate
The Fox School of Business
and Management
Temple University
[hidden email]

Judith Saebel

Re: Sign on regression coef?

In reply to this post by lts1

Hi Lisa,

You might also want to look at Tabachnick & Fidell (2001, pp. 148-149),
part 5.5.4 (Suppressor variables).

Tabachnick, B. G. & Fidell, L. S. (2001). Using multivariate
statistics (4th ed.). Boston, MA: Allyn and Bacon.
(ISBN 0-321--05677-9)

Best of luck,

Judith

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Lisa Stickney
Sent: Wednesday, 3 January 2007 13:25
To: [hidden email]
Subject: Re: Sign on regression coef?

Hi William,

Thanks for the reply. I considered supression but the correlation
between the two IVs is .385 (p < .001). Is that high enough for
suppression? Also, I thought that a condition of suppression was that
the
suppressor variable be uncorrelated with the DV (in my case r = .176, p
<
.05). Is this incorrect? Thanks in advance.

Best,
Lisa

----- Original Message -----
From: "William B. Ware" <[hidden email]>
To: <[hidden email]>
Sent: Tuesday, January 02, 2007 9:30 PM
Subject: Re: Sign on regression coef?

> Lisa,
>
> Look at the correlation between the two IVs. I'm going to guess that
it
> is pretty large... I think that what you have is a case of
"suppression."
> Check out one of the older editions of Cohen and Cohen or Cohen,
Cohen,
> Aiken, and West for an explanation...
>
> wbw
>
>
________________________________________________________________________
__
> William B. Ware, Professor Educational
Psychology,
> CB# 3500 Measurement, and
Evaluation
> University of North Carolina PHONE
(919)-962-7848
> Chapel Hill, NC 27599-3500 FAX:
(919)-962-1533
> Office: 118 Peabody Hall EMAIL:
[hidden email]
> Adjunct Professor School of Social
Work
>
________________________________________________________________________
__
>
>
> On Tue, 2 Jan 2007, Lisa Stickney wrote:
>
>> Hi SPSSers,
>>
>> First off, Happy New Year to all. Next, my apologies for posting
a
>> stats question & not an SPSS question, but I'm hoping someone can
help
>> me. I have 3 variables: 2 independent & 1 dependent. When I look at
the
>> correlation table, they all correlate positively with each other.
>> However, when I put them in a regression, the regression coefficient
on
>> one of the independent variables becomes negative. Can anyone tell
me

>> when/why this might occur? Thanks in advance.
>>
>> Best,
>> Lisa
>>
>> Lisa T. Stickney
>> Ph.D. Candidate
>> The Fox School of Business
>> and Management
>> Temple University
>> [hidden email]
>>
>

David Hitchin

Re: Sign on regression coef?

In reply to this post by lts1

--On 02 January 2007 20:51 -0500 Lisa Stickney <[hidden email]> wrote:

> However, when I put them in a regression, the regression coefficient on
> one of the independent variables becomes negative. Can anyone tell me
> when/why this might occur?

Here's an example. You are planning a marketing campaign, and you are most
likely to sell where people have a large disposable income.

You find a proxy variable for income, and on the whole, the more that
people earn, the larger their disposable income, so positively correlated.

You might also have a proxy for expenditure. On the whole, people who spend
a lot are likely to have more disposable income. Positive correlation.

However, the really useful value to estimate is income less expenditure,
which gives you the disposable income free to buy something else.

This has a positive coefficient for income, and a negative one for
expenditure.

David Hitchin

statisticsdoc

Re: Sign on regression coef?

In reply to this post by lts1

Lisa,

David's example is a good one. Another example that comes to mind has been
reported in research on self-esteem in early adolescence. In a study by
David Dubois, peer self-esteem and academic self-esteem both correlated with
students' academic performance, and with one another. However, when both
academic and peer self-esteem were entered into a reqression equation to
predict students' academic performance, the beta weight for peer-self esteem
is negative. Students' who have high levels of peer self-esteem relative to
academic self-esteem have poorer performance. Illustratively, students with
high peer and low academic self-esteem do poorly, while those who have low
peer and high academic self-esteem perform well. A key issue to examine
with supressor variables is the relationship between the unique variance in
the predictors and the criterion variable. In this example, the unique
variance of peer self-esteem (the portion that is not shared with academic
self-esteem) has a negative association with acsdemic performance. Such
situations can arise even when the predictor variables have a moderate
correlation (.4 to .5), depending on how the remaining unique variance is
associated with the criterion variable.

HTH,

Stephen Brand

For personalized and professional consultation in statistics and research
design, visit
www.statisticsdoc.com

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Lisa Stickney
Sent: Tuesday, January 02, 2007 8:51 PM
To: [hidden email]
Subject: Sign on regression coef?

Can anyone tell me when/why this might occur? Thanks in advance.

Best,
Lisa

Lisa T. Stickney
Ph.D. Candidate
The Fox School of Business
and Management
Temple University
[hidden email]

lts1

Re: Sign on regression coef?

My thanks to all for your comments, help & insight.

Best,
Lisa

----- Original Message -----
From: "Statisticsdoc" <[hidden email]>
To: <[hidden email]>
Sent: Wednesday, January 03, 2007 10:38 AM
Subject: Re: Sign on regression coef?

> Lisa,
>
> David's example is a good one. Another example that comes to mind has
> been
> reported in research on self-esteem in early adolescence. In a study by
> David Dubois, peer self-esteem and academic self-esteem both correlated
> with
> students' academic performance, and with one another. However, when both
> academic and peer self-esteem were entered into a reqression equation to
> predict students' academic performance, the beta weight for peer-self
> esteem
> is negative. Students' who have high levels of peer self-esteem relative
> to
> academic self-esteem have poorer performance. Illustratively, students
> with
> high peer and low academic self-esteem do poorly, while those who have low
> peer and high academic self-esteem perform well. A key issue to examine
> with supressor variables is the relationship between the unique variance
> in
> the predictors and the criterion variable. In this example, the unique
> variance of peer self-esteem (the portion that is not shared with academic
> self-esteem) has a negative association with acsdemic performance. Such
> situations can arise even when the predictor variables have a moderate
> correlation (.4 to .5), depending on how the remaining unique variance is
> associated with the criterion variable.
>
> HTH,
>
> Stephen Brand
>
> For personalized and professional consultation in statistics and research
> design, visit
> www.statisticsdoc.com
>
>
> -----Original Message-----
> From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
> Lisa Stickney
> Sent: Tuesday, January 02, 2007 8:51 PM
> To: [hidden email]
> Subject: Sign on regression coef?
>
> Can anyone tell me when/why this might occur? Thanks in advance.
>
> Best,
> Lisa
>
> Lisa T. Stickney
> Ph.D. Candidate
> The Fox School of Business
> and Management
> Temple University
> [hidden email]
>

Henrik Lolle

Re: Sign on regression coef?

Hi Lisa,

The phenomenon is also called for Simpson's Paradox (when the sign is
inverted and not just supressed). You can read more on:

http://plato.stanford.edu/entries/paradox-simpson/

Best,
Henrik

Lisa Stickney wrote:

> My thanks to all for your comments, help & insight.
>
> Best,
> Lisa
>
>
> ----- Original Message -----
> From: "Statisticsdoc" <[hidden email]>
> To: <[hidden email]>
> Sent: Wednesday, January 03, 2007 10:38 AM
> Subject: Re: Sign on regression coef?
>
>
>> Lisa,
>>
>> David's example is a good one. Another example that comes to mind has
>> been
>> reported in research on self-esteem in early adolescence. In a study by
>> David Dubois, peer self-esteem and academic self-esteem both correlated
>> with
>> students' academic performance, and with one another. However, when both
>> academic and peer self-esteem were entered into a reqression equation to
>> predict students' academic performance, the beta weight for peer-self
>> esteem
>> is negative. Students' who have high levels of peer self-esteem relative
>> to
>> academic self-esteem have poorer performance. Illustratively, students
>> with
>> high peer and low academic self-esteem do poorly, while those who have
>> low
>> peer and high academic self-esteem perform well. A key issue to examine
>> with supressor variables is the relationship between the unique variance
>> in
>> the predictors and the criterion variable. In this example, the unique
>> variance of peer self-esteem (the portion that is not shared with
>> academic
>> self-esteem) has a negative association with acsdemic performance. Such
>> situations can arise even when the predictor variables have a moderate
>> correlation (.4 to .5), depending on how the remaining unique variance is
>> associated with the criterion variable.
>>
>> HTH,
>>
>> Stephen Brand
>>
>> For personalized and professional consultation in statistics and research
>> design, visit
>> www.statisticsdoc.com
>>
>>
>> -----Original Message-----
>> From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
>> Lisa Stickney
>> Sent: Tuesday, January 02, 2007 8:51 PM
>> To: [hidden email]
>> Subject: Sign on regression coef?
>>
>> Can anyone tell me when/why this might occur? Thanks in advance.
>>
>> Best,
>> Lisa
>>
>> Lisa T. Stickney
>> Ph.D. Candidate
>> The Fox School of Business
>> and Management
>> Temple University
>> [hidden email]
>>

--

***********************************************************
Henrik Lolle
Associate Professor
Department of Economics, Politics and Public Administration
Aalborg University
Fibigerstraede 1
DK 9220 Aalborg East
Home page:
http://www.socsci.aau.dk/~lolle/
http://www.socsci.aau.dk/institut2/dansk/empl/lolle.htm
***********************************************************

Dominic Lusinchi

Re: Sign on regression coef?

In reply to this post by lts1

Lisa,

I haven't followed the thread very carefully, so this may be redundant...

To see how it is possible to have a positive correlation (r) between a
predictor (X1) and the outcome (Y) and a negative regression coefficient b1,
it is useful to look at the formula for the standardized coefficient for a
2-predictor model:

b1 = [ryx1 - (ryx2 x rx1x2)] / [1 - (rx1x2)^2]

in which "b1" is the standardized regression coefficient for x1, "ryx1" the
correlation between y and x1, "ryx2" the correlation between y and x2, and
"rx1x2" the correlation between x1 and x2. ("x" is the multiplication sign
in he numerator and "^" the exponentiation sign in the denominator.)

Now we can see that if the term "ryx2 x rx1x2" is larger than the
correlation between y and x1, we get a negative coefficient. In order for
that term to be larger than ryx1, we must have ryx2>ryx1 and the correlation
between the two predictors must be high enough so that when it multiplies
ryx2, the product is still larger than ryx1.

Good luck, and best for the New Year to all.

Dominic Lusinchi
Statistician
Far West Research
Statistical Consulting
San Francisco, California
415-664-3032
www.farwestresearch.com
-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Lisa Stickney
Sent: Wednesday, January 03, 2007 12:50 PM
To: [hidden email]
Subject: Re: Sign on regression coef?

My thanks to all for your comments, help & insight.

Best,
Lisa

----- Original Message -----
From: "Statisticsdoc" <[hidden email]>
To: <[hidden email]>
Sent: Wednesday, January 03, 2007 10:38 AM
Subject: Re: Sign on regression coef?