Simple Regression Question

[Salbod]

Top the Morning to Everyone:



            I've got a relatively simple question; I can not seem to find
the answer in any of the texts I have on interactions (e.g., Jaccard et
al.). I probably read the answer and it just didn't register. Can anyone
help?



I want to regress an outcome variable (Y) on 4 explanatory variables
(A,B,C,D) and three two-way interactions (AB,AC,AD). I am only interested in
the two-way interactions involving the A predicator; do I need to also
include the other two-way interactions in my model: BC, BD, CD?



TIA



Stephen Salbod, Pace University, NYC

[Swank, Paul R]

Yes you can. What procedure are you using? Are the explanatory variables
categorical? Using the GLM model, you need to specify the model yourself
rather than the default.


Paul R. Swank, Ph.D.
Professor, Developmental Pediatrics
Director of Research,


University of Texas Health Science Center at Houston

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Stephen Salbod
Sent: Tuesday, April 03, 2007 6:21 PM
To: [hidden email]
Subject: Simple Regression Question

Top the Morning to Everyone:



            I've got a relatively simple question; I can not seem to
find the answer in any of the texts I have on interactions (e.g.,
Jaccard et al.). I probably read the answer and it just didn't register.
Can anyone help?



I want to regress an outcome variable (Y) on 4 explanatory variables
(A,B,C,D) and three two-way interactions (AB,AC,AD). I am only
interested in the two-way interactions involving the A predicator; do I
need to also include the other two-way interactions in my model: BC, BD,
CD?



TIA



Stephen Salbod, Pace University, NYC

[Hector Maletta]

        Stephen,
        You do not need to include all interactions. To specify a particular
interaction, say A with B, create a new variable W=A*B and include W in the
regression alongside A, B, C, D, and the other chosen interactions, say
Z=A*C and X=A*D.
        Now consider this. You may be interested in just some of the
interactions, for whatever reason, but investigating whether other
interactions have a significant effect might be enlightening too, wouldn't
it? What if in the end your pet interactions explain less than the ones you
discard?
        In other words, you do not NEED to include any particular
interaction, or any particular variable for that matter, but you may try
those other interactions anyway, just in case, especially if they make any
conceptual sense. At least in one initial trial run, just to confirm they
are not significant or that they do not change anything in your expected
conclusions.
        Hector

        -----Mensaje original-----
De: SPSSX(r) Discussion [mailto:[hidden email]] En nombre de
Stephen Salbod
Enviado el: 04 April 2007 01:21
Para: [hidden email]
Asunto: Simple Regression Question

        Top the Morning to Everyone:



                    I've got a relatively simple question; I can not seem to
find
        the answer in any of the texts I have on interactions (e.g., Jaccard
et
        al.). I probably read the answer and it just didn't register. Can
anyone
        help?



        I want to regress an outcome variable (Y) on 4 explanatory variables
        (A,B,C,D) and three two-way interactions (AB,AC,AD). I am only
interested in
        the two-way interactions involving the A predicator; do I need to
also
        include the other two-way interactions in my model: BC, BD, CD?



        TIA



        Stephen Salbod, Pace University, NYC

[Burleson,Joseph A.]

Stephen:

No, unless you wish to look at the ABC, or any 3-way, then it would be
advisable to include any lower order 2-way antecedent interactions
(i.e., AB, AC, BC).

Joe Burleson

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Stephen Salbod
Sent: Tuesday, April 03, 2007 7:21 PM
To: [hidden email]
Subject: Simple Regression Question

Top the Morning to Everyone:



            I've got a relatively simple question; I can not seem to
find
the answer in any of the texts I have on interactions (e.g., Jaccard et
al.). I probably read the answer and it just didn't register. Can anyone
help?



I want to regress an outcome variable (Y) on 4 explanatory variables
(A,B,C,D) and three two-way interactions (AB,AC,AD). I am only
interested in
the two-way interactions involving the A predicator; do I need to also
include the other two-way interactions in my model: BC, BD, CD?



TIA



Stephen Salbod, Pace University, NYC

[Salbod]

To everyone who responded to my question: I thank you.



Answer: complete set of 2-way interactions is not required.



QUESTION:
I want to regress an outcome variable (Y) on 4 explanatory variables
(A,B,C,D) and three two-way interactions (AB,AC,AD). I am only interested in
the two-way interactions involving the A predicator; do I need to also
include the other two-way interactions in my model: BC, BD, CD?

[statisticsdoc]

Stephen,

You need not include the two-way interaction terms that do not include A in
order to test the two-way interactions that include A (assuming that you
have no interest in the 3-way and 4-way interactions).

Best,

Steve

For personalized and professional consultation in statistics and research
design, visit
www.statisticsdoc.com


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Stephen Salbod
Sent: Tuesday, April 03, 2007 7:21 PM
To: [hidden email]
Subject: Simple Regression Question


Top the Morning to Everyone:



            I've got a relatively simple question; I can not seem to find
the answer in any of the texts I have on interactions (e.g., Jaccard et
al.). I probably read the answer and it just didn't register. Can anyone
help?



I want to regress an outcome variable (Y) on 4 explanatory variables
(A,B,C,D) and three two-way interactions (AB,AC,AD). I am only interested in
the two-way interactions involving the A predicator; do I need to also
include the other two-way interactions in my model: BC, BD, CD?



TIA



Stephen Salbod, Pace University, NYC

[Richard Ristow]

At 07:20 PM 4/3/2007, Stephen Salbod wrote:

>I want to regress an outcome variable (Y) on 4 explanatory variables
>(A,B,C,D) and three two-way interactions (AB,AC,AD). I am only
>interested in the two-way interactions involving the A predicator; do
>I need to also include the other two-way interactions in my model: BC,
>BD, CD?

As you've heard from others: No, you don't have to; you can just add
the cross-terms A*B, A*C, and A*D as regressors. (I'm assuming your
predictors are all continuous; if not, see Paul R. Swank's post.)

You may want to offset the origin for the cross term. That is, instead
of

A*B,

use (A-A0)*(B-B0)

where A0 and B0 are constants around the mean or midrange of the
variables' observed distributions. That will reduce the tendency of A*B
to be correlated with the two factors, which is especially prominent
when the means of A and B are considerably larger than their standard
deviations.

Offsetting this way will change the coefficients of the linear terms,
but won't change the estimation space, the space of possible regression
estimates.

I don't like to call A*B the 'interaction'; I recommend 'cross term'.
In an ANOVA, the interaction analysis does cover everything that could
be observed about the interaction. The cross term A*B omits the full
spectrum of higher effects, like A**2*B, ... Don't go putting these in;
you'll probably use up degrees of freedom, to no good purpose. But do
recognize that you're modeling only a piece of what the interaction
might be.

Since the cross term is a non-linear effect, having one in the model
always makes me wonder about other second-order terms: A**2, B**2.
Don't go tossing them into the model just because I mention them. On
the other hand, in your discussion, you may need to mention why the
cross term is in, but the quadratic terms in the individual variables
are not.

Good luck,
Richard

[Muir Houston]

While this may not be relevant in this instance I was wondering about the use of Mallows' Cp
the issue:
While there may be no gender difference overall, do male and female students display different patterns of performance with advancing age.  To capture non-linear effects for each gender and how the linkage evolves, a dummy denoted female and the interaction terms age ´ female and age2 ´ female were considered also.  By including subsets of the five variables involving age and gender it is possible to estimate academic performance as being linearly or quadratically affected using pooled data on men and women.  To determine the 'different patterns' with advancing 'entry age', the method of best subsets was used (Mallows 1973; Levine et al., 1999).
The Mallows' (1973) approach, involves: (I) estimating a model with the 32 combinations of age and gender that are possible (ranging from including none of the combinations, through to including all five); (ii) calculating a summary measure called Mallows' Cp, based on the number of variables 'p' in the model; and, (iii) selecting as the 'best subset' the collection with Mallows' Cp closest to p+1.
 
Refs:
Mallows CL (1973): Some comments of Cp. Technometrics 15. pp 661--676.

Levine, D, Berenson, M and Stephen, D (1999), Statistics for managers using Microsoft Excel, Prentice Hall, Upper Saddle River

 
 
 
 
 
 
Muir Houston
Research Fellow
CRLL
Institute of Education
University of Stirling
FK9 4LA
01786-46-7615

________________________________

From: SPSSX(r) Discussion on behalf of Richard Ristow
Sent: Wed 04/04/2007 05:11
To: [hidden email]
Subject: Re: Simple Regression Question



At 07:20 PM 4/3/2007, Stephen Salbod wrote:

>I want to regress an outcome variable (Y) on 4 explanatory variables
>(A,B,C,D) and three two-way interactions (AB,AC,AD). I am only
>interested in the two-way interactions involving the A predicator; do
>I need to also include the other two-way interactions in my model: BC,
>BD, CD?

As you've heard from others: No, you don't have to; you can just add
the cross-terms A*B, A*C, and A*D as regressors. (I'm assuming your
predictors are all continuous; if not, see Paul R. Swank's post.)

You may want to offset the origin for the cross term. That is, instead
of

A*B,

use (A-A0)*(B-B0)

where A0 and B0 are constants around the mean or midrange of the
variables' observed distributions. That will reduce the tendency of A*B
to be correlated with the two factors, which is especially prominent
when the means of A and B are considerably larger than their standard
deviations.

Offsetting this way will change the coefficients of the linear terms,
but won't change the estimation space, the space of possible regression
estimates.

I don't like to call A*B the 'interaction'; I recommend 'cross term'.
In an ANOVA, the interaction analysis does cover everything that could
be observed about the interaction. The cross term A*B omits the full
spectrum of higher effects, like A**2*B, ... Don't go putting these in;
you'll probably use up degrees of freedom, to no good purpose. But do
recognize that you're modeling only a piece of what the interaction
might be.

Since the cross term is a non-linear effect, having one in the model
always makes me wonder about other second-order terms: A**2, B**2.
Don't go tossing them into the model just because I mention them. On
the other hand, in your discussion, you may need to mention why the
cross term is in, but the quadratic terms in the individual variables
are not.

Good luck,
Richard



--
The University of Stirling is a university established in Scotland by
charter at Stirling, FK9 4LA.  Privileged/Confidential Information may
be contained in this message.  If you are not the addressee indicated
in this message (or responsible for delivery of the message to such
person), you may not disclose, copy or deliver this message to anyone
and any action taken or omitted to be taken in reliance on it, is
prohibited and may be unlawful.  In such case, you should destroy this
message and kindly notify the sender by reply email.  Please advise
immediately if you or your employer do not consent to Internet email
for messages of this kind.

[Peck, Jon]

I would qualify this a bit.  It isn't mathematically necessary to include all the interactions just because you want to estimate or test one of them, but if the correct model is the one with all of the interactions present, then you are going to get the wrong results for the AB term if you include it alone in the absence of orthogonality.  So if there are plausible reasons for believing that all these interactions exist, you should do the test in the presence of all of them.

You could test the others and remove if (generously measured) they are not significant, but they can certainly affect your results.

HTH,
Jon Peck

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Stephen Salbod
Sent: Tuesday, April 03, 2007 7:22 PM
To: [hidden email]
Subject: Re: [SPSSX-L] Simple Regression Question

To everyone who responded to my question: I thank you.



Answer: complete set of 2-way interactions is not required.



QUESTION:
I want to regress an outcome variable (Y) on 4 explanatory variables
(A,B,C,D) and three two-way interactions (AB,AC,AD). I am only interested in
the two-way interactions involving the A predicator; do I need to also
include the other two-way interactions in my model: BC, BD, CD?

[Hector Maletta]

I strongly support Jon's advice, who has expressed more forcefully what I also tried to convey with weaker words in my own response to Stephen. Variables and interactions of variables do not exist (or can be tested) in isolation: they are related to each other to some extent, and that is the whole point of multiple regression: keeping other variables under control while you measure the effect of each. Ignoring an interaction is equivalent to assuming that their effects are orthogonal, independent of each other. Testing for this is not a bad idea when you are not actually sure. Otherwise, the interactions you ignore may confound those you are trying to assess.

Hector

----- Mensaje original -----
De: "Peck, Jon" <[hidden email]>
Fecha: Miércoles, Abril 4, 2007 3:03 pm
Asunto: Re: Simple Regression Question

> I would qualify this a bit.  It isn't mathematically necessary to
> include all the interactions just because you want to estimate or
> test one of them, but if the correct model is the one with all of
> the interactions present, then you are going to get the wrong
> results for the AB term if you include it alone in the absence of
> orthogonality.  So if there are plausible reasons for believing
> that all these interactions exist, you should do the test in the
> presence of all of them.
>
> You could test the others and remove if (generously measured) they
> are not significant, but they can certainly affect your results.
>
> HTH,
> Jon Peck
>
> -----Original Message-----
> From: SPSSX(r) Discussion [[hidden email]] On Behalf Of
> Stephen Salbod
> Sent: Tuesday, April 03, 2007 7:22 PM
> To: [hidden email]
> Subject: Re: [SPSSX-L] Simple Regression Question
>
> To everyone who responded to my question: I thank you.
>
>
>
> Answer: complete set of 2-way interactions is not required.
>
>
>
> QUESTION:
> I want to regress an outcome variable (Y) on 4 explanatory variables
> (A,B,C,D) and three two-way interactions (AB,AC,AD). I am only
> interested in
> the two-way interactions involving the A predicator; do I need to also
> include the other two-way interactions in my model: BC, BD, CD?
>

[Ornelas, Fermin]

You do not have to, unless there is some a priori information suggesting
that the other interactions are meaningful to your study. You may want
to do plots for each predictor against the response variable including
the interactions if there is some kind of trend in the plots then there
could be basis to include the interactions in the model.


Fermin Ornelas, Ph.D.
Management Analyst III, AZ DES
Tel: [phone]
E-mail: [hidden email]


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Stephen Salbod
Sent: Tuesday, April 03, 2007 4:21 PM
To: [hidden email]
Subject: Simple Regression Question

Top the Morning to Everyone:



            I've got a relatively simple question; I can not seem to
find
the answer in any of the texts I have on interactions (e.g., Jaccard et
al.). I probably read the answer and it just didn't register. Can anyone
help?



I want to regress an outcome variable (Y) on 4 explanatory variables
(A,B,C,D) and three two-way interactions (AB,AC,AD). I am only
interested in
the two-way interactions involving the A predicator; do I need to also
include the other two-way interactions in my model: BC, BD, CD?



TIA



Stephen Salbod, Pace University, NYC

NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the
specific individual(s) to whom it is addressed.  It may contain
information that is privileged and confidential under state and federal
law.  This information may be used or disclosed only in accordance with
law, and you may be subject to penalties under law for improper use or
further disclosure of the information in this e-mail and its
attachments. If you have received this e-mail in error, please
immediately notify the person named above by reply e-mail, and then
delete the original e-mail.  Thank you.