Factor analysis & Interaction terms

[mikesampson]

Hi everyone,

Is it possible to include interaction terms in a factor analysis. Specifically, if we find that a few variables are highly correlated, can we create an interaction (between those two highly correlated variables) and re-run the factor analysis with it (the interaction term)?

Any advice is greatly appreciated.

thanks!
Mike

[David Marso]

WTF are you talking about here?
I suspect you don't know!

mikesampson wrote

Hi everyone,

Is it possible to include interaction terms in a factor analysis. Specifically, if we find that a few variables are highly correlated, can we create an interaction (between those two highly correlated variables) and re-run the factor analysis with it (the interaction term)?

Any advice is greatly appreciated.

thanks!
Mike

[Art Kendall]

I am not sure what you are asking.
Are you distinguishing "factor" as an independent or other predictor/explanatory in an analysis such as ANOVA  or regression
vs
"factor" as a way to represent what is common to several variables in a single variable, ie., to create a summative scale?

A simple example of the latter might repainting a series of test items by the number correct.

[Art Kendall]

Please ask you question in more detail, including what your study is about, what questions you are trying to answer, what constructs you have and   how you are operationalizing them in variables.
What are you variable?

[mikesampson]

Thank you for your response. I'm referring to the latter - running a factor analysis among a series of variables and including amongst them an interaction term. Can this be done or is this statistical garbage, for the lack of a better term.

[mikesampson]

I should've clarified that I was referring more on a theoretical level.

Sorry for the confusion.

Mike

[Rich Ulrich]

Okay, after reading a few responses - I get it.  The key here is to use
the notion of "highly correlated" as the starting point.  And then, you
think of "interaction" as the broader idea of, "What information is here
that is not measured directly?"

Given two variables that are *very* highly correlated, like two protein
or hormone levels or two economic indicators, it is *usually* useful and
meaningful to compute and use their ratio (or the log of their ratio) or
their simple difference.  - Only use a simple difference, A minus B, if you
have made sure that the scaling is compatible for A and B.

This does not arise very often, but it is (unfortunately) likely to be
overlooked when it does.  When do you need to define the set of
variables to use, and how to transform them? Answer: - "at the start." 

Do you, as analyst, always use the variables as they are handed to you,
or do you argue for a basic set of measures that are individually
well-distributed have other good properties?   - by "good properties",
I am thinking precisely of the notion that two near-identical variables
do not form an "orthogonal basis set" (Is that the mathematical term?)
where their sum and difference do.

So, in answer to your question -- I think it is fine to replace (A,B) that are
highly correlated with (A, f(A,B))  that are nearly uncorrelated.  Your
mention of factor analysis was a bit of a distraction, since I would have
probably considered the change before I got that far.  But if that is when
you notice that variables are too mutually-confounding to be useful... sure,
reformulate the set of variables and start over.  You don't want two near-
identical variables in later analyses.  For the sort of measures that I used
as examples, I would pick A or B to go along with their "difference" function.
For rating scales, I would consider using their sum or average.

--
Rich Ulrich

> Date: Fri, 26 Sep 2014 06:18:50 -0700

> From: [hidden email]
> Subject: Factor analysis & Interaction terms
> To: [hidden email]
>
> Hi everyone,
>
> Is it possible to include interaction terms in a factor analysis.
> Specifically, if we find that a few variables are highly correlated, can we
> create an interaction (between those two highly correlated variables) and
> re-run the factor analysis with it (the interaction term)?
>
> Any advice is greatly appreciated.
>
> thanks!
> Mike
>

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Kirill Orlov]

--> notion that two near-identical variables
do not form an "orthogonal basis set" (Is that the mathematical term?)
where their sum and difference do.

Are you referring to that when X and Y are of equal variances then r b\w X+Y and X-Y is 0, whatever r b\w X and Y?

26.09.2014 21:02, Rich Ulrich пишет:

Okay, after reading a few responses - I get it.  The key here is to use
the notion of "highly correlated" as the starting point.  And then, you
think of "interaction" as the broader idea of, "What information is here
that is not measured directly?"

Given two variables that are *very* highly correlated, like two protein
or hormone levels or two economic indicators, it is *usually* useful and
meaningful to compute and use their ratio (or the log of their ratio) or
their simple difference.  - Only use a simple difference, A minus B, if you
have made sure that the scaling is compatible for A and B.

This does not arise very often, but it is (unfortunately) likely to be
overlooked when it does.  When do you need to define the set of
variables to use, and how to transform them? Answer: - "at the start." 

Do you, as analyst, always use the variables as they are handed to you,
or do you argue for a basic set of measures that are individually
well-distributed have other good properties?   - by "good properties",
I am thinking precisely of the notion that two near-identical variables
do not form an "orthogonal basis set" (Is that the mathematical term?)
where their sum and difference do.

So, in answer to your question -- I think it is fine to replace (A,B) that are
highly correlated with (A, f(A,B))  that are nearly uncorrelated.  Your
mention of factor analysis was a bit of a distraction, since I would have
probably considered the change before I got that far.  But if that is when
you notice that variables are too mutually-confounding to be useful... sure,
reformulate the set of variables and start over.  You don't want two near-
identical variables in later analyses.  For the sort of measures that I used
as examples, I would pick A or B to go along with their "difference" function.
For rating scales, I would consider using their sum or average.

--
Rich Ulrich

> Date: Fri, 26 Sep 2014 06:18:50 -0700
> From: [hidden email]
> Subject: Factor analysis & Interaction terms
> To: [hidden email]
>
> Hi everyone,
>
> Is it possible to include interaction terms in a factor analysis.
> Specifically, if we find that a few variables are highly correlated, can we
> create an interaction (between those two highly correlated variables) and
> re-run the factor analysis with it (the interaction term)?
>
> Any advice is greatly appreciated.
>
> thanks!
> Mike
>

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Rich Ulrich]

[posting inverted compared to the SPSSX-List  ordinary order. ]

---

Date: Sat, 27 Sep 2014 08:39:15 +0400
From: [hidden email]
Subject: Re: Factor analysis & Interaction terms
To: [hidden email]
 
quoting me>
--> notion that two near-identical variables
do not form an "orthogonal basis set" (Is that the mathematical term?)
where their sum and difference do.

Kirill>
Are you referring to that when X and Y are of equal variances then r b\w X+Y and X-Y is 0, whatever r b\w X and Y?

[snip, rest of my post]

Yes.  And if X and Y are not of equal variance, you can get the same result -- that is, 
creating r=0  *exactly*  for the two derived variables -- by using weighted sum and difference.

By the way, in regard to this process:  Technically, I think of this as dealing with "confounding"
by finding un-confounded variables to use in modeling; and I think that this is better terminology
for statisticians than "interaction terms."    "Interaction" tends to be reserved by statisticians
for terms computed as X*Y.

I do have some sympathy for the common-sense collapsing of confounding and interaction,
which is how I recognized the question in the first place.  - Consider that X*Y  is expressed as
a sum after you take logs, and X/Y  is X*(1/Y), which is a difference after logs.  So when
arbitrary re-scaling is available, "fixing" a model may look either like dealing with interaction
(multiplication) or with confounding (sum and difference).

--
Rich Ulrich

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD