EFA or CFA?

[Muir Houston-3]

Listers,
Looking for some advice in relation to exploratory or confirmatory
factor analysis. The study - 6000 school pupils completed a Motivation
to Learn inventory - the instrument has been subject to previous
research by the inventory designers into validity, reliability, internal
consistency, etc. and proposes a 7 factor structure.

Now my question - should I carry out EFA or CFA? I carried out an EFA
(principal components) and could not replicate the 7 factor structure -
it suggested a four factor structure (it is perhaps worth mentioning
that this is the largest sample size to use this inventory according to
a trawl of the relevant research)

It has been suggested however, that I should carry out CFA using AMOS as
I want to confirm the factor structure.

Any advice or suggestions welcome

Thanks

Muir

Muir Houston, HNC, BA (Hons), M.Phil., PhD, FHEA
Research Fellow
School of Education
University of Glasgow
0044+141-330-4699

R3L+ Project - Adult education in the light of the European Quality
Strategy
http://www.learning-regions.net/

GINCO Project - Grundtvig International Network of Course Organisers
http://www.ginconet.eu/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Mike]

A few points:

(1)  A principal components analysis is different from a factor analysis.
A principal components analysis attempts to account for total variance
among a group of variables by reducing the original variable to a smaller
number of independent components.  Factor analysis attempts to explain
the observed correlation/covariance structure among variables by identifying
a smaller number of factors that can reproduce the correlations.  It should
not be surprising that the two might lead to different results and/or
conclusions.

(2) If your sample is very different from the samples used in other studies,
then you can ask whether the factors present are the same as those found
previously or differ in a variety of ways (e.g., number of factors, loadings
of variable on factors, etc.).  To answer this question you could use one
of two strategies:

(a) Purely exploratory factor analysis:  let the factor analysis program
try to determine the number of factors under the standard assumptions.
However, I would suggest using either maximum likelihood or generalized
least square factor analyses because these will give a goodness of fit
test for how well the extracted model fits your data.  Do not be surprised
if the goodness of fit test (usually a chi-square statistic) is statistically
significant, that is, there is a significant discrepancy between the factor
model and the observed data.  This is likely to be due to a violation of
one or more assumptions (e.g., uncorrelated error/unique variances).
It is also possible that the data cannot be modeled by a factor model.

(b) Confirmatory factor analysis:  previous results can guide you on
the number of factors one should find with your instrument, whether
the factors are orthogonal or correlated, whether an empirical
variable loads on one or two/more factors, whether errors are
correlated, etc.  If the previous research reported a goodness of fit
statistics, check to see if it was statistically significant or not.  Some
folks may report a statistically significant goodness of fit statistics
and not say anything more even though this statistic tells one that the
model one has does not fit the data (i.e., the model has to be
modified in order to make the goodness of fit statistic nonsignificant,
that is, that the factor model you have is a good fit to your data).
If the previous research reports a nonsignificant goodness of fit
statistic, then I would suggest replicating the original analysis with
your data.  If you are lucky, you will obtain similar results.  If you
get different results, such as the goodness of fit statistic is significant
in your sample, then you have to figure out why this is the case.  Is the
factor model incorrectly specified?  Does your sample come from
a different population than the previous research participants and
a different factor model hold for them?  And so on.

In summary, if you think that the previous research is good, then
use CFA to determine whether the previous model holds in your
data.  If you think that the previous research is unreliable, then
you might want to start with EFA.

But, really, it is easy enough to do both.  The only question is can
you explain to others why you chose one model over another.

-Mike Palij
New York University
[hidden email]



----- Original Message -----
From: "Muir Houston" <[hidden email]>
To: <[hidden email]>
Sent: Thursday, February 17, 2011 7:07 AM
Subject: EFA or CFA?

> Listers,
> Looking for some advice in relation to exploratory or confirmatory
> factor analysis. The study - 6000 school pupils completed a Motivation
> to Learn inventory - the instrument has been subject to previous
> research by the inventory designers into validity, reliability, internal
> consistency, etc. and proposes a 7 factor structure.
>
> Now my question - should I carry out EFA or CFA? I carried out an EFA
> (principal components) and could not replicate the 7 factor structure -
> it suggested a four factor structure (it is perhaps worth mentioning
> that this is the largest sample size to use this inventory according to
> a trawl of the relevant research)
>
> It has been suggested however, that I should carry out CFA using AMOS as
> I want to confirm the factor structure.
>
> Any advice or suggestions welcome
>
> Thanks
>
> Muir
>
> Muir Houston, HNC, BA (Hons), M.Phil., PhD, FHEA
> Research Fellow
> School of Education
> University of Glasgow
> 0044+141-330-4699
>
> R3L+ Project - Adult education in the light of the European Quality
> Strategy
> http://www.learning-regions.net/
>
> GINCO Project - Grundtvig International Network of Course Organisers
> http://www.ginconet.eu/
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Art Kendall]

***You may want to check what kind of factor analysis did the previous studies use?
The principal components kind does dimension reduction on the total variance: common, unique, and error. (1.00 on the diagonal)
The principal  axes kind does dimension reduction on the common variance,  unique is pooled with error. (Communality estimates SMCs on the diagonal.) This is the conventional kind in test/scale construction.

***What did they use as a stopping rule?
I suggest that you compare your eigenvalues to those from a parallel factor analysis. 

https://people.ok.ubc.ca/brioconn/nfactors/nfactors.html

SPSS reports the eigenvalues only on the the total variance.  The parallel analysis syntax reports those from the matrix with communality measures on the diagonal.

Last week I ran a parallel analysis on 14 variables and 40,000 cases with 1000 random permutations of the values across cases and it took about 50 minutes.

You might also run parallel analyses on random data the sizes that the previous studies did and see how their eigenvalues compare.

***In test/scale construction in order to maximize divergent validity I suggest the use of varimax rotations and retaining only items with clean loadings.

When you have your new scoring key, comparing it to that provided by the developers may give you some insight into the differences.
Might there be development or cultural differences in denotations and connotations in the question vocabulary. 

If you have access to AMOS or other SEM software why not run it  EFA, and CFA with the previous key, and CFA with your key?  Then compare and contrast the keys semantically.  After all almost all of the effort is in gathering, cleaning, and exploring the data.

Art Kendall
Social Research Consultants


On 2/17/2011 7:07 AM, Muir Houston wrote:

Listers,
Looking for some advice in relation to exploratory or confirmatory
factor analysis. The study - 6000 school pupils completed a Motivation
to Learn inventory - the instrument has been subject to previous
research by the inventory designers into validity, reliability, internal
consistency, etc. and proposes a 7 factor structure.

Now my question - should I carry out EFA or CFA? I carried out an EFA
(principal components) and could not replicate the 7 factor structure -
it suggested a four factor structure (it is perhaps worth mentioning
that this is the largest sample size to use this inventory according to
a trawl of the relevant research)

It has been suggested however, that I should carry out CFA using AMOS as
I want to confirm the factor structure.

Any advice or suggestions welcome

Thanks

Muir

Muir Houston, HNC, BA (Hons), M.Phil., PhD, FHEA
Research Fellow
School of Education
University of Glasgow
0044+141-330-4699

R3L+ Project - Adult education in the light of the European Quality
Strategy
http://www.learning-regions.net/

GINCO Project - Grundtvig International Network of Course Organisers
http://www.ginconet.eu/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD