http://spssx-discussion.165.s1.nabble.com/Pretest-to-Posttest-A-question-of-reliability-tp5717078p5717094.html
"... proceed to correlate F1 and F2"? "... measurement problem"?
Please read my previous post more closely -- You should START by
correlating F1 and F2, to see if there is any consistency (Reliability
of one sort) between the two times. As I said.
IS there is a "measurement problem" -- which I would more likely call
a "scale development problem"? If so, what it affects is what you can say
about your results so far. What is the narrative going to be? Is this to
be regarded as a pilot study, which mainly reveals problems in the scale?
You say that you have looked at which items are "loaded significantly."
Okay. I haven't used AMOS, but I assume that this is what I referred to,
explicitly, as testing the item vs corrected item-total correlations. But
"significant" is not an effect size or measure of association... and that is
why I said, also explicitly, that the lack could indicate tiny N or disparate
items. You have yet to mention the N - Even small correlations could be
"significant" if the N is large enough. Chronbach's alpha is a measure
using the average correlation, and it tells you something about how well
the items do hang together. The procedure will also give you "alpha if
item is deleted". If that one goes UP for some item, then you know that
this item is hurting the internal consistency of the set of items.
A composite score does not *have* to have similar, correlated items in
order to be useful. But you do call this one a latent factor. And you
do describe it as comprising 10 likert items: I don't expect that 10
items on a symmetric scale of "How much do you agree" would be used
without some notion of latent factors. However, an important aspect of
"reliability" is that it is always "reliability in THIS sample." That implies,
for instance, that you expect higher measured reliability when the sample
is diverse on the latent factor; and lower when the sample is pre-selected
as being similar.
- Thus, for data I have used, I expect a stronger factor structure among
patients' symptoms at PRE, and a weaker structure at POST, corresponding
to the greater/fewer symptoms present.
- By similar logic, for a long, active educational intervention on a classroom,
I might expect more structure at POST, after the intervention shows an effect.
--
Rich Ulrich
Date: Fri, 21 Dec 2012 22:35:47 +0800
From:
[hidden email]Subject: Re: Pretest to Posttest: A question of reliability
To:
[hidden email]Dear RB, Ulrich, et al.
Sorry for the poor English.
Let me rephrase the scenario. Our variable is latent with 10
items. The ten-item five point likert type questionnaire were administered to a sample in two different occasions, F1 then F2. So we have two correlated factors F1 and F2 and we want to treat them as latent variables using AMOS 20. We expect that the 10 items are loaded significantly (p<.0.05) to F1 and then to F2. However, the actual data showed that some items have insignificant (p>.05) factor loadings on F1 and F2; thus, the set of items that loaded to F1 are different from the set of items that loaded to F2. Our question was: Can we proceed to correlate F1 and F2. Is this not a measurement problem?
Eins
... snip, previous