Scale Validity
Locked
 
|
Dear all,
is there any default process to check the validity of a scale? Many thanks. Tolis Linardis |
|
|
Since validity depends upon the purpose of the measure, the answer must
be no. You need to consider how the scale will be used to properly address validity. Paul R. Swank, Ph.D. Professor Director of Reseach Children's Learning Institute University of Texas Health Science Center-Houston -----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Tolis Linardis Sent: Wednesday, June 20, 2007 2:47 AM To: [hidden email] Subject: Scale Validity Dear all, is there any default process to check the validity of a scale? Many thanks. Tolis Linardis |
|
|
Theoretical maximum validity is the square root of your reliability measure. So if your scale reliability score is, e.g. Cronbach's Alpha = .81, then your maximum validity would be .9.
King Douglas American Airlines Customer Research "Swank, Paul R" <[hidden email]> wrote: Since validity depends upon the purpose of the measure, the answer must be no. You need to consider how the scale will be used to properly address validity. Paul R. Swank, Ph.D. Professor Director of Reseach Children's Learning Institute University of Texas Health Science Center-Houston -----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Tolis Linardis Sent: Wednesday, June 20, 2007 2:47 AM To: [hidden email] Subject: Scale Validity Dear all, is there any default process to check the validity of a scale? Many thanks. Tolis Linardis |
|
|
That's only useful with concurrent validity or maybe predictive.
Although, if you have convergent validity coefficients approaching the maximum then you have problems. But it also depends upon the reliability of the criterion. The original question was about a default process for establishing validity. Not all validity processes even involve correlations. Paul R. Swank, Ph.D. Professor Director of Reseach Children's Learning Institute University of Texas Health Science Center-Houston ________________________________ From: King Douglas [mailto:[hidden email]] Sent: Wednesday, June 20, 2007 3:28 PM To: Swank, Paul R; [hidden email] Subject: Re: Scale Validity Theoretical maximum validity is the square root of your reliability measure. So if your scale reliability score is, e.g. Cronbach's Alpha = .81, then your maximum validity would be .9. King Douglas American Airlines Customer Research "Swank, Paul R" <[hidden email]> wrote: Since validity depends upon the purpose of the measure, the answer must be no. You need to consider how the scale will be used to properly address validity. Paul R. Swank, Ph.D. Professor Director of Reseach Children's Learning Institute University of Texas Health Science Center-Houston -----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Tolis Linardis Sent: Wednesday, June 20, 2007 2:47 AM To: [hidden email] Subject: Scale Validity Dear all, is there any default process to check the validity of a scale? Many thanks. Tolis Linardis |
|
|
In reply to this post by King Douglas
Both Paul Swank and King Douglas are right, IMHO, from different
points of view and with different meanings of the word "validity". King refers to the internal and statistical meaning of validity in the jargon arising from reliability analysis, in which case it refers to the whole scale measuring a single concept, whereas Paul refers to an external and substantive criterion of validity in relation to the purpose of research and the content of the scale; in this case validity means measuring what you intend to measure. Personally, I find this latter meaning more important and telling, while the former issue I prefer to address through the concept of reliability, but both uses of the word are common. Hector -----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of King Douglas Sent: 20 June 2007 17:28 To: [hidden email] Subject: Re: Scale Validity Theoretical maximum validity is the square root of your reliability measure. So if your scale reliability score is, e.g. Cronbach's Alpha = .81, then your maximum validity would be .9. King Douglas American Airlines Customer Research "Swank, Paul R" <[hidden email]> wrote: Since validity depends upon the purpose of the measure, the answer must be no. You need to consider how the scale will be used to properly address validity. Paul R. Swank, Ph.D. Professor Director of Reseach Children's Learning Institute University of Texas Health Science Center-Houston -----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Tolis Linardis Sent: Wednesday, June 20, 2007 2:47 AM To: [hidden email] Subject: Scale Validity Dear all, is there any default process to check the validity of a scale? Many thanks. Tolis Linardis |
|
|
In reply to this post by Swank, Paul R
Paul,
I learn something new here every day. Thanks! King "Swank, Paul R" <[hidden email]> wrote: That's only useful with concurrent validity or maybe predictive. Although, if you have convergent validity coefficients approaching the maximum then you have problems. But it also depends upon the reliability of the criterion. The original question was about a default process for establishing validity. Not all validity processes even involve correlations. Paul R. Swank, Ph.D. Professor Director of Reseach Children's Learning Institute University of Texas Health Science Center-Houston --------------------------------- From: King Douglas [mailto:[hidden email]] Sent: Wednesday, June 20, 2007 3:28 PM To: Swank, Paul R; [hidden email] Subject: Re: Scale Validity Theoretical maximum validity is the square root of your reliability measure. So if your scale reliability score is, e.g. Cronbach's Alpha = .81, then your maximum validity would be .9. King Douglas American Airlines Customer Research "Swank, Paul R" <[hidden email]> wrote: Since validity depends upon the purpose of the measure, the answer must be no. You need to consider how the scale will be used to properly address validity. Paul R. Swank, Ph.D. Professor Director of Reseach Children's Learning Institute University of Texas Health Science Center-Houston -----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Tolis Linardis Sent: Wednesday, June 20, 2007 2:47 AM To: [hidden email] Subject: Scale Validity Dear all, is there any default process to check the validity of a scale? Many thanks. Tolis Linardis |
|
|
In reply to this post by Hector Maletta
Thank you all for the useful infromation.
Having a look at different papers for scale validity, there is a distinction between: a) surface validity b) content validity and c) construct validity. Concerning the first two validity types, I detected that there is no statistical process. In papers they usually say that content validity for example "was confirmed by specialists"... In other papers they don't even refer to these types of validity.. Regarding construct validity I detected an approach that is in accordance with Paul's option... "You need to consider how the scale will be used to properly address validity" ... that is really a subjective matter.. The usual method met, is to estimate the correlation coefficients of an aggregated variable with other variables and if there is an expected statistically significant correlation then they usually say that construct validity holds... Tolis Linardis. ----- Original Message ----- From: "Hector Maletta" <[hidden email]> To: <[hidden email]> Sent: Wednesday, June 20, 2007 11:51 PM Subject: Re: Scale Validity > Both Paul Swank and King Douglas are right, IMHO, from different > points of view and with different meanings of the word "validity". King > refers to the internal and statistical meaning of validity in the jargon > arising from reliability analysis, in which case it refers to the whole > scale measuring a single concept, whereas Paul refers to an external and > substantive criterion of validity in relation to the purpose of research > and > the content of the scale; in this case validity means measuring what you > intend to measure. Personally, I find this latter meaning more important > and > telling, while the former issue I prefer to address through the concept of > reliability, but both uses of the word are common. > > Hector > > > -----Original Message----- > From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of > King Douglas > Sent: 20 June 2007 17:28 > To: [hidden email] > Subject: Re: Scale Validity > > Theoretical maximum validity is the square root of your > reliability > measure. So if your scale reliability score is, e.g. Cronbach's Alpha = > .81, then your maximum validity would be .9. > > > King Douglas > American Airlines Customer Research > > "Swank, Paul R" <[hidden email]> wrote: Since validity > depends upon the purpose of the measure, the answer must > be no. You need to consider how the scale will be used to properly > address validity. > > > Paul R. Swank, Ph.D. Professor > Director of Reseach > Children's Learning Institute > University of Texas Health Science Center-Houston > > -----Original Message----- > From: SPSSX(r) Discussion [mailto:[hidden email]] On > Behalf Of > Tolis Linardis > Sent: Wednesday, June 20, 2007 2:47 AM > To: [hidden email] > Subject: Scale Validity > > Dear all, > > is there any default process to check the validity of a scale? > Many thanks. > > Tolis Linardis > > |
|
|
The major problem with the way most people report validity evidence (and
it is eevidence of validity, not validity per se) fails to recognize the purpose of the instrument. "Just correlate it with something and if the correlation is significant, the test is valid." I susapect your "surface" validity is another term for "face" validity which is not validity at all. It merely determines whether the instrument looks valid to those who take it. Many psychological tests have no face validity because the purpose of the test remains hidden. Content validity is, as you note, not usually documented with coefficients but more a matter of expert opinion. Construct validity includes a whole host of methods that are indirect sources of evidence for validity. For example, some researchers consider confirmatory factor analysis to be construct validity. Of course, as you say, convergent validity is most common. Correlating a measure with another measure which assesses a different but correlated trait. However, this is relatively weak evidence by itself since, for example, a spelling test might be correalted with IQ but that doesn't establich that the spelling test is measuring what it is supposed to be. Many constructs are correlated with IQ. The most straightforward way to obtain validity evidence is to have another measure of the same constrcut (criterion) which you can correalte the measure under consideration with. The fact that the Wechsler scales correlate strongly with the Stanford-Binet gives good evidence of the validity of those scales, as the Stanford-Binet is a long established and accepted measure of IQ. Then there is also predictive validity. I could go on for hours (I usually spend at least 6 hours on validity in my doctoral measurement course). What is most important is to understand what it is you are trying to measure and then develop a plan for establishing evidence of validity by considering the purpose of the measure and how it will be used. Paul R. Swank, Ph.D. Professor Director of Reseach Children's Learning Institute University of Texas Health Science Center-Houston -----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Tolis Linardis Sent: Thursday, June 21, 2007 5:39 AM To: [hidden email] Subject: Re: Scale Validity Thank you all for the useful infromation. Having a look at different papers for scale validity, there is a distinction between: a) surface validity b) content validity and c) construct validity. Concerning the first two validity types, I detected that there is no statistical process. In papers they usually say that content validity for example "was confirmed by specialists"... In other papers they don't even refer to these types of validity.. Regarding construct validity I detected an approach that is in accordance with Paul's option... "You need to consider how the scale will be used to properly address validity" ... that is really a subjective matter.. The usual method met, is to estimate the correlation coefficients of an aggregated variable with other variables and if there is an expected statistically significant correlation then they usually say that construct validity holds... Tolis Linardis. ----- Original Message ----- From: "Hector Maletta" <[hidden email]> To: <[hidden email]> Sent: Wednesday, June 20, 2007 11:51 PM Subject: Re: Scale Validity > Both Paul Swank and King Douglas are right, IMHO, from > different points of view and with different meanings of the word > "validity". King refers to the internal and statistical meaning of > validity in the jargon arising from reliability analysis, in which > case it refers to the whole scale measuring a single concept, whereas > Paul refers to an external and substantive criterion of validity in > relation to the purpose of research and the content of the scale; in > this case validity means measuring what you intend to measure. > Personally, I find this latter meaning more important and telling, > while the former issue I prefer to address through the concept of > reliability, but both uses of the word are common. > > Hector > > > -----Original Message----- > From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf > Of King Douglas > Sent: 20 June 2007 17:28 > To: [hidden email] > Subject: Re: Scale Validity > > Theoretical maximum validity is the square root of your > reliability measure. So if your scale reliability score is, e.g. > Cronbach's Alpha = .81, then your maximum validity would be .9. > > > King Douglas > American Airlines Customer Research > > "Swank, Paul R" <[hidden email]> wrote: Since > validity depends upon the purpose of the measure, the answer must > be no. You need to consider how the scale will be used to > address validity. > > > Paul R. Swank, Ph.D. Professor > Director of Reseach > Children's Learning Institute > University of Texas Health Science Center-Houston > > -----Original Message----- > From: SPSSX(r) Discussion [mailto:[hidden email]] On > Behalf Of > Tolis Linardis > Sent: Wednesday, June 20, 2007 2:47 AM > To: [hidden email] > Subject: Scale Validity > > Dear all, > > is there any default process to check the validity of a scale? > Many thanks. > > Tolis Linardis > > |
«
Return to SPSSX Discussion
|
1 view|%1 views
| Free forum by Nabble | Edit this page |