Guidelines for reporting Scale Development and Validation Results

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Guidelines for reporting Scale Development and Validation Results

E. Bernardo
Dear Folks,

In scale development and validation, some researchers use EFA and subsequently CFA (using different sample).  This approach has clear guidelines for reporting Scale Development and Validation Results in the Journal of the Society for Social Work and Research (available at http://www.jsswr.org/article/view/6974/4882 ).

Sometime ago, there were exchanges in this list regarding the above procedure (EFA then CFA).  Some folks do not favor the use of EFA.  If you know references/guidelines for reporting scale development and validation without the EFA, please post them here.  

If you know also some guidelines for reporting scale development and validation using Rasch Model, please post them also here.

Thank you.
Eins
Reply | Threaded
Open this post in threaded view
|

Re: Guidelines for reporting Scale Development and Validation Results

Ryan
Eins,

A couple of points:

1. The standard dichotomous Rasch model is nothing more than a simple
logistic regression, parameterized without a grand intercept. In fact,
one could actually fit a Rasch model in SPSS quite easily.

2. Providing guidelines with respect to "reporting scale development
and validation" would take quite a long time.

Here's a non-exhaustive list in no particular order off the top of my
head of what I tend to examine upon fitting a Rasch model:

Assuming you are dealing with standard dichotomous Rasch model, it
would be important to assess for:
1. item fit (infit and outfit) via mean-squares
2. person fit (infit and outfit) via mean-squares
3. item reliability & separation
4. person reliability & separation
5. dimensionality via an unrotated PCA on the probability scale
residuals, or perhaps a bifactor model
6. person-item map to examine person-item targeting, gaps,
distribution of persons and items, bandwidth, etc.
7. item discriminations by fitting a 2-PL model
8. global fit (e.g., test if there is a significant improvement
between the 2-PL and Rasch)
9. item characteristic curves (ICCs)
10. Differential item function via examination of ICCs
11. item information curves
12. test information curve
13. atypical response patterns (e.g., Depression: non-endorsement of
"mild" depressive items, yet endorsement of suicidal ideation)

...and the list goes on and on...

Several of the points listed above are inter-related. This list won't
mean much to those who are unfamiliar with Rasch modeling, but I think
this is a reasonable answer given the question and the forum.

Ryan

On Thu, Dec 6, 2012 at 10:51 PM, E. Bernardo <[hidden email]> wrote:

> Dear Folks,
>
> In scale development and validation, some researchers use EFA and
> subsequently CFA (using different sample).  This approach has clear
> guidelines for reporting Scale Development and Validation Results in the
> Journal of the Society for Social Work and Research (available at
> http://www.jsswr.org/article/view/6974/4882 ).
>
> Sometime ago, there were exchanges in this list regarding the above
> procedure (EFA then CFA).  Some folks do not favor the use of EFA.  If you
> know references/guidelines for reporting scale development and validation
> without the EFA, please post them here.
>
> If you know also some guidelines for reporting scale development and
> validation using Rasch Model, please post them also here.
>
> Thank you.
> Eins

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD