Handling Don't Know Responses in Scale Construction

[Hannah State-Davey]

I am currently developing an attitude scale and have collected data for
initial item reduction. The scale uses a 7-point likert scale (1=strongly
disagree, 2=disagree, 3=slightly disagree, 4=neutral, 5=slightly agree,
6=agree, 7=strongly agree).  I also included a 'Dont Know' category to
filter out any items that respondents consistently could not answer. My
problem is how to handle these 'Don't Know' responses correctly. It
clearly provides an 'attitude' and so is important information that should
be included in analysis. There are a number of options that I have come up
with:

1) to treat DK responses as missing values and replace with mean, EM or
such like. I feel this option distorts the data somewhat.
2) to collapse DK responses into the mid-point (i.e. neutral) - despite
neutral and DK having different meanings. This is an option that I have
seen several researchers use.
3) to treat DK responses as a valid response in the scale. The problem
here is that they are of a different charachter to the 1-7 scale. This
presumably creates problems when you come to factor analysis??

If anyone can provide any advice on this i would be really grateful?

Many thanks

Hannah

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[George_L]

The way I see it, there are two possible outcomes: people who know and people who don't know. One solution would be to create a dummy variable with these two pieces of information and use this dummy in future calculations. I would not replace the "don't knows" with means but treat them as missings.

Hope this helps,
G.

Hannah State-Davey wrote

I am currently developing an attitude scale and have collected data for
initial item reduction. The scale uses a 7-point likert scale (1=strongly
disagree, 2=disagree, 3=slightly disagree, 4=neutral, 5=slightly agree,
6=agree, 7=strongly agree).  I also included a 'Dont Know' category to
filter out any items that respondents consistently could not answer. My
problem is how to handle these 'Don't Know' responses correctly. It
clearly provides an 'attitude' and so is important information that should
be included in analysis. There are a number of options that I have come up
with:

1) to treat DK responses as missing values and replace with mean, EM or
such like. I feel this option distorts the data somewhat.
2) to collapse DK responses into the mid-point (i.e. neutral) - despite
neutral and DK having different meanings. This is an option that I have
seen several researchers use.
3) to treat DK responses as a valid response in the scale. The problem
here is that they are of a different charachter to the 1-7 scale. This
presumably creates problems when you come to factor analysis??

If anyone can provide any advice on this i would be really grateful?

Many thanks

Hannah

=====================
To manage your subscription to SPSSX-L, send a message to
[e-mail] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[bdates]

Hannah,

There is a lot of information on how to treat don't know responses in
Likert-type scales, at least 90% of which indicates that don't know's
should be treated as missing data.  IF you decide to impute values for
the missing data DON'T use mean or hot deck or regression methods.
These approaches, while all we had years ago, attenuate the variance and
create problems with interpreting the results.  Use an MLE (EM) approach
or an MI approach.  There is free software available (NORM) which will
do this for you.  But if you decide to impute values, read about the
types of missing data and the extent to which imputation can provide
valid data.

Brian

View this message in context:
http://www.nabble.com/Handling-Don%27t-Know-Responses-in-Scale-Construct
ion-tp17642408p17643466.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[David Futrell]

Hannah,
They should be treated as "missing" data. The respondents had the option of choosing the mid-point (i.e., neutral) point on the scale, but chose "don't know" instead. Usually, this means they didn't have enough information to answer the question.

This probably means something qualitatively different from "neutral" which might imply something like ambivalence (i.e., being torn between the ends of the scale) or apathy (not caring).

David Futrell, Ph.D.
Senior Workforce Research Consultant
Eli Lilly and Company


Hannah State-Davey <[hidden email]> wrote: I am currently developing an attitude scale and have collected data for
initial item reduction. The scale uses a 7-point likert scale (1=strongly
disagree, 2=disagree, 3=slightly disagree, 4=neutral, 5=slightly agree,
6=agree, 7=strongly agree).  I also included a 'Dont Know' category to
filter out any items that respondents consistently could not answer. My
problem is how to handle these 'Don't Know' responses correctly. It
clearly provides an 'attitude' and so is important information that should
be included in analysis. There are a number of options that I have come up
with:

1) to treat DK responses as missing values and replace with mean, EM or
such like. I feel this option distorts the data somewhat.
2) to collapse DK responses into the mid-point (i.e. neutral) - despite
neutral and DK having different meanings. This is an option that I have
seen several researchers use.
3) to treat DK responses as a valid response in the scale. The problem
here is that they are of a different charachter to the 1-7 scale. This
presumably creates problems when you come to factor analysis??

If anyone can provide any advice on this i would be really grateful?

Many thanks

Hannah

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bob Dozier]

Are you saying you ask the question as an 8 point scale and included DK
in the scale? If you have DK as a separate question then I do not see
the problem. Even if it is in the scale I think you will have to treat
it as two questions and use the DK as a qualifier for the attitude
scale. If this had been an interactive survey you could have ask the
"know" question first and only present the 7 scale if the answer was "do
know." This of course assumes that people do not have an opinion about
things they don't "know" about which in my experience is a false
assumption but that is a discussion for another place.

Thanks
bob dozier

Confidentiality Notice:  This message, including any attachments, is
for the sole use of the individual or entity named above.  The message
may contain confidential health and/or legally privileged information.
If you are not the above-named recipient, you are hereby notified that
any disclosure, copying distribution, or action taken in reliance on the
contents of this message is strictly prohibited.  If you have received
this message in error, please notify the sender at 558-3956 immediately
and destroy all copies of the original message.

>>> Hannah State-Davey <[hidden email]> 6/4/2008 5:06 AM >>>
I am currently developing an attitude scale and have collected data
for
initial item reduction. The scale uses a 7-point likert scale
(1=strongly
disagree, 2=disagree, 3=slightly disagree, 4=neutral, 5=slightly
agree,
6=agree, 7=strongly agree).  I also included a 'Dont Know' category to
filter out any items that respondents consistently could not answer.
My
problem is how to handle these 'Don't Know' responses correctly. It
clearly provides an 'attitude' and so is important information that
should
be included in analysis. There are a number of options that I have come
up
with:

1) to treat DK responses as missing values and replace with mean, EM
or
such like. I feel this option distorts the data somewhat.
2) to collapse DK responses into the mid-point (i.e. neutral) -
despite
neutral and DK having different meanings. This is an option that I
have
seen several researchers use.
3) to treat DK responses as a valid response in the scale. The problem
here is that they are of a different charachter to the 1-7 scale. This
presumably creates problems when you come to factor analysis??

If anyone can provide any advice on this i would be really grateful?

Many thanks

Hannah

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except
the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Richard Ristow]

At 06:17 AM 6/4/2008, George_L wrote:

>One solution would be to create a dummy variable [for "don't know]
>and use this dummy in future calculations. I would not replace the
>"don't knows" with means but treat them as missings.

I assume this means as independent variables in methods like linear regression.

As a matter of how SPSS handles data, you shouldn't make the scale
values missing for the 'don't know' cases; that would exclude them
from the analysis altogether (using listwise deletion, as you should).

Assign a fixed arbitrary response for the scale value, when the
response is 'don't know'. I'd use 0. But: this is valid *only* when
the dummy for "don't know" is also used in the analysis.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Richard Ristow]

At 05:29 AM 6/6/2008, Hannah Clarke wrote, off-list:

>>One solution would be to create a dummy variable [for "don't know]

>How would you create the dummy variable and what would this be? Can
>you give me an example?
>
>I have come across this approach being suggested in various texts
>but cannot quite get my head round how to actually do it in practice.

Suppose your scale variable is Likert, coded as you said at first, i.e.
>1=strongly disagree, 2=disagree, 3=slightly disagree, 4=neutral,
>5=slightly agree,    6=agree,    7=strongly agree
plus 9="Don't know".

Then, create two new variables as follows (not tested):

NUMERIC LkrtVal LkrtDK (F2).
VAR LABELS
           LkrtVal 'Value of "Likert", if not "don''t know"'
           LkrtDK  'Dummy: indicates "Likert" = "don''t know"'.

RECODE  Likert
     (9    = 0)   /* For interpretabality, "DK" is now 0 */
     (ELSE = COPY)
      INTO LkrtVal.
RECODE  Likert
     (9    = 1)
     (ELSE = 0)
      INTO LkrtDK.

VAL LABELS
           LkrtDK   1 "Don't know"  0 "Response".

If you then use LkrtVal as an independent variable, you *must* use
LkrtDK as well. (However, there could be reasons to use LkrtDK alone.)

>Can this approach be used in factor analysis and how?

I should think it could, but I'm not a factor-analysis maven. To
repeat, always include the dummy LkrtDK if you use LkrtVal.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD