reliability of a questionnaire and missing values in items

[Doris-18]

Hi,

to check internal consistency of a questionnaire I use cronbach`s alpha -
the problem I have is the missings in items. As a consequence cronbachs
alpha could not be calculatedbecause all cases were deleted.

Looking on the syntax I saw that there is the possibility to
define "MISSING=Include" in the reliability syntax. Rerunning the
procedure yielded a considerable alpha (about 0.80).

My question: does the statement "MISSING=Include" means that the procedure
is set to pairwise deletion of missing data? (with other words the default
is listwise deletion in the reliability statement)

Many thanks for your answer, Doris

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[John F Hall]

Send more info:

 

Your syntax

Your questionnaire items

Your frequencies for these items

Your correlation matrix.

----- Original Message -----

From: [hidden email]

To: [hidden email]

Sent: Wednesday, May 05, 2010 11:56 AM

Subject: reliability of a questionnaire and missing values in items

Hi,

to check internal consistency of a questionnaire I use cronbach`s alpha -
the problem I have is the missings in items. As a consequence cronbachs
alpha could not be calculatedbecause all cases were deleted.

Looking on the syntax I saw that there is the possibility to
define "MISSING=Include" in the reliability syntax. Rerunning the
procedure yielded a considerable alpha (about 0.80).

My question: does the statement "MISSING=Include" means that the procedure
is set to pairwise deletion of missing data? (with other words the default
is listwise deletion in the reliability statement)

Many thanks for your answer, Doris

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Mike]

A few points:

(1) A procedure like reliability assumes that you have complete
data for all cases, this is why the Reliability procedure will not
run unless you have "missing=include" specified.

(2) Before even considering using reliability on a dataset in
which everyone appears to have at least one missing value,
you should ask yourself why the values are missing and is there
a systematic pattern to which items are missing (e.g., items
dealing with sensitive issues like drug use, criminal behavior,
sexcual behavior, etc., may not be answered).

(3)  You should probably use the SPSS missing values procedure
to examine the data though I have not used this procedure myself.
You should determine whether any items have complete data (i.e.,
everyone answered those questions) and what is the percentage
of missing values for items that have them.  Any items that have
many missing values (what defines many is subjective; some might
say anything greater than 10% missing is too much while others
might anything greater than 50% is too much and such items should
be excluded from analysis -- you should be able to defend your
decision as to which items to keep and which to drop).

Without this kind of information it is hard to recommend what you
should do outside of approaching data analyses very cautiously.

-Mike Palij
New York University
[hidden email]



----- Original Message -----
From: "Doris Gerstner" <[hidden email]>
To: <[hidden email]>
Sent: Wednesday, May 05, 2010 5:56 AM
Subject: reliability of a questionnaire and missing values in items

> Hi,
>
> to check internal consistency of a questionnaire I use cronbach`s alpha -
> the problem I have is the missings in items. As a consequence cronbachs
> alpha could not be calculatedbecause all cases were deleted.
>
> Looking on the syntax I saw that there is the possibility to
> define "MISSING=Include" in the reliability syntax. Rerunning the
> procedure yielded a considerable alpha (about 0.80).
>
> My question: does the statement "MISSING=Include" means that the procedure
> is set to pairwise deletion of missing data? (with other words the default
> is listwise deletion in the reliability statement)
>
> Many thanks for your answer, Doris

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[statisticsdoc]

Doris,

You have no cases with complete data using listwise deletion, so I would
first ask where the missing data comes from.  Are there specific items
that are answered by a relatively small proportion of your sample?  An
individual item can have a small response rate for a number of reasons:
the item might deal with sensitive material. the item might be part of a
skip pattern such that only some of the subjects are supposed to answer
it, there might have been a typo on the questionnaire so the item made
no sense to subjects and was answered only by a few who were not paying
attention or who guessed at the meaning of the item, the data entry
program or scanning equipment did not work properly for a specific set
of items, etc.  Another possibility is that you have a moderate response
rate per item, and a very long questionnaire, so that every subject
misses at least one item.  If the problem is limited to a small number
of items, then the solution is simpler (drop the items from the
analysis,  re-enter the data for the items, etc).

You can use SPSS Missing Values to see whether the missing data follows
a pattern or occurs completely at random, but I would first look at the
basis response rates for items to see if a handful of items are the
source of the problem.

Best,

Stephen Brand

www.StatisticsDoc.com


On Wed, May 5, 2010 at 5:56 AM, Doris Gerstner wrote:

> Hi,
>
> to check internal consistency of a questionnaire I use cronbach`s
> alpha -
> the problem I have is the missings in items. As a consequence
> cronbachs
> alpha could not be calculatedbecause all cases were deleted.
>
> Looking on the syntax I saw that there is the possibility to
> define "MISSING=Include" in the reliability syntax. Rerunning the
> procedure yielded a considerable alpha (about 0.80).
>
> My question: does the statement "MISSING=Include" means that the
> procedure
> is set to pairwise deletion of missing data? (with other words the
> default
> is listwise deletion in the reliability statement)
>
> Many thanks for your answer, Doris
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except
> the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Ruben Geert van den Berg]

Dear Doris,

To address your original question as well: MISSING=INCLUDE does NOT mean pairwise (rather than listwise) deletion. See the command syntax reference (p. 1517 for version 15) for an explanation of the subcommand:

MISSING Subcommand

MISSING controls the deletion of cases with user-missing data.

?? RELIABILITY deletes cases from analysis if they have a missing value for any variable

named on the VARIABLES subcommand. By default, both system-missing and user-missing

values are excluded.

EXCLUDE Exclude user-missing and system-missing values. This is the default.

INCLUDE Treat user-missing values as valid. Only system-missing values are excluded.

Now say you have Likert scales like:

1: Very dissatisfied

2

3

4

5: Very satisfied

6: Not applicable/don't want to answer

If you want to do any calculations (like Cronbach's alpha) with these answers, you should specify '6' as user missing. Since higher values imply a more positive feeling in your respondents, '6' will mean even more satisfied than '5: very satisfied' if not specified as missing. MISSING=INCLUDE would also consider '6' as a more positive answer than '5' when reliability is estimated and -at least for the example above- this clearly doesn't make sense.

HTH,

Ruben van den Berg

Methodologist

TNS NIPO

E: [hidden email]

P: [phone]

I: www.tns-nipo.com

> Date: Wed, 5 May 2010 05:56:12 -0400
> From: [hidden email]
> Subject: reliability of a questionnaire and missing values in items
> To: [hidden email]
>
> Hi,
>
> to check internal consistency of a questionnaire I use cronbach`s alpha -
> the problem I have is the missings in items. As a consequence cronbachs
> alpha could not be calculatedbecause all cases were deleted.
>
> Looking on the syntax I saw that there is the possibility to
> define "MISSING=Include" in the reliability syntax. Rerunning the
> procedure yielded a considerable alpha (about 0.80).
>
> My question: does the statement "MISSING=Include" means that the procedure
> is set to pairwise deletion of missing data? (with other words the default
> is listwise deletion in the reliability statement)
>
> Many thanks for your answer, Doris
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD

---

New Windows 7: Find the right PC for you. Learn more.

[Genevieve Odoom]

Hello,
Thank you for your email. I will be out of the office on Thursday, May 6th, returning on Friday, May 7th and will respond to your email upon my return.

Thanks!
Genevieve Odoom
Policy and Program Analyst
OANHSS
Suite 700 - 7050 Weston Rd. Woodbridge,
ON L4L 8G7
Tel: [phone] x 241 Fax: [phone]
[hidden email]
www.oanhss.org

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Ruben Geert van den Berg]

Dear Doris,

I'm about 99% sure there's no pairwise deletion available for Cronbach's alpha in SPSS. Because it would otherwise have been documented.

If your variables have been coded 0/1/99 then (MISSING=INCLUDE) in RELIABILITY will turn out way worse than the example with 5 point Likert scales. This is because 99 would be considered an outlier as well as an influential data point for this case. For a demonstration, please run the syntax underneath this e-mail. Something like this could have caused your reliability to exceed .80 if you actually used (MISSING=INCLUDE).

A practical approach could be this: copy the variables needed for the reliability analysis into a new dataset/-file. Then see if any variables have very high percentages missing (use DESCRIPTIVES for this, it will also tell you quickly whether all variables really contain only 0 and 1 as valid values). Then see whether any respondents have high numbers of missings with the count function (count items_skipped=v1 to v25 (mis)). Remove any respondents/variables with excessive missingness. Calculate the overall percentage of missing data points. This should preferably be < 10%. Now impute the remaining missing values and calculate the reliability.

And please keep in mind that a high Cronbach's alpha does not imply that test scores will be stable over time (test-retest reliability).

HTH,

Ruben van den Berg

Methodologist

TNS NIPO

E: [hidden email]

P: [phone]

I: www.tns-nipo.com

*Create fake data. Just one out of 50 respondents decided not to cooperate with the test.

input program.

loop id=1 to 50.

end case.

end loop.

end file.

end input program.

do repeat v=v1 to v10.

compute v=rnd(rv.uni(-.5,1.5)).

if $casenum=1 v=99.

end repeat.

*I INCLUDE the missing values as valid and just look at the correlations!!.

correlations v1 to v10.

*Now we correctly specify '99' as missing. 

missing values v1 to v10(99).

The correlations are mostly insignificant since these data are random.

correlations v1 to v10.

---

From: [hidden email]
To: [hidden email]
Date: Thu, 6 May 2010 20:15:35 +0200
Subject: AW: reliability of a questionnaire and missing values in items

Dear Ruben,

 

thanks for your answer - but please let me ask you one thing again (referring to your example):

if I specify '6' as user missing and run the reliability syntax with "missing=include", spss treats these values as valid: does this mean '6' would be treated as the highest category? clearly, this would be wrong! So far, I always suggested a pairwise deletion using this option. In the questionnaire used in our study we defined missing values by '99'; all questions can be answered by 0/1 (yes/no).  

 

Is there a possibility in spss to calculate Cronbachs alpha using pairwise deletion?

 

Best, Doris

 

---

Von: SPSSX(r) Discussion [[hidden email]] im Auftrag von Ruben van den Berg [[hidden email]]
Gesendet: Donnerstag, 6. Mai 2010 10:00
An: [hidden email]
Betreff: Re: reliability of a questionnaire and missing values in items

Dear Doris,

To address your original question as well: MISSING=INCLUDE does NOT mean pairwise (rather than listwise) deletion. See the command syntax reference (p. 1517 for version 15) for an explanation of the subcommand:

MISSING Subcommand

MISSING controls the deletion of cases with user-missing data.

?? RELIABILITY deletes cases from analysis if they have a missing value for any variable

named on the VARIABLES subcommand. By default, both system-missing and user-missing

values are excluded.

EXCLUDE Exclude user-missing and system-missing values. This is the default.

INCLUDE Treat user-missing values as valid. Only system-missing values are excluded.

Now say you have Likert scales like:

1: Very dissatisfied

2

3

4

5: Very satisfied

6: Not applicable/don't want to answer

If you want to do any calculations (like Cronbach's alpha) with these answers, you should specify '6' as user missing. Since higher values imply a more positive feeling in your respondents, '6' will mean even more satisfied than '5: very satisfied' if not specified as missing. MISSING=INCLUDE would also consider '6' as a more positive answer than '5' when reliability is estimated and -at least for the example above- this clearly doesn't make sense.

HTH,

Ruben van den Berg

Methodologist

TNS NIPO

E: [hidden email]

P: [phone]

I: www.tns-nipo.com

> Date: Wed, 5 May 2010 05:56:12 -0400
> From: [hidden email]
> Subject: reliability of a questionnaire and missing values in items
> To: [hidden email]
>
> Hi,
>
> to check internal consistency of a questionnaire I use cronbach`s alpha -
> the problem I have is the missings in items. As a consequence cronbachs
> alpha could not be calculatedbecause all cases were deleted.
>
> Looking on the syntax I saw that there is the possibility to
> define "MISSING=Include" in the reliability syntax. Rerunning the
> procedure yielded a considerable alpha (about 0.80).
>
> My question: does the statement "MISSING=Include" means that the procedure
> is set to pairwise deletion of missing data? (with other words the default
> is listwise deletion in the reliability statement)
>
> Many thanks for your answer, Doris
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD

---

New Windows 7: Find the right PC for you. Learn more.

---

New Windows 7: Find the right PC for you. Learn more.

[Nicholas.kuus]

i am an graduate student with questionnaire with missing items-this is because they are filter questions, and i have problem with reliability. I left the space for missing values blank during the coding. How do i go about it. it is urgent please. The file has been uploaded for your perusal

[kwame woei]

To get rid of the missing values use the RECODE INTO function.

For reability use Chronbach's Alpha. You should get a reliability between .60 and .80 (in the output screen you'll see which items to drop to get the desired reliability).



Op 2 apr. 2013 om 18:03 heeft "Nicholas.kuus" <[hidden email]> het volgende geschreven:

> i am an graduate student with questionnaire with missing items-this is
> because they are filter questions, and i have problem with reliability. I
> left the space for missing values blank during the coding. How do i go about
> it. it is urgent please. The file has been uploaded for your perusal
>
>
>
> --
> View this message in context: http://spssx-discussion.1045642.n5.nabble.com/reliability-of-a-questionnaire-and-missing-values-in-items-tp1091554p5719240.html
> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD