SPSSX Discussion

Re: Multiple-choice questions with one correct answer

Classic

List

Threaded

11 messages Options

Art Kendall

Re: Multiple-choice questions with one correct answer

That means one item is scored as wrong when more of the others in the scale are scored right and vice versa.
When you go back and look at the meaning of that question does it make sense?

Below there is some example syntax. After you run it to understand it. Then adapt it to your situation. Go back to the data view and change the option to see value labels rather than values. Check whether when the value label for item1 is "carrots" the variable carrots says "right".

If the squared item total correlation is missing that means that one or more items is (are) perfectly predictable from one or a subset of other items. Look at the corrected item total correlation for very high values.

LIST MEMBERS. I have a client expected to skype from Bangladesh in a few minutes so I cannot look at it right now.

It also seems that there should be some PYTHON extension way to use the value labels for the raw answers to set up the names of the dichotomous variables and double check the key. E.g. input a list of 23 strings for the correct answer to match to the value labels of the items.

Art Kendall
Social Research Consultants

On 8/9/2011 8:16 PM, Aishairma Aris wrote:

Dear Dr. Kendall,

I have 1 negative corrected item-total correlation

The inter item correlations neither exactly -1.00 nor +1.00

How to obtain the squared multiple? I do not really get it. If you do not mind explaining it to me, I would really appreciate.

I am very sure the scoring key was accurately applied.

The questions have either 4 or 3 choices of answer (a,b,c,d or a,b,c) with one correct answer only. I coded each question =1 if the answer was correct and 0 if the answer was wrong.

I would love to try coding the data as it is (as you suggested earlier). However, I don't really understand how to do it. I had tried it: I coded a=1, b=2, c=3, d=4 and I also tried: 1=wrong, 2=wrong, 3=correct, 4=correct. Unfortunately the Crobanch's alpha was -.111. I may have coded the item wrongly. I hope you could advise me.

Thank you

Kind regards,

Aishairma

From: Art Kendall [hidden email]
To: [hidden email]
Sent: Monday, August 8, 2011 1:44 PM
Subject: Re: Multiple-choice questions with one correct answer

Other responses have pointed out some of the possibilities.

Are you _very_ sure that the scoring key was accurately applied to the
original data?

Are any of the corrected item-total correlations negative?

Are any of the interitem correlations exactly -1.00 or +1.00?

Are any of the squared multiple correlations 1.00?

Is there any pattern to the negative inter-item correlations e.g., many
on a row or or column?

Art Kendall
Social Research Consultants

On 8/2/2011 6:46 AM, [hidden email] wrote:
> Hi, many thanks for the response
>
> Items in the questionnaire: 23 items
> Missing data: none
> N=159
> Construct: Knowledge about diabetes. I give 1 mark to a correct answer
> (each item). Each item has multiple choices of answer ranging from
> a,b,c,d and a,b,c. However, only 1 correct answer for each item. For
> example;
> 1. Which of the following is highest in carbohydrate?
> a. Chicken
> b. Milk
> c. Rice (correct answer)
> d. Peanut butter
> 1 mark is given if the research participant answer 'C', and zero mark
> will be given if they answer either a,b or d.
>
> In the SPSS, I only enter either zero or one. In the Label, I code it
> as '0=wrong and 1=correct'.
>
> What do you mean by entering the data as it is? Is it, I need to code
> the data as it is (in the questionnaire)? e.g. a,b,c,d and then give
> the value either zero or one?
>
> This scale has been used in previous researches.
>
> The Alpha reading for research was low with many negative correlation.
>
> Thank you
> Aishairma
>

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Art Kendall
Social Research Consultants

Art Kendall

Re: Multiple-choice questions with one correct answer

data list list /id (f3) item1 to item23(23a1).
begin data
001 a b c d a b c d a b c d a b c d a b c d a b c
002 a b a d a b c d a b a d a b c d a b c d a b c
003 a a c d a b c d a b c d a b c d a b c d a b c
004 a b c d a b c d a b c d a b c d a b c d a b c
005 b c d a b c d a b c d a b c d a b c d a b c a
006 b c d a b c d a b c d a b c d a b c d a b c a
007 a a a a a a a a a a a a a a a a a a a a a a a
008 b b b b b b b b b b b b b b b b b b b b b b b
009 c c c c c c c c c c c c c c c c c c c c c c c
010 d d d d d d d d d d d d d d d d d d d d d d d
end data.
do repeat
item= item1 to item23/
key = 'a','b','c','d','a','b','c','d','a','b','c','d','a','b','c','d','a','b','c','d','a','b','c'/
num_item = carrots celery lettuce zucchini crab oysters broiled steamed boiled raw chicken
nuts spelt quinoa bran sprouts olives tuna clams shrimp romaine artichokes mushrooms.
compute num_item = item eq key.
end repeat.
value labels carrots to mushrooms
1 'right'
2 'wrong'.
value labels item1 to item23
'a' 'words wrong'
'b' 'blah wrong'
'c' 'stuff right'
'd' 'text wrong'.
variable level item1 to item23 carrots to mushrooms (nominal).
reliability variables = carrots to mushrooms
/scale (knowledge) = carrots to mushrooms
/model = alpha
/statistics = correlations scale
/summary = means variances correlations total.

On 8/10/2011 8:58 AM, Art Kendall wrote:

That means one item is scored as wrong when more of the others in the scale are scored right and vice versa.
When you go back and look at the meaning of that question does it make sense?

Below there is some example syntax. After you run it to understand it. Then adapt it to your situation. Go back to the data view and change the option to see value labels rather than values. Check whether when the value label for item1 is "carrots" the variable carrots says "right".

If the squared item total correlation is missing that means that one or more items is (are) perfectly predictable from one or a subset of other items. Look at the corrected item total correlation for very high values.

LIST MEMBERS. I have a client expected to skype from Bangladesh in a few minutes so I cannot look at it right now.

It also seems that there should be some PYTHON extension way to use the value labels for the raw answers to set up the names of the dichotomous variables and double check the key. E.g. input a list of 23 strings for the correct answer to match to the value labels of the items.

Art Kendall
Social Research Consultants

On 8/9/2011 8:16 PM, Aishairma Aris wrote:

Dear Dr. Kendall,

I have 1 negative corrected item-total correlation

The inter item correlations neither exactly -1.00 nor +1.00

How to obtain the squared multiple? I do not really get it. If you do not mind explaining it to me, I would really appreciate.

I am very sure the scoring key was accurately applied.

The questions have either 4 or 3 choices of answer (a,b,c,d or a,b,c) with one correct answer only. I coded each question =1 if the answer was correct and 0 if the answer was wrong.

I would love to try coding the data as it is (as you suggested earlier). However, I don't really understand how to do it. I had tried it: I coded a=1, b=2, c=3, d=4 and I also tried: 1=wrong, 2=wrong, 3=correct, 4=correct. Unfortunately the Crobanch's alpha was -.111. I may have coded the item wrongly. I hope you could advise me.

Thank you

Kind regards,

Aishairma

From: Art Kendall [hidden email]
To: [hidden email]
Sent: Monday, August 8, 2011 1:44 PM
Subject: Re: Multiple-choice questions with one correct answer

Other responses have pointed out some of the possibilities.

Are you _very_ sure that the scoring key was accurately applied to the
original data?

Are any of the corrected item-total correlations negative?

Are any of the interitem correlations exactly -1.00 or +1.00?

Are any of the squared multiple correlations 1.00?

Is there any pattern to the negative inter-item correlations e.g., many
on a row or or column?

Art Kendall
Social Research Consultants

On 8/2/2011 6:46 AM, [hidden email] wrote:
> Hi, many thanks for the response
>
> Items in the questionnaire: 23 items
> Missing data: none
> N=159
> Construct: Knowledge about diabetes. I give 1 mark to a correct answer
> (each item). Each item has multiple choices of answer ranging from
> a,b,c,d and a,b,c. However, only 1 correct answer for each item. For
> example;
> 1. Which of the following is highest in carbohydrate?
> a. Chicken
> b. Milk
> c. Rice (correct answer)
> d. Peanut butter
> 1 mark is given if the research participant answer 'C', and zero mark
> will be given if they answer either a,b or d.
>
> In the SPSS, I only enter either zero or one. In the Label, I code it
> as '0=wrong and 1=correct'.
>
> What do you mean by entering the data as it is? Is it, I need to code
> the data as it is (in the questionnaire)? e.g. a,b,c,d and then give
> the value either zero or one?
>
> This scale has been used in previous researches.
>
> The Alpha reading for research was low with many negative correlation.
>
> Thank you
> Aishairma
>

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Art Kendall
Social Research Consultants

Bruce Weaver

Re: Multiple-choice questions with one correct answer

Administrator

Just in case anyone is wondering, this thread originated in the comp.soft-sys.stat.spss newsgroup. I think someone took it offline, and then when it came back online, it ended up here rather than in the usenet group. ;-)

Cheers,
Bruce

Art Kendall wrote

data list list /id (f3) item1 to item23(23a1).
begin data
001 a b c d a b c d a b c d a b c d a b c d a b c
002 a b a d a b c d a b a d a b c d a b c d a b c
003 a a c d a b c d a b c d a b c d a b c d a b c
004 a b c d a b c d a b c d a b c d a b c d a b c
005 b c d a b c d a b c d a b c d a b c d a b c a
006 b c d a b c d a b c d a b c d a b c d a b c a
007 a a a a a a a a a a a a a a a a a a a a a a a
008 b b b b b b b b b b b b b b b b b b b b b b b
009 c c c c c c c c c c c c c c c c c c c c c c c
010 d d d d d d d d d d d d d d d d d d d d d d d
end data.
do repeat
item= item1 to item23/
key =
'a','b','c','d','a','b','c','d','a','b','c','d','a','b','c','d','a','b','c','d','a','b','c'/
num_item = carrots celery lettuce zucchini crab oysters broiled
steamed boiled raw chicken
nuts spelt quinoa bran sprouts olives tuna clams shrimp romaine
artichokes mushrooms.
compute num_item = item eq key.
end repeat.
value labels carrots to mushrooms
1 'right'
2 'wrong'.
value labels item1 to item23
'a' 'words wrong'
'b' 'blah wrong'
'c' 'stuff right'
'd' 'text wrong'.
variable level item1 to item23 carrots to mushrooms (nominal).
reliability variables = carrots to mushrooms
/scale (knowledge) = carrots to mushrooms
/model = alpha
/statistics = correlations scale
/summary = means variances correlations total.

On 8/10/2011 8:58 AM, Art Kendall wrote:

That means one item is scored as wrong when more of the others in
the scale are scored right and vice versa.
When you go back and look at the meaning of that question does it
make sense?

Below there is some example syntax. After you run it to
understand it. Then adapt it to your situation. Go back to the
data view and change the option to see value labels rather than
values. Check whether when the value label for item1 is "carrots"
the variable carrots says "right".

If the squared item total correlation is missing that means that
one or more items is (are) perfectly predictable from one or a
subset of other items. Look at the corrected item total
correlation for very high values.

LIST MEMBERS. I have a client expected to skype from Bangladesh
in a few minutes so I cannot look at it right now.

It also seems that there should be some PYTHON extension way to
use the value labels for the raw answers to set up the names of
the dichotomous variables and double check the key. E.g. input a
list of 23 strings for the correct answer to match to the value
labels of the items.

Art Kendall
Social Research Consultants

On 8/9/2011 8:16 PM, Aishairma Aris wrote:

Dear Dr. Kendall,

I
have 1 negative corrected item-total correlation

The inter item correlations neither exactly
-1.00 nor +1.00

How to obtain the squared multiple? I do
not really get it. If you do not mind explaining
it to me, I would really appreciate.

I am very sure the scoring key was
accurately applied.
The questions have either 4 or 3 choices of
answer (a,b,c,d or a,b,c) with one correct answer
only. I coded each question =1 if the answer was
correct and 0 if the answer was wrong.

I would love to try coding the data as it is (as
you suggested earlier). However, I don't really
understand how to do it. I had tried it: I coded
a=1, b=2, c=3, d=4 and I also tried: 1=wrong,
2=wrong, 3=correct, 4=correct. Unfortunately the
Crobanch's alpha was -.111. I may have coded the
item wrongly. I hope you could advise me.

Thank you

Kind regards,
Aishairma

From:
Art Kendall <[hidden email]>
To:
[hidden email]
Sent:
Monday, August 8, 2011 1:44 PM
Subject:
Re: Multiple-choice questions with one correct
answer

Other responses have pointed out some of the
possibilities.

Are you _very_ sure that the scoring key was
accurately applied to the
original data?

Are any of the corrected item-total correlations
negative?

Are any of the interitem correlations exactly
-1.00 or +1.00?

Are any of the squared multiple correlations 1.00?

Is there any pattern to the negative inter-item
correlations e.g., many
on a row or or column?

Art Kendall
Social Research Consultants

On 8/2/2011 6:46 AM, [hidden email]
wrote:
> Hi, many thanks for the response
>
> Items in the questionnaire: 23 items
> Missing data: none
> N=159
> Construct: Knowledge about diabetes. I give 1
mark to a correct answer
> (each item). Each item has multiple choices
of answer ranging from
> a,b,c,d and a,b,c. However, only 1 correct
answer for each item. For
> example;
> 1. Which of the following is highest in
carbohydrate?
> a. Chicken
> b. Milk
> c. Rice (correct answer)
> d. Peanut butter
> 1 mark is given if the research participant
answer 'C', and zero mark
> will be given if they answer either a,b or d.
>
> In the SPSS, I only enter either zero or one.
In the Label, I code it
> as '0=wrong and 1=correct'.
>
> What do you mean by entering the data as it
is? Is it, I need to code
> the data as it is (in the questionnaire)?
e.g. a,b,c,d and then give
> the value either zero or one?
>
> This scale has been used in previous
researches.
>
> The Alpha reading for research was low with
many negative correlation.
>
> Thank you
> Aishairma
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text
except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

PLEASE NOTE THE FOLLOWING:
1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above.
2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/).

Garry Gelade

Re: Multiple-choice questions with one correct answer

In reply to this post by Art Kendall

Art/ Aishairma

Is there any particular reason for treating this as a reflective construct? I am not familiar with the research area, but I would suspect that on theoretical grounds ‘Knowledge of Diabetes’ could more plausibly be construed as a formative construct.

If you think of it as a formative construct, the low Alpha problem goes away, because there is no need for the scale indicators to correlate. In fact, the non-existent alpha suggests you do have a formative construct here.

Garry

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Art Kendall
Sent: 10 August 2011 14:12
To: [hidden email]
Subject: Re: Multiple-choice questions with one correct answer

Dear Dr. Kendall,

I have 1 negative corrected item-total correlation

The inter item correlations neither exactly -1.00 nor +1.00

How to obtain the squared multiple? I do not really get it. If you do not mind explaining it to me, I would really appreciate.

I am very sure the scoring key was accurately applied.

The questions have either 4 or 3 choices of answer (a,b,c,d or a,b,c) with one correct answer only. I coded each question =1 if the answer was correct and 0 if the answer was wrong.

I would love to try coding the data as it is (as you suggested earlier). However, I don't really understand how to do it. I had tried it: I coded a=1, b=2, c=3, d=4 and I also tried: 1=wrong, 2=wrong, 3=correct, 4=correct. Unfortunately the Crobanch's alpha was -.111. I may have coded the item wrongly. I hope you could advise me.

Thank you

Kind regards,

Aishairma

From: Art Kendall [hidden email]
To: [hidden email]
Sent: Monday, August 8, 2011 1:44 PM
Subject: Re: Multiple-choice questions with one correct answer

Other responses have pointed out some of the possibilities.

Are you _very_ sure that the scoring key was accurately applied to the
original data?

Are any of the corrected item-total correlations negative?

Are any of the interitem correlations exactly -1.00 or +1.00?

Are any of the squared multiple correlations 1.00?

Is there any pattern to the negative inter-item correlations e.g., many
on a row or or column?

Art Kendall
Social Research Consultants

On 8/2/2011 6:46 AM, [hidden email] wrote:
> Hi, many thanks for the response
>
> Items in the questionnaire: 23 items
> Missing data: none
> N=159
> Construct: Knowledge about diabetes. I give 1 mark to a correct answer
> (each item). Each item has multiple choices of answer ranging from
> a,b,c,d and a,b,c. However, only 1 correct answer for each item. For
> example;
> 1. Which of the following is highest in carbohydrate?
> a. Chicken
> b. Milk
> c. Rice (correct answer)
> d. Peanut butter
> 1 mark is given if the research participant answer 'C', and zero mark
> will be given if they answer either a,b or d.
>
> In the SPSS, I only enter either zero or one. In the Label, I code it
> as '0=wrong and 1=correct'.
>
> What do you mean by entering the data as it is? Is it, I need to code
> the data as it is (in the questionnaire)? e.g. a,b,c,d and then give
> the value either zero or one?
>
> This scale has been used in previous researches.
>
> The Alpha reading for research was low with many negative correlation.
>
> Thank you
> Aishairma
>

Art Kendall

Re: Multiple-choice questions with one correct answer

In reply to this post by Art Kendall

The syntax will run if the value labels are not there.
Some reasons to put them in are:
it is good analytic practice to complete the metadata (variables view).
Having the meta data makes explicit to you and to others what exactly you are doing. The other include people trying to help, your team members doing quality assurance, recipients of your research data, archivists if your work is part of a funded project etc.
etc.

When you adapt the syntax to your situation, you would change variable names to fit your situation. You would also put the appropriate letters in the key.
In my experience errors in scoring keys are very very common. so I have developed some habits to check on them. E.g., I made up names of the dichotomous variables because this is often something that helps people double check their work.

Art Kendall
Social Research Consultants

On 8/10/2011 9:55 AM, Aishairma Aris wrote:

Many thanks Dr. Kendall.

Do I need to type in value labels for each item? This because the key of the correct answer of each item is not the same. Some correct answers are 'a' or 'b' or 'c' or 'd'

Kind regards

Aishairma

From: Art Kendall [hidden email]
To: [hidden email]
Cc: [hidden email]
Sent: Wednesday, August 10, 2011 2:11 PM
Subject: Re: [SPSSX-L] Multiple-choice questions with one correct answer

data list list /id (f3) item1 to item23(23a1).
begin data
001 a b c d a b c d a b c d a b c d a b c d a b c
002 a b a d a b c d a b a d a b c d a b c d a b c
003 a a c d a b c d a b c d a b c d a b c d a b c
004 a b c d a b c d a b c d a b c d a b c d a b c
005 b c d a b c d a b c d a b c d a b c d a b c a
006 b c d a b c d a b c d a b c d a b c d a b c a
007 a a a a a a a a a a a a a a a a a a a a a a a
008 b b b b b b b b b b b b b b b b b b b b b b b
009 c c c c c c c c c c c c c c c c c c c c c c c
010 d d d d d d d d d d d d d d d d d d d d d d d
end data.
do repeat
item= item1 to item23/
key = 'a','b','c','d','a','b','c','d','a','b','c','d','a','b','c','d','a','b','c','d','a','b','c'/
num_item = carrots celery lettuce zucchini crab oysters broiled steamed boiled raw chicken
nuts spelt quinoa bran sprouts olives tuna clams shrimp romaine artichokes mushrooms.
compute num_item = item eq key.
end repeat.
value labels carrots to mushrooms
1 'right'
2 'wrong'.
value labels item1 to item23
'a' 'words wrong'
'b' 'blah wrong'
'c' 'stuff right'
'd' 'text wrong'.
variable level item1 to item23 carrots to mushrooms (nominal).
reliability variables = carrots to mushrooms
/scale (knowledge) = carrots to mushrooms
/model = alpha
/statistics = correlations scale
/summary = means variances correlations total.

On 8/10/2011 8:58 AM, Art Kendall wrote:
That means one item is scored as wrong when more of the others in the scale are scored right and vice versa.
When you go back and look at the meaning of that question does it make sense?

Below there is some example syntax. After you run it to understand it. Then adapt it to your situation. Go back to the data view and change the option to see value labels rather than values. Check whether when the value label for item1 is "carrots" the variable carrots says "right".

If the squared item total correlation is missing that means that one or more items is (are) perfectly predictable from one or a subset of other items. Look at the corrected item total correlation for very high values.

LIST MEMBERS. I have a client expected to skype from Bangladesh in a few minutes so I cannot look at it right now.

It also seems that there should be some PYTHON extension way to use the value labels for the raw answers to set up the names of the dichotomous variables and double check the key. E.g. input a list of 23 strings for the correct answer to match to the value labels of the items.

Art Kendall
Social Research Consultants

On 8/9/2011 8:16 PM, Aishairma Aris wrote:

Dear Dr. Kendall,

I have 1 negative corrected item-total correlation

The inter item correlations neither exactly -1.00 nor +1.00

How to obtain the squared multiple? I do not really get it. If you do not mind explaining it to me, I would really appreciate.

I am very sure the scoring key was accurately applied.

The questions have either 4 or 3 choices of answer (a,b,c,d or a,b,c) with one correct answer only. I coded each question =1 if the answer was correct and 0 if the answer was wrong.

I would love to try coding the data as it is (as you suggested earlier). However, I don't really understand how to do it. I had tried it: I coded a=1, b=2, c=3, d=4 and I also tried: 1=wrong, 2=wrong, 3=correct, 4=correct. Unfortunately the Crobanch's alpha was -.111. I may have coded the item wrongly. I hope you could advise me.

Thank you

Kind regards,

Aishairma

From: Art Kendall [hidden email]
To: [hidden email]
Sent: Monday, August 8, 2011 1:44 PM
Subject: Re: Multiple-choice questions with one correct answer

Other responses have pointed out some of the possibilities.

Are you _very_ sure that the scoring key was accurately applied to the
original data?

Are any of the corrected item-total correlations negative?

Are any of the interitem correlations exactly -1.00 or +1.00?

Are any of the squared multiple correlations 1.00?

Is there any pattern to the negative inter-item correlations e.g., many
on a row or or column?

Art Kendall
Social Research Consultants

On 8/2/2011 6:46 AM, [hidden email] wrote:
> Hi, many thanks for the response
>
> Items in the questionnaire: 23 items
> Missing data: none
> N=159
> Construct: Knowledge about diabetes. I give 1 mark to a correct answer
> (each item). Each item has multiple choices of answer ranging from
> a,b,c,d and a,b,c. However, only 1 correct answer for each item. For
> example;
> 1. Which of the following is highest in carbohydrate?
> a. Chicken
> b. Milk
> c. Rice (correct answer)
> d. Peanut butter
> 1 mark is given if the research participant answer 'C', and zero mark
> will be given if they answer either a,b or d.
>
> In the SPSS, I only enter either zero or one. In the Label, I code it
> as '0=wrong and 1=correct'.
>
> What do you mean by entering the data as it is? Is it, I need to code
> the data as it is (in the questionnaire)? e.g. a,b,c,d and then give
> the value either zero or one?
>
> This scale has been used in previous researches.
>
> The Alpha reading for research was low with many negative correlation.
>
> Thank you
> Aishairma
>

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Art Kendall
Social Research Consultants

Art Kendall

Re: Multiple-choice questions with one correct answer

In reply to this post by Garry Gelade

I would lean toward considering this a reflective construct in that in knowledge, achievement, aptitude, attitudes, etc, the items are sort of repeated imperfect measures of a construct which are substitutable for each other.
Here the more items that are correct the more knowledge there is. This is similar to the idea that spelling items are used as rough measures of spelling ability.

In my experience, YMMV, many errors are made in applying scoring keys. Many parts of the RELIABILITY output are useful for this. Sizable negative interitem correlations are red flags saying to go back and check that things have been done correctly especially when there are many in a row or column.
negative corrected item-total correlations are also red flags to go back and check that things are done correctly.

This OP ended up having an item with several high negative correlations. I would stay with the a priori assumption that the scoring was not what was intended until very thorough checks have been made.

This is very much like my experience that suspicious values are much more likely to be user error than "true outliers" calling for special action. Again, YMMV.

Art Kendall
Social Research Consultants

On 8/10/2011 10:27 AM, Garry Gelade wrote:

Art/ Aishairma

Is there any particular reason for treating this as a reflective construct? I am not familiar with the research area, but I would suspect that on theoretical grounds ‘Knowledge of Diabetes’ could more plausibly be construed as a formative construct.

If you think of it as a formative construct, the low Alpha problem goes away, because there is no need for the scale indicators to correlate. In fact, the non-existent alpha suggests you do have a formative construct here.

Garry

From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Art Kendall
Sent: 10 August 2011 14:12
To: [hidden email]
Subject: Re: Multiple-choice questions with one correct answer

data list list /id (f3) item1 to item23(23a1).
begin data
001 a b c d a b c d a b c d a b c d a b c d a b c
002 a b a d a b c d a b a d a b c d a b c d a b c
003 a a c d a b c d a b c d a b c d a b c d a b c
004 a b c d a b c d a b c d a b c d a b c d a b c
005 b c d a b c d a b c d a b c d a b c d a b c a
006 b c d a b c d a b c d a b c d a b c d a b c a
007 a a a a a a a a a a a a a a a a a a a a a a a
008 b b b b b b b b b b b b b b b b b b b b b b b
009 c c c c c c c c c c c c c c c c c c c c c c c
010 d d d d d d d d d d d d d d d d d d d d d d d
end data.
do repeat
item= item1 to item23/
key = 'a','b','c','d','a','b','c','d','a','b','c','d','a','b','c','d','a','b','c','d','a','b','c'/
num_item = carrots celery lettuce zucchini crab oysters broiled steamed boiled raw chicken
nuts spelt quinoa bran sprouts olives tuna clams shrimp romaine artichokes mushrooms.
compute num_item = item eq key.
end repeat.
value labels carrots to mushrooms
1 'right'
2 'wrong'.
value labels item1 to item23
'a' 'words wrong'
'b' 'blah wrong'
'c' 'stuff right'
'd' 'text wrong'.
variable level item1 to item23 carrots to mushrooms (nominal).
reliability variables = carrots to mushrooms
/scale (knowledge) = carrots to mushrooms
/model = alpha
/statistics = correlations scale
/summary = means variances correlations total.

On 8/10/2011 8:58 AM, Art Kendall wrote:

That means one item is scored as wrong when more of the others in the scale are scored right and vice versa.
When you go back and look at the meaning of that question does it make sense?

Below there is some example syntax. After you run it to understand it. Then adapt it to your situation. Go back to the data view and change the option to see value labels rather than values. Check whether when the value label for item1 is "carrots" the variable carrots says "right".

If the squared item total correlation is missing that means that one or more items is (are) perfectly predictable from one or a subset of other items. Look at the corrected item total correlation for very high values.

LIST MEMBERS. I have a client expected to skype from Bangladesh in a few minutes so I cannot look at it right now.

It also seems that there should be some PYTHON extension way to use the value labels for the raw answers to set up the names of the dichotomous variables and double check the key. E.g. input a list of 23 strings for the correct answer to match to the value labels of the items.

Art Kendall
Social Research Consultants

On 8/9/2011 8:16 PM, Aishairma Aris wrote:

Dear Dr. Kendall,

I have 1 negative corrected item-total correlation

The inter item correlations neither exactly -1.00 nor +1.00

How to obtain the squared multiple? I do not really get it. If you do not mind explaining it to me, I would really appreciate.

I am very sure the scoring key was accurately applied.

The questions have either 4 or 3 choices of answer (a,b,c,d or a,b,c) with one correct answer only. I coded each question =1 if the answer was correct and 0 if the answer was wrong.

I would love to try coding the data as it is (as you suggested earlier). However, I don't really understand how to do it. I had tried it: I coded a=1, b=2, c=3, d=4 and I also tried: 1=wrong, 2=wrong, 3=correct, 4=correct. Unfortunately the Crobanch's alpha was -.111. I may have coded the item wrongly. I hope you could advise me.

Thank you

Kind regards,

Aishairma

From: Art Kendall [hidden email]
To: [hidden email]
Sent: Monday, August 8, 2011 1:44 PM
Subject: Re: Multiple-choice questions with one correct answer

Other responses have pointed out some of the possibilities.

Are you _very_ sure that the scoring key was accurately applied to the
original data?

Are any of the corrected item-total correlations negative?

Are any of the interitem correlations exactly -1.00 or +1.00?

Are any of the squared multiple correlations 1.00?

Is there any pattern to the negative inter-item correlations e.g., many
on a row or or column?

Art Kendall
Social Research Consultants

On 8/2/2011 6:46 AM, [hidden email] wrote:
> Hi, many thanks for the response
>
> Items in the questionnaire: 23 items
> Missing data: none
> N=159
> Construct: Knowledge about diabetes. I give 1 mark to a correct answer
> (each item). Each item has multiple choices of answer ranging from
> a,b,c,d and a,b,c. However, only 1 correct answer for each item. For
> example;
> 1. Which of the following is highest in carbohydrate?
> a. Chicken
> b. Milk
> c. Rice (correct answer)
> d. Peanut butter
> 1 mark is given if the research participant answer 'C', and zero mark
> will be given if they answer either a,b or d.
>
> In the SPSS, I only enter either zero or one. In the Label, I code it
> as '0=wrong and 1=correct'.
>
> What do you mean by entering the data as it is? Is it, I need to code
> the data as it is (in the questionnaire)? e.g. a,b,c,d and then give
> the value either zero or one?
>
> This scale has been used in previous researches.
>
> The Alpha reading for research was low with many negative correlation.
>
> Thank you
> Aishairma
>

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Art Kendall
Social Research Consultants

Art Kendall

Re: Multiple-choice questions with one correct answer

In reply to this post by Bruce Weaver

Yikes! You are correct. I erred as to which forum when I tried to put the discussion back in public.

BTW has anyone created an extension to work with the value labels of string or numeric variables, a scoring key, and creating a new set of dichotomous variables with names indicating the response that is right or wrong.

Art

On 8/10/2011 9:49 AM, Bruce Weaver wrote:

Just in case anyone is wondering, this thread originated in the
comp.soft-sys.stat.spss newsgroup.  I think someone took it offline, and
then when it came back online, it ended up here rather than in the usenet
group.  ;-)

Cheers,
Bruce



Art Kendall wrote:

data list list /id (f3) item1 to item23(23a1).
      begin data
      001 a b c d a b c d a b c d a b c d a b c d a b c
      002 a b a d a b c d a b a d a b c d a b c d a b c
      003 a a c d a b c d a b c d a b c d a b c d a b c
      004 a b c d a b c d a b c d a b c d a b c d a b c
      005 b c d a b c d a b c d a b c d a b c d a b c a
      006 b c d a b c d a b c d a b c d a b c d a b c a
      007 a a a a a a a a a a a a a a a a a a a a a a a
      008 b b b b b b b b b b b b b b b b b b b b b b b
      009 c c c c c c c c c c c c c c c c c c c c c c c
      010 d d d d d d d d d d d d d d d d d d d d d d d
      end data.
      do repeat
      &nbsp;item= item1 to item23/
      key =
'a','b','c','d','a','b','c','d','a','b','c','d','a','b','c','d','a','b','c','d','a','b','c'/
      &nbsp;num_item = carrots celery lettuce zucchini crab oysters
broiled
      steamed boiled raw chicken
      &nbsp;nuts spelt quinoa bran sprouts olives tuna clams shrimp
romaine
      artichokes mushrooms.
      compute num_item = item eq key.
      end repeat.
      value labels carrots to mushrooms
      &nbsp;1 'right'
      &nbsp;2 'wrong'.
      value labels item1 to item23
      &nbsp;'a' 'words wrong'
      &nbsp;'b' 'blah wrong'
      &nbsp;'c' 'stuff right'
      &nbsp;'d' 'text wrong'.
      variable level item1 to item23 carrots to mushrooms (nominal).
      reliability variables = carrots to mushrooms
      &nbsp;/scale (knowledge) = carrots to mushrooms
      &nbsp;/model = alpha
      &nbsp;/statistics = correlations scale
      &nbsp;/summary = means variances correlations total.


    On 8/10/2011 8:58 AM, Art Kendall wrote:


      That means one item is scored as wrong when more of the others in
      the scale are scored right and vice versa.
      When you go back and look at the meaning of that question does it
      make sense?

      Below there is some example syntax.&nbsp; After you run it to
      understand it.&nbsp; Then adapt it to your situation. Go back to the
      data view and change the option to see value labels rather than
      values. Check whether when the value label for item1 is
"carrots"&nbsp;
      the variable carrots says "right".

      If the squared item total correlation is missing that means that
      one or more items is (are) perfectly predictable from one or a
      subset of other items.&nbsp; Look at the corrected item total
      correlation for very high values.

      LIST MEMBERS.&nbsp; I have a client expected to skype from
Bangladesh
      in a few minutes so I cannot look at it right now.


      It also seems that there should be some PYTHON extension way to
      use the value labels for the raw answers to set up the names of
      the dichotomous variables and double check the key.&nbsp; E.g. input
a
      list of 23 strings for the correct answer to match to the value
      labels of the items.

      Art Kendall
      Social Research Consultants


      On 8/9/2011 8:16 PM, Aishairma Aris wrote:

        Dear Dr. Kendall,





            I
                have 1 negative corrected item-total correlation










                  The inter item correlations neither exactly
                      -1.00 nor +1.00


                  How to obtain the squared multiple? I do
                      not really get it. If you do not mind explaining
                      it to me, I would really appreciate.&nbsp;


                  I am very sure the scoring key was
                      accurately applied.
                  The questions have either 4 or 3 choices of
                      answer (a,b,c,d or a,b,c) with one correct answer
                      only. I coded each question =1 if the answer was
                      correct and 0 if the answer was wrong.&nbsp;


                  I would love to try coding the data as it is (as
                    you suggested earlier). However, I don't really
                    understand how to do it. I had tried it: I coded
                    a=1, b=2, c=3, d=4 and I also tried: 1=wrong,
                    2=wrong, 3=correct, 4=correct. Unfortunately the
                    Crobanch's alpha was -.111. I may have coded the
                    item wrongly. I hope you could advise me.


                  Thank you


                  Kind regards,
                  Aishairma





                        From:
                        Art Kendall &lt;[hidden email]&gt;
                        To:
                        [hidden email]
                        Sent:
                        Monday, August 8, 2011 1:44 PM
                        Subject:
                        Re: Multiple-choice questions with one correct
                        answer

                      Other responses have pointed out some of the
                      possibilities.

                      Are you _very_ sure that the scoring key was
                      accurately applied to the
                      original data?

                      Are any of the corrected item-total correlations
                      negative?

                      Are any of the interitem correlations exactly
                      -1.00&nbsp; or +1.00?

                      Are any of the squared multiple correlations 1.00?

                      Is there any pattern to the negative inter-item
                      correlations e.g., many
                      on a row or or column?

                      Art Kendall
                      Social Research Consultants




                      On 8/2/2011 6:46 AM, [hidden email]
                      wrote:
                      &gt; Hi, many thanks for the response
                      &gt;
                      &gt; Items in the questionnaire: 23 items
                      &gt; Missing data: none
                      &gt; N=159
                      &gt; Construct: Knowledge about diabetes. I give 1
                      mark to a correct answer
                      &gt; (each item). Each item has multiple choices
                      of answer ranging from
                      &gt; a,b,c,d and a,b,c. However, only 1 correct
                      answer for each item. For
                      &gt; example;
                      &gt; 1.&nbsp; Which of the following is highest in
                      carbohydrate?
                      &gt; a.&nbsp; &nbsp; &nbsp; Chicken
                      &gt; b.&nbsp; &nbsp; &nbsp; Milk
                      &gt; c.&nbsp; &nbsp; &nbsp; Rice (correct answer)
                      &gt; d.&nbsp; &nbsp; &nbsp; Peanut butter
                      &gt; 1 mark is given if the research participant
                      answer 'C', and zero mark
                      &gt; will be given if they answer either a,b or d.
                      &gt;
                      &gt; In the SPSS, I only enter either zero or one.
                      In the Label, I code it
                      &gt; as '0=wrong and 1=correct'.
                      &gt;
                      &gt; What do you mean by entering the data as it
                      is? Is it, I need to code
                      &gt; the data as it is (in the questionnaire)?
                      e.g. a,b,c,d and then give
                      &gt; the value either zero or one?
                      &gt;
                      &gt; This scale has been used in previous
                      researches.
                      &gt;
                      &gt; The Alpha reading for research was low with
                      many negative correlation.
                      &gt;
                      &gt; Thank you
                      &gt; Aishairma
                      &gt;












      =====================
      To manage your subscription to SPSSX-L, send a message to
      [hidden email]  (not to SPSSX-L), with no body text
      except the
      command. To leave the list, send the command
      SIGNOFF SPSSX-L
      For a list of commands to manage subscriptions, send the command
      INFO REFCARD




=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Re-Multiple-choice-questions-with-one-correct-answer-tp4685738p4685846.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Art Kendall
Social Research Consultants

Garry Gelade

Re: Multiple-choice questions with one correct answer

In reply to this post by Art Kendall

Art

“The more items that are correct, the more knowledge there is” – a good definition of a formative construct! I can see why something like reading ability/memory/cognitive ability might drive spelling, but here we have a test of domain-specific knowledge about a very narrow topic, not quite the same thing.

That said, I agree it’s easy to make mistakes. My mileage doesn’t vary on that point. But I would not be distressed if I found a low alpha for this type of scale.

Garry

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Art Kendall
Sent: 10 August 2011 17:02
To: [hidden email]
Subject: Re: Multiple-choice questions with one correct answer

Art/ Aishairma

Garry

From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Art Kendall
Sent: 10 August 2011 14:12
To: [hidden email]
Subject: Re: Multiple-choice questions with one correct answer

Dear Dr. Kendall,

I have 1 negative corrected item-total correlation

The inter item correlations neither exactly -1.00 nor +1.00

How to obtain the squared multiple? I do not really get it. If you do not mind explaining it to me, I would really appreciate.

I am very sure the scoring key was accurately applied.

The questions have either 4 or 3 choices of answer (a,b,c,d or a,b,c) with one correct answer only. I coded each question =1 if the answer was correct and 0 if the answer was wrong.

I would love to try coding the data as it is (as you suggested earlier). However, I don't really understand how to do it. I had tried it: I coded a=1, b=2, c=3, d=4 and I also tried: 1=wrong, 2=wrong, 3=correct, 4=correct. Unfortunately the Crobanch's alpha was -.111. I may have coded the item wrongly. I hope you could advise me.

Thank you

Kind regards,

Aishairma

From: Art Kendall [hidden email]
To: [hidden email]
Sent: Monday, August 8, 2011 1:44 PM
Subject: Re: Multiple-choice questions with one correct answer

Other responses have pointed out some of the possibilities.

Are you _very_ sure that the scoring key was accurately applied to the
original data?

Are any of the corrected item-total correlations negative?

Are any of the interitem correlations exactly -1.00 or +1.00?

Are any of the squared multiple correlations 1.00?

Is there any pattern to the negative inter-item correlations e.g., many
on a row or or column?

Art Kendall
Social Research Consultants

On 8/2/2011 6:46 AM, [hidden email] wrote:
> Hi, many thanks for the response
>
> Items in the questionnaire: 23 items
> Missing data: none
> N=159
> Construct: Knowledge about diabetes. I give 1 mark to a correct answer
> (each item). Each item has multiple choices of answer ranging from
> a,b,c,d and a,b,c. However, only 1 correct answer for each item. For
> example;
> 1. Which of the following is highest in carbohydrate?
> a. Chicken
> b. Milk
> c. Rice (correct answer)
> d. Peanut butter
> 1 mark is given if the research participant answer 'C', and zero mark
> will be given if they answer either a,b or d.
>
> In the SPSS, I only enter either zero or one. In the Label, I code it
> as '0=wrong and 1=correct'.
>
> What do you mean by entering the data as it is? Is it, I need to code
> the data as it is (in the questionnaire)? e.g. a,b,c,d and then give
> the value either zero or one?
>
> This scale has been used in previous researches.
>
> The Alpha reading for research was low with many negative correlation.
>
> Thank you
> Aishairma
>

Rich Ulrich

Re: Multiple-choice questions with one correct answer

Garry,
I think you are quoting me about "The more items that are correct...", and
I also stated that I would not be surprised at a low alpha for this type; and
I added the information that "reliability" of any sort depends on the variability
which exists in the sample.

I see (for the first time) the information from the OP that there is a variable
with a negative value for the corrected item-total correlation, and that should
be a cause for special attention. It is a pretty fair indication, for similar scales,
that the Key is wrong or the item was badly worded. A further check would
be to look at the responses for a handful of subjects who aced the test.
(Was the item ambiguous? Is there a response - chosen by the Aces - that
*does* correlate with the total of the other items? Is there any controversy
on the subject? Is there a current Urban legend that accounts for the error?)

--
Rich Ulrich

Date: Wed, 10 Aug 2011 18:17:38 +0100
From: [hidden email]
Subject: Re: Multiple-choice questions with one correct answer
To: [hidden email]

Art

That said, I agree it’s easy to make mistakes. My mileage doesn’t vary on that point. But I would not be distressed if I found a low alpha for this type of scale.

Garry

John F Hall

Re: Multiple-choice questions with one correct answer

I’ve just come back from a 5-day break, so forgive my two cents’ worth.

I can’t see any evidence of data checking or initial analyses here. Have you done a frequency count on the initial responses? If your data are in 1 = “correct” 0 = “wrong” format, have you used MULT RESPONSE in dichotomous mode to produce a summary table of “correct” answers? Have you thought of using COUNT or COMPUTE to generate a score?

So before you get to the fancy stuff, and assuming varnames v1 to v23 I suggest you try:

File > new > syntax:

Count score1 = v1 to v23 (1) .

Compute score2 = sum.23 (v1 to v23) .

Freq score1 score2 /his nor .

Cros score1 score2 by v1 to v23 /cel cou col .

Means score1 score2 by v1 to v23 .

Mult resp correct (v1 to v23 (1))

/freq correct .

Corr score1 score2 / score1 score2 with v1 to v23 .

[Bruce, Art, Garry, Rich, Is it OK to use CORR for 0/1? I’m not a statistician.]

John Hall

[hidden email]

www.surveyresearch.weebly.com

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Rich Ulrich
Sent: 11 August 2011 05:24
To: [hidden email]
Subject: Re: Multiple-choice questions with one correct answer

Date: Wed, 10 Aug 2011 18:17:38 +0100
From: [hidden email]
Subject: Re: Multiple-choice questions with one correct answer
To: [hidden email]

Art

That said, I agree it’s easy to make mistakes. My mileage doesn’t vary on that point. But I would not be distressed if I found a low alpha for this type of scale.

Garry

David Marso

Re: Multiple-choice questions with one correct answer

Administrator

BUT PLEASE DO *NOT POST THE OUTPUT TO THE NEWSGROUP! Send it directly to John!!!
I'm assuming your email supports LARGE attachments.
=========
FYI:Correlations are fine with 0/1. The beast is known as a Phi Coefficient when x and y are 0/1 (or any other dich). A Point Biserial correlation when 1 is dichotomous and other is 'continuous'.

John F Hall wrote

I've just come back from a 5-day break, so forgive my two cents' worth.

I can't see any evidence of data checking or initial analyses here. Have
you done a frequency count on the initial responses? If your data are in 1
= "correct" 0 = "wrong" format, have you used MULT RESPONSE in dichotomous
mode to produce a summary table of "correct" answers? Have you thought of
using COUNT or COMPUTE to generate a score?

So before you get to the fancy stuff, and assuming varnames v1 to v23 I
suggest you try:

File > new > syntax:

Count score1 = v1 to v23 (1) .

Compute score2 = sum.23 (v1 to v23) .

Freq score1 score2 /his nor .

Cros score1 score2 by v1 to v23 /cel cou col .

Means score1 score2 by v1 to v23 .

Mult resp correct (v1 to v23 (1))

/freq correct .

Corr score1 score2 / score1 score2 with v1 to v23 .

[Bruce, Art, Garry, Rich, Is it OK to use CORR for 0/1? I'm not a
statistician.]

John Hall

[hidden email]

www.surveyresearch.weebly.com <http://surveyresearch.weebly.com/>

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Rich Ulrich
Sent: 11 August 2011 05:24
To: [hidden email]
Subject: Re: Multiple-choice questions with one correct answer

Garry,
I think you are quoting me about "The more items that are correct...", and
I also stated that I would not be surprised at a low alpha for this type;
and
I added the information that "reliability" of any sort depends on the
variability
which exists in the sample.

I see (for the first time) the information from the OP that there is a
variable
with a negative value for the corrected item-total correlation, and that
should
be a cause for special attention. It is a pretty fair indication, for
similar scales,
that the Key is wrong or the item was badly worded. A further check would
be to look at the responses for a handful of subjects who aced the test.
(Was the item ambiguous? Is there a response - chosen by the Aces - that
*does* correlate with the total of the other items? Is there any
controversy
on the subject? Is there a current Urban legend that accounts for the
error?)

--
Rich Ulrich

_____

Date: Wed, 10 Aug 2011 18:17:38 +0100
From: [hidden email]
Subject: Re: Multiple-choice questions with one correct answer
To: [hidden email]

Art

"The more items that are correct, the more knowledge there is" - a good
definition of a formative construct! I can see why something like reading
ability/memory/cognitive ability might drive spelling, but here we have a
test of domain-specific knowledge about a very narrow topic, not quite the
same thing.

That said, I agree it's easy to make mistakes. My mileage doesn't vary on
that point. But I would not be distressed if I found a low alpha for this
type of scale.

Garry

Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me.
---
"Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis."
Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?"