validating a questionnaire
Locked
 
|
Dear all,
Hello, I have developed an instrument (questionnaire) in which a scenario is followed by four different reactions. The respondents need to choose one of those reactions as their dominant response in that particular occasion (each reaction exemplifies a particular approach). The research question I'm going to respond is if there is any relationship between respondents' gender (and some other variables) and their choice of particular approaches. The problem I have is in validating this questionnaire. May I run factor analysis by turning the questionnaire into a Likert scale in which the respondents read each reaction and then identify on the Likert scale the extent to which they might exhibit each reaction in that particular occasion (mentioned in the scenario)? Then, the questionnaire seems something like this:
This way each reaction works as a separate item in SPSS and the scenarios are left. I will highly appreciate your insights. Leila Tajik |
||||||||||||||||||||||||||||||||||||||||||
|
Leila
Factor analysis is a bit previous: you need to
start with frequencies etc. for each item.
You need to pilot your questionnaire on a few
people first.
Off the cuff I suggest you run the battery of
scenarios as it is, without asking for dominant reaction, then present a listing
of the possible reactions and ask something like:
Regardless of the answers you've just given, which
of the items on this card would you be most likely to do?
....and which next?
Is your questionnaire online, by self-completion or
by personal interview (by phone or face-to-face)?
Assuming self-completion or on-line, something like
this
Scenario 1
For each of the following, please circle the answer
which best applies to you:
Always do Sometimes do Sometimes
do Never do Not
sure
Reaction
1
4
3
2
1
9
Reaction
2
4
3
2
1
9
Reaction
3
4
3
2
1
9
Reaction
4
4
3
2
1
9
...and which of the above are you most likely to
do?
Reaction
1 Reaction 2 Reaction 3
Reaction 4 None of these.
1 2 3
4 0
Repeat for each scenario
Here's an example of a self-completion
questionnaire used in a survey of 15-16 year olds.
This example is from my website http://surveyresearch.weebly.com where you will find extensive syntax-based tutorials on SPSS for survey analysis. This particular questionnaire and data set are included in Introduction to COUNT and COMPUTE. Everything is freely downloadable.
Remember, many respondents are pretty poor at assessing their own priorities.
Hope I haven't been doing your assessment work for you! Good luck.
John Hall
----- Original Message -----
|
||||||||||||||||||||||||||||||||||||||||||
|
|
In reply to this post by Leila Tajik
Dear Leila,
That first thing you need to consider is -- what is validity? A good place to begin reading about the nature of validity is Denny Borsboom http://sites.google.com/site/borsboomdenny/papers. From his references you can find most of the major work on validity during the 20th century. His perspective is that it is important to have a theoretical and conceptual argument for why responses measure whatever it is we have designed them to measure. Thus, to be logically defensible, it is important to narrowly define what that is. In your case you might argue that you are measuring distal behavioral intentions. You may also be interested in the predictive ability of your questionnaire and in that case you would want to know the strength of the relationship between your questionnaire and a set of behavioral outcomes similar to the scenarios themselves. In other words -- does a particular answer or set of answers increase the probability of engaging in that behavior at a later time, does it reduce the probability, or are the distal behavioral intentions unrelated? To your questionnaire specifically -- the decision about whether to use a Likert type response scale versus one exclusively anchored by potential behaviors is important because each has different cognitive requirements and thus result in arguably different constructs. Does the Likert scale provide important information that covers more of the intended conceptual space of your construct or does it potentially confuse the respondent and increase measurement error? You can certainly do what you propose, but I recommend spending some time thinking about the nature of validity as you move forward with your research. Best regards, Jim
On Tue, May 18, 2010 at 2:17 AM, Leila Tajik <[hidden email]> wrote: Dear all, -- Psychometric Consultant 2 Wolf Ridge Gap Ledyard, CT 06339 Phone: 860.389.0414 |
||||||||||||||||||||||||||||||||||||||||||
|
|
In reply to this post by Leila Tajik
|
|||||||||||||||||||||||||||||||||||||||||||
|
|
In reply to this post by Leila Tajik
why factor analysis is better choice than factor analysis in validating questionnaire? thanks a lot! 网易为中小企业免费提供企业邮箱（自主域名） |
|
|
|
Administrator
|
In reply to this post by SR Millis-3
There must be some really bad jokes that play on "rash" versus "Rasch". Let's not be Rasch here, etc. Does anyone know any? ;-)
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
|
|
In reply to this post by John F Hall
|
Re: Moving variables
|
Dear Humphrey,
I was unable to replicate your finding. In data view, I can select one or many columns (variables) and drag and drop them in order to move them. When I have selected them and keep my (left) mouse button pressed, an icon appears that suggests that this is a useless action but I can drag and drop them anyway, both in V17.0.3 as well as in v18.0.0. However, in order to change the order of variables in a dataset I'd always use match files file * /keep v1 v3 v2. execute. Since some syntax depends on the order of your variables, it may be wise to include a reordering of your variables in your syntax so your work stays replicable. Best regards and have a shiny happy weekend! Ruben van den Berg Methodologist TNS NIPO P: +31 20 522 5738 Date: Fri, 28 May 2010 03:07:53 -0700 From: [hidden email] Subject: Moving variables To: [hidden email]
Express yourself instantly with MSN Messenger! MSN Messenger |
|
match files file *
/keep v1 v3 v2 ALL. execute. On 5/28/2010 7:15 AM, Ruben van den Berg wrote: Dear Humphrey,===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Art Kendall
Social Research Consultants |
Comparing standard deviations
|
|
|
Re: Comparing standard deviations
|
|
How about the "coefficient of variation?" It is defined as the standard
deviation divided by the mean... wbw __________________________________________________________________________ William B. Ware, Professor Educational Psychology, CB# 3500 Measurement, and Evaluation University of North Carolina PHONE (919)-962-2511 Chapel Hill, NC 27599-3500 FAX: (919)-962-1533 Office: 118 Peabody Hall EMAIL: [hidden email] Adjunct Professor School of Social Work Academy of Distinguished Teaching Scholars at UNC-Chapel Hill __________________________________________________________________________ On Tue, 1 Jun 2010, Humphrey Paulie wrote: > > Dear all, > > I have several tests which have different number of questions (from 20 to > 100) so the maximum possible score on each test form is different. > Therefore, I cannot compare the standard deviations of the samples who have > taken each test form. What can I do to bring the standard deviations onto > the same scale and make them comparable? I want to know which test has > spread the subjectes more widely. > > Cheers > > Humphrey > > > > > > ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Re: Comparing standard deviations
|
|
In reply to this post by Humphrey Paulie
I agree with William. You should compare CV.
Take a look at this paper (sorry, you'll have to download each page one by one...). http://babel.hathitrust.org/cgi/pt?seq=3&view=pdf&size=100&id=umn.31951d03009506q&u=1&num=1 HTH, Marta GG Humphrey Paulie escribió: > > Dear all, > > I have several tests which have different number of questions (from 20 > to 100) so the maximum possible score on each test form is different. > Therefore, I cannot compare the standard deviations of the samples who > have taken each test form. What can I do to bring the standard > deviations onto the same scale and make them comparable? I want to > know which test has spread the subjectes more widely. > > Cheers > > Humphrey > > > > -- For miscellaneous SPSS related statistical stuff, visit: http://gjyp.nl/marta/ ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Re: Comparing standard deviations
|
|
This is a better link, where you can download the complete paper mentioned above
http://www.treesearch.fs.fed.us/pubs/31993 Lukas Fehrenbach 2010/6/1 Marta García-Granero <[hidden email]> I agree with William. You should compare CV. |
|
|
In reply to this post by Humphrey Paulie
Humphrey Dragging and dropping variables became more cumbersome starting
in v16 and hasn’t gotten much better in v18. I have spoken to SPSS
support about this and they gave me a bit of help, though there was still a bit
that was unresolved. If you are going to drag and drop variables, it is better to do
so in Variable View rather than Data View – that is to say, it is better
to drag and drop when the variables look like rows rather than columns. For
whatever reason, it is less processor intensive and the overall behavior of the
GUI is easier to work with when in Variable View. The issue that I noticed between “pre v16” (v15 and
earlier) and “post v16” (v16, v17, v18) was the speed with which
you can drag the variables. In “pre v16”, if you drag a
variable from the end of the file to the beginning of the file, the drag speed
appears to have an acceleration and a max speed – for lack of a specific
value, I’ll just call it “very fast”. In “post
v16”, if you drag a variable from the end of the file to the beginning of
the file, the drag speed appears to not have an acceleration and the speed is “pretty
slow”. There is a significant change in GUI behavior as well.
In “pre v16”, you can drag your cursor arrow the top of the screen in
Variable View (or the left of the screen in Data View) – which triggers
the “very fast” drag speed. In “post v16”, you
can only drag your cursor to the top of the area that displays the variables
(or the left of the area that displays the data). After you pass that ‘line’
in Variable View, then the drag speed comes to an immediate halt if your cursor
is in the area that shows the variable number OR it jumps to the beginning of
the file if your cursor is in the area that shows the variable name, type, etc.
After you pass that ‘line’ in Data View, then the drag
speed comes to an immediate halt if your cursor is in the area that shows the
variable name OR it jumps to the beginning of the file if your cursor is in the
area that shows your data. The “jumps to the beginning of the file”
behavior does not work for going to the end of the file. Also in “post v16”, if you are dragging and dropping
variables in Data View, selecting multiple columns doesn’t always work on
the first try. Specifically, if I drag a drop a variable (putting v4
before v3 for example), then I can’t immediately select multiple
variables. This gets a bit confusing with just words, so here’s an
example with variables names. Suppose I have v1 through v10 in the usual
order. Suppose I want to move v4 before v3 and then I want to move v5 through
v8 before v2. If I first move v4 before v3, I cannot immediately select
v5 & v6. I would first have to select some other variable – but
not the variable that I want to start dragging from. So, select v7.
Then, select v5 through v8 and drag them to before v2. I am not
sure if there are other circumstances that you can’t immediately select
multiple variables in Data View, but this is at least one that I can replicate. So, if you just need to move variables a short distance, then
drag and drop is fine. If you need to move a variable to the beginning of
a file, then drag and drop is fine. If you need to move a variable to the
middle or end of the file, then you should use the syntax Ruben suggested.
This, by the way, is the bit that was unresolved that I made reference to,
because the drag and drop functionality was previously able to handle this case
in a timely manner, but is now best suited to be handled via syntax. I hope that helps, -Eric Syntax to define a file with 1001 variables – just so you
can play around with moving variables a long distance. new file. data list /First 1. begin data 1 end data. dataset name temp window=front. vector v (F8.0, 1000). From: Ruben van den Berg
[mailto:[hidden email]] Dear Humphrey, Ruben van den Berg Methodologist TNS
NIPO P:
+31 20 522 5738
Date: Fri, 28 May 2010 03:07:53 -0700
Express
yourself instantly with MSN Messenger! MSN
Messenger |
Re: Comparing standard deviations
|
|
In reply to this post by William B. Ware
Humphrey, William, Marta,
When I read this the first thing that I thought of was that you (Humphrey) should calculate the score as the mean of the items because that would fix the range problem--provided all the tests have the same response format. And from that I thought you could compare the standard deviations. But then it occurred to me that the SDs are going to be slightly different for different numbers of items. My reasoning is this: Suppose the response format is the same for all forms, e.g.,1-5, a perfectly symmetric distribution for each item would give the same scale mean irrespective of the number of items. But, and I have not proof of this, just an intuition, the standard deviations for tests of varying lengths, even if the items have perfectly symmetric distributions, will be different because the distribution of scores for a long test is different from that for a short test because there is more possible values, and that will result in different standard deviations. At what decimal point it begin to matter, I don't know. If my reasoning is correct, it is only correct, I think, a for categorical response format. I don't think it would be true for a truly continuous response format. Is my thinking on this faulty? And, if my reasoning is correct, does using the CV actually fix this problem? Gene Maguin -----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of William B. Ware Sent: Tuesday, June 01, 2010 10:16 AM To: [hidden email] Subject: Re: Comparing standard deviations How about the "coefficient of variation?" It is defined as the standard deviation divided by the mean... wbw On Tue, 1 Jun 2010, Humphrey Paulie wrote: > > Dear all, > > I have several tests which have different number of questions (from 20 to > 100) so the maximum possible score on each test form is different. > Therefore, I cannot compare the standard deviations of the samples who have > taken each test form. What can I do to bring the standard deviations onto > the same scale and make them comparable? I want to know which test has > spread the subjectes more widely. > > Cheers > > Humphrey > > > > > > ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
In reply to this post by Langston, Eric
|
|
|
George- I’m very glad to help! And I will certainly help clarify. You do need to restate all the variables occurring before the
point where you want to drop the variable – but you can also use a shortcut for
that (you can use TO in the keep subcommand). Suppose that you wanted to move
v151 to after v10. Here’s the syntax: match files file=* /keep v1 to v10 v151 all. exe. Does that help? -Eric From: George Musick
[mailto:[hidden email]]
|
|
|
|
Re: Comparing standard deviations
|
|
In reply to this post by Lukas Fehrenbach-2
|
| Free forum by Nabble | Edit this page |