SPSSX Discussion

Version 15 on current Laptops

Classic

List

Threaded

30 messages Options

John F Hall

Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

Syntax is often quicker and easier for data transformations. See the relevant pages in 2.3: Data transformations on my website

http://surveyresearch.weebly.com/23-data-transformations.html .

John F Hall (Mr)

[retired academic survey researcher]

Email: [hidden email]

Website: www.surveyresearch.weebly.com

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Peter Spangler
Sent: 11 February 2013 21:59
To: [hidden email]
Subject: Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

Absolutely! As a new list serve member I am happy to see such an active group.

Peter

Sent from my iPhone

On Feb 11, 2013, at 12:02 PM, Hong A Ng <[hidden email]> wrote:

Hi J.R. and Peter,

Thank you so much for your speedy response! This worked beautifully. I didn't realize that you could recode so many variables at once ( I tried selecting multiple ones in the transform module and it didn't work so I assumed it was true for every other module). Thank you both so much! This really saved me a lot of time :)

Date: Mon, 11 Feb 2013 11:43:29 -0800
Subject: Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?
From: [hidden email]
To: [hidden email]
CC: [hidden email]

Why not simply Transform, Recode into Same Variables, Select all variables at once, Old and New Values, System Missing --> "0"
On Mon, Feb 11, 2013 at 11:36 AM, Hong A Ng <[hidden email]> wrote:
Hi all,

I have a very long an manual chore ahead of me unless I can find an easier approach to this.

I have many variables that are currently showing missing (.) that I need transformed into 0's instead. Besides doing this manually; how else can I get this done?

I thought of doing it with IF statements but then I would be repeating the below maybe several hundred times (there are a lot of variables that need replacing).

The code would look like this I'd imagine:

if (n15_1_5 is missing ) then n15_1_5=0 if (n15_2_5 is missing ) then n15_2_5=0 if (n15_3_5 is missing ) then n15_3_5=0 ...

I also thought about doing this in excel using the 'Replace' but I honestly don't want to do any merging that might jeopardize moving the data around and potentially making an error that way.

Anyone have a good idea on how to approach this in SPSS?

Merci!

J. R. Carroll-3

Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

I second John's comment, but would like to add the following clarifications:

"Syntax is often quicker and easier for data transformations"... **__IF__** you are familiar with SPSS syntax (and the transformations are... more verbose than just replacing "sysmis", for example). If you just need to get your work done without becoming an SPSS wizard or be syntax-literate, the menus are an immediate godsend. NB: the menus can also be a crutch - often causing headaches down the road if you choose to become an "SPSS junkie". Menus-4-newbies, syntax-4-junkies (should we turn that into a listserve t-shirt?)... syntax is also for those who are forced into positions were they sit at the focal point of all things that role down hill - note, the latter must enjoy brutal punishment in order to find SPSS syntax enjoyable =P. Put another way: my calves look fantastic from climbing the SPSS curve and while it's a great exercise, it's not something I would force (or offer as an option) on most people (aka undergraduate students who don't plan on making a career with SPSS, etc).

*puts his two cents on the ground*

-J

----

J. R. Carroll

Independent Researcher through Hurtz Labs

Research Methods, Test Development, and Statistics

www.jrcresearch.net

www.ontvp.com

Cell: (650) 776-6613

Email: [hidden email]

[hidden email]

On Tue, Feb 12, 2013 at 12:58 PM, John F Hall <[hidden email]> wrote:

Syntax is often quicker and easier for data transformations. See the relevant pages in 2.3: Data transformations on my website

http://surveyresearch.weebly.com/23-data-transformations.html .

John F Hall (Mr)
[retired academic survey researcher]

Email: [hidden email]

Website: www.surveyresearch.weebly.com

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Peter Spangler
Sent: 11 February 2013 21:59
To: [hidden email]

Subject: Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

Absolutely! As a new list serve member I am happy to see such an active group.

Peter

Sent from my iPhone

On Feb 11, 2013, at 12:02 PM, Hong A Ng <[hidden email]> wrote:

Hi J.R. and Peter,

Thank you so much for your speedy response! This worked beautifully. I didn't realize that you could recode so many variables at once ( I tried selecting multiple ones in the transform module and it didn't work so I assumed it was true for every other module). Thank you both so much! This really saved me a lot of time :)

Date: Mon, 11 Feb 2013 11:43:29 -0800
Subject: Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?
From: [hidden email]
To: [hidden email]
CC: [hidden email]

Why not simply Transform, Recode into Same Variables, Select all variables at once, Old and New Values, System Missing --> "0"
On Mon, Feb 11, 2013 at 11:36 AM, Hong A Ng <[hidden email]> wrote:

Hi all,

I have a very long an manual chore ahead of me unless I can find an easier approach to this.

I have many variables that are currently showing missing (.) that I need transformed into 0's instead. Besides doing this manually; how else can I get this done?

I thought of doing it with IF statements but then I would be repeating the below maybe several hundred times (there are a lot of variables that need replacing).

The code would look like this I'd imagine:

if (n15_1_5 is missing ) then n15_1_5=0 if (n15_2_5 is missing ) then n15_2_5=0 if (n15_3_5 is missing ) then n15_3_5=0 ...

I also thought about doing this in excel using the 'Replace' but I honestly don't want to do any merging that might jeopardize moving the data around and potentially making an error that way.

Anyone have a good idea on how to approach this in SPSS?

Merci!

Barnett, Adrian (DECD)

Re: Version 15 on current Laptops

In reply to this post by Rick Oliver-3

Hi Rick

What I was referring to is the fact that large amounts of information have been removed from the PDF manuals. These are now syntax references only.

I have just pulled out the manuals for Advanced Statistics, Professional Statistics and Trends for version 6.1 (1994).

Advanced Statistics has 309 pages of explanation and examples before it gets to the syntax Reference section.

Trends has 247 and Professional Statistics has 170.

Yes, the Statistics Coach does cover some of this, but:

(1) It takes over 5 minutes to open given the slowness of the server at the other end; it’s just awful and I doubt many people would be prepared to endure this on a regular basis.

(2) It is not the same amount of detail

This material would have been much more helpful had it been left in the PDF manuals. Given that IBM no longer supply printed manuals (or even a CD!), there can be no argument on cost grounds for removing it from the standard distribution.

Basically, if you want good coverage of what everything means, and how to interpret it all, you are much better off shelling out for Marija Norusis’ manuals.

Certainly IBM does not require a fee for this – it’s the choice of the individual whether they do so or not, but it is essentially necessary if you want a good understanding, because the removal of the very good explanatory material from the manuals has not been adequately compensated for by the Statistics Coach and Help. These are simply not good enough substitutes.

Regards,

Adrian

Adrian Barnett

Project Officer

Educational Measurement and Analysis

Data and Information Systems

Department for Education and Child Development

“Children and young people are at the centre of everything we do”

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Rick Oliver
Sent: Tuesday, 12 February 2013 5:10 AM
To: [hidden email]
Subject: Re: Version 15 on current Laptops

As fa as I know, there is no documentation with current releases of the product that requires an additional fee. There are no longer any printed manuals, and all the documentation is available from the link Alex posted, including manuals in PDF form. As noted, these are also shipped with the product. In general, the help system has more information than the PDF manuals. If the specific case of syntax reference material in release 15, the complete Command Syntax Reference in PDF form should be available from the Help menu, if my memory is correct.

Rick Oliver
Senior Information Developer
IBM Business Analytics (SPSS)
E-mail: [hidden email]

From: Alex Reutter/Burlington/IBM@IBMUS
To: [hidden email],
Date: 02/11/2013 12:02 PM
Subject: Re: Version 15 on current Laptops
Sent by: "SPSSX(r) Discussion" <[hidden email]>

Hi Adrian,

There are examples in the online help; for example, go to http://pic.dhe.ibm.com/infocenter/spssstat/v21r0m0/index.jsp and look under "Case Studies" in the table of contents on the left. These "Case Studies" are also part of the help that is installed with the product.

Alex

Adrian Barnett wrote:
(I noted with some chagrin that where once everything was in the SPSS manuals, some years ago they pulled out lots of explanatory stuff and examples from the statistical procedures sections and put them into separate manuals you have to pay for. What you get is basically a terse syntax reference. Not their finest achievement. When you are paying through the nose for something, all the documentation should be part of it, not an extra expense)

John F Hall

Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

In reply to this post by J. R. Carroll-3

Justin

“aka undergraduate students who don't plan on making a career with SPSS,”

The SPSS-based course on my site was actually a compulsory module for BSC Social Science and BSc Sociology. It aimed to produce data literate social scientists: SPSS was incidental, but the best available software at the time (and, as far as I am concerned, still is).

There is a new initiative in the UK group (backed by the Economic and Social Research Council and the Nuffield Foundation) on teaching quantitative methods to undergraduate social science students. It is convened by:

Professor John MacInnes
ESRC Strategic Advisor on Quantitative Methods Training
School of Social and Political Science,
Room 5.05
Chrystal Macmillan Building
University of Edinburgh
Edinburgh EH8 9LD
+44 (0)131 651 3867

. . and has been soliciting suggestions for recommended introductory texts, teaching ideas and other resources. ([hidden email] to subscribe)

Survey research and SPSS figure among the most productive suggestions offered so far, but many contributors are still approaching the issue from a (mathematical) statistical standpoint rather than from sociology, political science and the like. Logic and process should come first: equations can come later. SPSS offers one route to acquiring the necessary skills.

Professor McInnes writes (30 Jan 2013)

The new portal for web resources for QM teaching has gone live. It should function with any of the major browsers, as well as tablets and phone.

It includes the ability to review and rate resources to help guide other users to the most useful stuff, and the ability to recommend new resources, and add details of training events or relevant news.

I'm sure there will be the odd teething trouble, and I'm even more sure that there is good, obvious stuff out there that I have missed. So do look around the site and make suggestions. The aim is for the site to evolve over time as resources and the teaching environment change.

The web address is

http://www.quantitativemethods.ac.uk

Kandy Woofield (Natcen, 7 Feb 2013) writes:

One element of our ESRC funded RDI project includes providing an online space for lecturers to share their ideas, challenges and innovations around QM teaching. The platform is currently being used by participants in our 2012 cohort but we'd love to broaden the group and hear from other lecturers who are interested in the debates around how to engage students with QM. If you'd like to sign up for the platform you can do that here: http://tinyurl.com/ahgjboa

We see this platform as complementary to the new QM portal for resources

A persuasive flyer (from the British Academy) for social scientists to acquire QM skills is:

<a href="cid:image003.jpg@01CE052E.6B367A90">cid:image003.jpg@01CE052E.6B367A90]<http://www.britac.ac.uk/policy/Stand_Out_and_Be_Counted.cfm>

John F Hall (Mr)

[retired academic survey researcher]

Email: [hidden email]

Website: www.surveyresearch.weebly.com

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of J. R. Carroll
Sent: 12 February 2013 19:18
To: [hidden email]
Subject: Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

I second John's comment, but would like to add the following clarifications:

*puts his two cents on the ground*

-J

----

J. R. Carroll

Independent Researcher through Hurtz Labs

Research Methods, Test Development, and Statistics

www.jrcresearch.net

www.ontvp.com

Cell: (650) 776-6613

Email: [hidden email]

[hidden email]

On Tue, Feb 12, 2013 at 12:58 PM, John F Hall <[hidden email]> wrote:

Syntax is often quicker and easier for data transformations. See the relevant pages in 2.3: Data transformations on my website

http://surveyresearch.weebly.com/23-data-transformations.html .

John F Hall (Mr)

[retired academic survey researcher]

Email: [hidden email]

Website: www.surveyresearch.weebly.com

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Peter Spangler
Sent: 11 February 2013 21:59
To: [hidden email]

Subject: Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

Absolutely! As a new list serve member I am happy to see such an active group.

Peter

Sent from my iPhone

On Feb 11, 2013, at 12:02 PM, Hong A Ng <[hidden email]> wrote:

Hi J.R. and Peter,

Thank you so much for your speedy response! This worked beautifully. I didn't realize that you could recode so many variables at once ( I tried selecting multiple ones in the transform module and it didn't work so I assumed it was true for every other module). Thank you both so much! This really saved me a lot of time :)

Date: Mon, 11 Feb 2013 11:43:29 -0800
Subject: Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?
From: [hidden email]
To: [hidden email]
CC: [hidden email]

Why not simply Transform, Recode into Same Variables, Select all variables at once, Old and New Values, System Missing --> "0"
On Mon, Feb 11, 2013 at 11:36 AM, Hong A Ng <[hidden email]> wrote:
Hi all,

I have a very long an manual chore ahead of me unless I can find an easier approach to this.

I have many variables that are currently showing missing (.) that I need transformed into 0's instead. Besides doing this manually; how else can I get this done?

I thought of doing it with IF statements but then I would be repeating the below maybe several hundred times (there are a lot of variables that need replacing).

The code would look like this I'd imagine:

if (n15_1_5 is missing ) then n15_1_5=0 if (n15_2_5 is missing ) then n15_2_5=0 if (n15_3_5 is missing ) then n15_3_5=0 ...

I also thought about doing this in excel using the 'Replace' but I honestly don't want to do any merging that might jeopardize moving the data around and potentially making an error that way.

Anyone have a good idea on how to approach this in SPSS?

Merci!

LaurenceMoseley

Re: [Quantitative Methods Teaching] Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

GUI and Syntax files

Simple changes over time

When you come back to a project after, say, a year of analytical inactivity (because data are being gathered, you have been overseas, the main quant guy was off sick, or whatever reason) it is very easy to forget exactly what you did a year or longer ago. It is often easier still to misunderstand what someone else did a year or more ago. In data analysis terms the “exactly” is important. What did you do with the missing values, for example? To fail to replicate that exactly can be disastrous.

I have seen cases where children’s’ ages were coded as integers and the value 99 was recorded for any missing cases. Unfortunately, although 99 meant “missing value” no one had bothered to tell the computer that fact i.e. it had not been declared as a missing value of any kind. As a result the poor researchers suddenly found that there were classes in primary schools in which the mean age was 32.5, for example. If the earlier operations had been recorded in a syntax file, it is usually easy to spot the causes of such apparent mysteries.

Spotting errors

Syntax files often help you to understand what was previously done. I once had to re-analyse some data from 9 large-scale surveys which had previously been analysed by someone else. The results from that someone were mysterious. It was only when I looked at the data files (not a syntax file in this case) that I spotted that whereas the coding frame said that 1=”Yes” and 2= “No”, the previous analyst had labelled them (in the GUI) 1 as “No” and 2 as “Yes” i.e. the wrong way round. It had been through several hands and no-one else had spotted that infelicity. In syntax that would have been very obvious, and easy to correct – just change the syntax, run it again, and save the data file – end of story.

What happens when you have several transformations?

When there are a number of transformations to be undertaken and a number of outputs to be produced, it is very easy to forget what you did last time. A typical example is when you have a Likert-appearing scale running from, say 1= “Strongly agree” to 5=”Strongly disagree”. You can easily print out frequencies about agreement with different question stubs.

However, one often has (and it is good practice to have) some statements which are positive, e.g. “I loved the course”, and others which are negative e.g. “I hated the course”. A positive opinion arises when either a respondent agrees with a positive statement or disagrees with a negative one. So if one wants to measure, not the agreement, but positive or negative opinions, one has to reverse the codes on some questions. The simplest way would be something like:

Compute Reversed_Var = (Max_value +1) – Observed_Var.

(For a 5-point scale that would give you (5+1) – 5 = 1, (5+1) -4 = 2, (5+1) -3 = 3, (5+1)-2 = 4, and (5+1) – 1 = 5) and can be applied to a large number of variables in one Compute statement.

Alternatively, albeit less elegantly, you could do a bunch of Recodes such as

Recode Old_var1 (5=1) (4=2) (3=3) (2=4) (1=5) probably using a Recode into new variable and doing it as a list of many variables at once.

That is easy to do in syntax. If you have many transformations you can simply type in the first one, copy it several times, and edit the copies. That avoids many of the errors that occur when you use the GUI to do the same job – and the syntax files can be reviewed by someone else, say the PI.

What happens if the order of processes matters?

If you are then going to use these reversed variables to form some sort of scale, and use that scale for some further calculations, it matters whether you do the calculations before or after the reversing. If what you did last year is recorded in a syntax file, it is easy to reproduce the operations exactly.

If you wish to make some changes this year (e.g. the age of eligibility for something has changed, the level of benefit is different, people who are eligible now form a different group – the possibilities are endless) then it is easy merely to edit the syntax file, save it (under a new and dated meaningful name), and run it to produce the updated results. If you had to rely on the memory of last year’s researcher (who by then may well have left the department, university, country, or even be dead) you will have a problem. If it has all been recorded in syntax it is easy either to reproduce or modify it.

The epistemological dimension

The reason why we set high methodological standards is not some high ideological position, but a mundane and very simple one. People make mistakes. We are poor observers (just read the Invisible Gorilla), we are poor at interpreting what we have observed (just read Gilovich, Kahneman and the many other summaries of the relevant research), and we are particularly poor at handling confounding variables.

Many of the problems that researchers have to solve involve many variables, and more importantly those variables interact. It is not uncommon for a real-world research problem to have a search space of 10⁷ or larger. I have worked on one problem in which the search space (the number of possible outcomes in this case) was 10²⁹.

To solve those problems using common-sense, intuition, clinical judgement, or other non-replicable methods, we are limited by Miller’s Magical Number. That represents the number of items that we can hold and actively manipulate in our short-term memory at any one time. Over 50 years of research has repeatedly estimated its value at 7 ± 2.

The inequality is striking:

Problem: 10⁷ or larger

Solution: 7 ± 2 or 10⁰. That is 7 orders of magnitude of difference.

It is hardly surprising that when we have to tackle such problems with our unaided minds we use heuristics rather than logic, and fallible heuristics at that (representativeness, recency, salience, negative evidence, false causal claims, the list is endless). Not to use the tools available (especially statistics, algorithms, and computers) is at best indicative of an amateur approach.

Our main safeguard against such amateurism is the principle of replicability. If other people tackle the same problem and come to a different conclusion then either their method is different, reality is different, or one of us has made a mistake. For the part of the solutions that require some statistical analysis, syntax files provide exact replicability. They avoid the dangers of relying on our own or someone else’s memory. Using syntax may get it wrong, but it won’t get it different.

They also permit us to avoid silly errors when we undertake any comparative studies. If you apply the same questionnaire to two different hospitals, say, or at two different points in time, one form of error that you should and can avoid (even if the data are gathered in identical ways) is to undertake the analysis in a different way in the two situations. If you have the first analysis in a syntax file, you can merely change the name of the data file to cover the second hospital, point in time, etc. and re-run the program.

If the two sets of results differ then you can conclude that the two situations differ and that they are not due to your analysis being different in those two situations. I think that we should be sharing code more than we do at the moment.

At least it is something that we can do. We cannot as accurately share oral memories.

Implications for teaching

I think that no student should be awarded a degree unless they have written (and interpreted the results from) at least one program which includes not only output procedures but also some data transformations. We have plenty of data sets available free of charge from ESDS/UKDA. After using such data sets, they should then be able to analyze their own data.

With research students, as soon as they have data on, say, their first 50 cases, I ask them to write a program to analyze the data. That means that (a) they learn analysis and (b) they can understand the output. A year or two later when they have their full sample they can simply re-run the program (with the appropriate change of data file name). After all, the program should work and they should be able to understand the output.

The task is not very burdensome. After all, with SPSS one can simply issue the GUI commands, copy the automatically generated syntax from the output file, and paste it into one’s syntax file. That is an efficient way of learning. I, for example, find it difficult to remember the details of the syntax to read a data file in Excel and the GUI does the work for me.

Summary

There is a case for students learning SPSS syntax as well as the GUI, and using that syntax. Of the arguments that I have given above, the epistemological one about replicability is the most important one, but the others all have merit.

Laurie Moseley

University of Glamorgan

14rh February 2013

From: [hidden email] [mailto:[hidden email]] On Behalf Of John F Hall
Sent: 13 February 2013 08:54
To: [hidden email]
Cc: [hidden email]; [hidden email]
Subject: Re: [Quantitative Methods Teaching] Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

Justin

“aka undergraduate students who don't plan on making a career with SPSS,”

. . and has been soliciting suggestions for recommended introductory texts, teaching ideas and other resources. ([hidden email] to subscribe)

Professor McInnes writes (30 Jan 2013)

The new portal for web resources for QM teaching has gone live. It should function with any of the major browsers, as well as tablets and phone.

The web address is

http://www.quantitativemethods.ac.uk

Kandy Woofield (Natcen, 7 Feb 2013) writes:

A persuasive flyer (from the British Academy) for social scientists to acquire QM skills is:

<a href="cid:image003.jpg@01CE052E.6B367A90">cid:image003.jpg@01CE052E.6B367A90]<http://www.britac.ac.uk/policy/Stand_Out_and_Be_Counted.cfm>

John F Hall (Mr)

[retired academic survey researcher]

Email: [hidden email]

Website: www.surveyresearch.weebly.com

From: SPSSX(r) Discussion [[hidden email]] On Behalf Of J. R. Carroll
Sent: 12 February 2013 19:18
To: [hidden email]
Subject: Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

I second John's comment, but would like to add the following clarifications:

*puts his two cents on the ground*

-J

----

J. R. Carroll

Independent Researcher through Hurtz Labs

Research Methods, Test Development, and Statistics

www.jrcresearch.net

www.ontvp.com

Cell: (650) 776-6613

Email: [hidden email]

[hidden email]

On Tue, Feb 12, 2013 at 12:58 PM, John F Hall <[hidden email]> wrote:

Syntax is often quicker and easier for data transformations. See the relevant pages in 2.3: Data transformations on my website

http://surveyresearch.weebly.com/23-data-transformations.html .

John F Hall (Mr)

[retired academic survey researcher]

Email: [hidden email]

Website: www.surveyresearch.weebly.com

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Peter Spangler
Sent: 11 February 2013 21:59
To: [hidden email]

Subject: Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

Absolutely! As a new list serve member I am happy to see such an active group.

Peter

Sent from my iPhone

On Feb 11, 2013, at 12:02 PM, Hong A Ng <[hidden email]> wrote:

Hi J.R. and Peter,

Thank you so much for your speedy response! This worked beautifully. I didn't realize that you could recode so many variables at once ( I tried selecting multiple ones in the transform module and it didn't work so I assumed it was true for every other module). Thank you both so much! This really saved me a lot of time :)

Date: Mon, 11 Feb 2013 11:43:29 -0800
Subject: Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?
From: [hidden email]
To: [hidden email]
CC: [hidden email]

Why not simply Transform, Recode into Same Variables, Select all variables at once, Old and New Values, System Missing --> "0"
On Mon, Feb 11, 2013 at 11:36 AM, Hong A Ng <[hidden email]> wrote:
Hi all,

I have a very long an manual chore ahead of me unless I can find an easier approach to this.

I have many variables that are currently showing missing (.) that I need transformed into 0's instead. Besides doing this manually; how else can I get this done?

I thought of doing it with IF statements but then I would be repeating the below maybe several hundred times (there are a lot of variables that need replacing).

The code would look like this I'd imagine:

if (n15_1_5 is missing ) then n15_1_5=0 if (n15_2_5 is missing ) then n15_2_5=0 if (n15_3_5 is missing ) then n15_3_5=0 ...

I also thought about doing this in excel using the 'Replace' but I honestly don't want to do any merging that might jeopardize moving the data around and potentially making an error that way.

Anyone have a good idea on how to approach this in SPSS?

Merci!

John F Hall

Re: [Quantitative Methods Teaching] Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

Laurence

Nice contribution. I still have most of the syntax jobs I ever wrote, but some were lost forever when my institution did its spring-cleaning without letting me have backup tapes rom before 1986. That’s how I was able to recreate the Quality of Life in Britain surveys I did from 1971 to 1975 (I’d been using the data after 1986) together with a host of others for which a colleague in IT services thoughtfully provided back-up tapes when she knew I was taking early retirement in 1992. By then Essex no longer had a Dec or Vax, but they did transfer all the files from 7 tapes to 2 CDs for me. Quite a bit of tweaking needed to convert 1972 syntax for use with SPSS 11, 15, 18, 19 for Windows, but it’s all there on my website for posterity to enjoy.

On forcing students to use syntax, see the syllabus and specimen assessment on my site. The examples are from 1992, but the same syllabus and assessment format was used for 10 years or so before then. I can’t give a direct URL for them, but you can access them from the links on:

http://surveyresearch.weebly.com/background-to-current-project.html

Survey Analysis Workshop – Syllabus

Syllabus for postgraduate course Survey Analysis Workshop (using SPSS) taught at Polytechnic of North London as at March 1992. Forms basis for updated course materials on this website

Survey Analysis Workshop: specimen assessment

Specimen assessment for course, as distributed to students. Gives an idea of the sort of things you should understand and have learned to do by the end of the course.

Your reply was a bit long to reproduce, and everyone on the lists will have it anyway, so I’ve truncated this message.

Best

John F Hall (Mr)

[retired academic survey researcher]

Email: [hidden email]

Website: www.surveyresearch.weebly.com

Hart, Kimberly (hartkb)

Automatic reply: [Quantitative Methods Teaching] Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

I will be out of office until Tuesday, February 19, 2013, with limited access to email. I will respond to your email when I return.

Best, Kim

Lorin Drake

Automatic reply: [Quantitative Methods Teaching] Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

In reply to this post by John F Hall

I am traveling on business and will return to the office on Monday 2/18. I will have occasional email access.

For more immediate assistance, please call our main switchboard and someone will direct your call.

(813) 207-0332

Art Kendall

Re: [Quantitative Methods Teaching] Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

In reply to this post by LaurenceMoseley

Excellent post.

I do have a quibble about the time frame. I have been using SPSS since 1972 and consulted on it since 1973. My experience for myself and my clients is that the time span for memory loss is more in the order of a phone call, a call of nature, or a coffee break than it is in the order of a year.

I would also add the usefulness of syntax in getting help and in the quality assurance review that due diligence calls for. [see the archives for my soapboxes on syntax as a communication aid and the importance of have teams that do QA reviews of each others work right from the beginning of SPSS use.]

Art Kendall
Social Research Consultants

On 2/14/2013 1:43 PM, Laurence Moseley wrote:

GUI and Syntax files

Simple changes over time

When you come back to a project after, say, a year of analytical inactivity (because data are being gathered, you have been overseas, the main quant guy was off sick, or whatever reason) it is very easy to forget exactly what you did a year or longer ago. It is often easier still to misunderstand what someone else did a year or more ago. In data analysis terms the “exactly” is important. What did you do with the missing values, for example? To fail to replicate that exactly can be disastrous.

I have seen cases where children’s’ ages were coded as integers and the value 99 was recorded for any missing cases. Unfortunately, although 99 meant “missing value” no one had bothered to tell the computer that fact i.e. it had not been declared as a missing value of any kind. As a result the poor researchers suddenly found that there were classes in primary schools in which the mean age was 32.5, for example. If the earlier operations had been recorded in a syntax file, it is usually easy to spot the causes of such apparent mysteries.

Spotting errors

Syntax files often help you to understand what was previously done. I once had to re-analyse some data from 9 large-scale surveys which had previously been analysed by someone else. The results from that someone were mysterious. It was only when I looked at the data files (not a syntax file in this case) that I spotted that whereas the coding frame said that 1=”Yes” and 2= “No”, the previous analyst had labelled them (in the GUI) 1 as “No” and 2 as “Yes” i.e. the wrong way round. It had been through several hands and no-one else had spotted that infelicity. In syntax that would have been very obvious, and easy to correct – just change the syntax, run it again, and save the data file – end of story.

What happens when you have several transformations?

When there are a number of transformations to be undertaken and a number of outputs to be produced, it is very easy to forget what you did last time. A typical example is when you have a Likert-appearing scale running from, say 1= “Strongly agree” to 5=”Strongly disagree”. You can easily print out frequencies about agreement with different question stubs.

However, one often has (and it is good practice to have) some statements which are positive, e.g. “I loved the course”, and others which are negative e.g. “I hated the course”. A positive opinion arises when either a respondent agrees with a positive statement or disagrees with a negative one. So if one wants to measure, not the agreement, but positive or negative opinions, one has to reverse the codes on some questions. The simplest way would be something like:

Compute Reversed_Var = (Max_value +1) – Observed_Var.

(For a 5-point scale that would give you (5+1) – 5 = 1, (5+1) -4 = 2, (5+1) -3 = 3, (5+1)-2 = 4, and (5+1) – 1 = 5) and can be applied to a large number of variables in one Compute statement.

Alternatively, albeit less elegantly, you could do a bunch of Recodes such as

Recode Old_var1 (5=1) (4=2) (3=3) (2=4) (1=5) probably using a Recode into new variable and doing it as a list of many variables at once.

That is easy to do in syntax. If you have many transformations you can simply type in the first one, copy it several times, and edit the copies. That avoids many of the errors that occur when you use the GUI to do the same job – and the syntax files can be reviewed by someone else, say the PI.

What happens if the order of processes matters?

If you are then going to use these reversed variables to form some sort of scale, and use that scale for some further calculations, it matters whether you do the calculations before or after the reversing. If what you did last year is recorded in a syntax file, it is easy to reproduce the operations exactly.

If you wish to make some changes this year (e.g. the age of eligibility for something has changed, the level of benefit is different, people who are eligible now form a different group – the possibilities are endless) then it is easy merely to edit the syntax file, save it (under a new and dated meaningful name), and run it to produce the updated results. If you had to rely on the memory of last year’s researcher (who by then may well have left the department, university, country, or even be dead) you will have a problem. If it has all been recorded in syntax it is easy either to reproduce or modify it.

The epistemological dimension

The reason why we set high methodological standards is not some high ideological position, but a mundane and very simple one. People make mistakes. We are poor observers (just read the Invisible Gorilla), we are poor at interpreting what we have observed (just read Gilovich, Kahneman and the many other summaries of the relevant research), and we are particularly poor at handling confounding variables.

Many of the problems that researchers have to solve involve many variables, and more importantly those variables interact. It is not uncommon for a real-world research problem to have a search space of 10⁷ or larger. I have worked on one problem in which the search space (the number of possible outcomes in this case) was 10²⁹.

To solve those problems using common-sense, intuition, clinical judgement, or other non-replicable methods, we are limited by Miller’s Magical Number. That represents the number of items that we can hold and actively manipulate in our short-term memory at any one time. Over 50 years of research has repeatedly estimated its value at 7 ± 2.

The inequality is striking:

Problem:            10⁷ or larger

Solution:            7 ± 2 or 10⁰. That is 7 orders of magnitude of difference.

It is hardly surprising that when we have to tackle such problems with our unaided minds we use heuristics rather than logic, and fallible heuristics at that (representativeness, recency, salience, negative evidence, false causal claims, the list is endless). Not to use the tools available (especially statistics, algorithms, and computers) is at best indicative of an amateur approach.

Our main safeguard against such amateurism is the principle of replicability. If other people tackle the same problem and come to a different conclusion then either their method is different, reality is different, or one of us has made a mistake. For the part of the solutions that require some statistical analysis, syntax files provide exact replicability. They avoid the dangers of relying on our own or someone else’s memory. Using syntax may get it wrong, but it won’t get it different.

They also permit us to avoid silly errors when we undertake any comparative studies. If you apply the same questionnaire to two different hospitals, say, or at two different points in time, one form of error that you should and can avoid (even if the data are gathered in identical ways) is to undertake the analysis in a different way in the two situations. If you have the first analysis in a syntax file, you can merely change the name of the data file to cover the second hospital, point in time, etc. and re-run the program.

If the two sets of results differ then you can conclude that the two situations differ and that they are not due to your analysis being different in those two situations. I think that we should be sharing code more than we do at the moment.

At least it is something that we can do. We cannot as accurately share oral memories.

Implications for teaching

I think that no student should be awarded a degree unless they have written (and interpreted the results from) at least one program which includes not only output procedures but also some data transformations. We have plenty of data sets available free of charge from ESDS/UKDA. After using such data sets, they should then be able to analyze their own data.

With research students, as soon as they have data on, say, their first 50 cases, I ask them to write a program to analyze the data. That means that (a) they learn analysis and (b) they can understand the output. A year or two later when they have their full sample they can simply re-run the program (with the appropriate change of data file name). After all, the program should work and they should be able to understand the output.

The task is not very burdensome. After all, with SPSS one can simply issue the GUI commands, copy the automatically generated syntax from the output file, and paste it into one’s syntax file. That is an efficient way of learning. I, for example, find it difficult to remember the details of the syntax to read a data file in Excel and the GUI does the work for me.

Summary

There is a case for students learning SPSS syntax as well as the GUI, and using that syntax. Of the arguments that I have given above, the epistemological one about replicability is the most important one, but the others all have merit.

Laurie Moseley

University of Glamorgan

14rh February 2013

From: [hidden email] [[hidden email]] On Behalf Of John F Hall
Sent: 13 February 2013 08:54
To: [hidden email]
Cc: [hidden email]; [hidden email]
Subject: Re: [Quantitative Methods Teaching] Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

Justin

“aka undergraduate students who don't plan on making a career with SPSS,”

The SPSS-based course on my site was actually a compulsory module for BSC Social Science and BSc Sociology. It aimed to produce data literate social scientists: SPSS was incidental, but the best available software at the time (and, as far as I am concerned, still is).

There is a new initiative in the UK group (backed by the Economic and Social Research Council and the Nuffield Foundation) on teaching quantitative methods to undergraduate social science students. It is convened by:

Professor John MacInnes
ESRC Strategic Advisor on Quantitative Methods Training
School of Social and Political Science,
Room 5.05
Chrystal Macmillan Building
University of Edinburgh
Edinburgh EH8 9LD
+44 (0)131 651 3867

. . and has been soliciting suggestions for recommended introductory texts, teaching ideas and other resources. ([hidden email] to subscribe)

Survey research and SPSS figure among the most productive suggestions offered so far, but many contributors are still approaching the issue from a (mathematical) statistical standpoint rather than from sociology, political science and the like. Logic and process should come first: equations can come later. SPSS offers one route to acquiring the necessary skills.

Professor McInnes writes (30 Jan 2013)

The new portal for web resources for QM teaching has gone live. It should function with any of the major browsers, as well as tablets and phone.

It includes the ability to review and rate resources to help guide other users to the most useful stuff, and the ability to recommend new resources, and add details of training events or relevant news.

I'm sure there will be the odd teething trouble, and I'm even more sure that there is good, obvious stuff out there that I have missed. So do look around the site and make suggestions. The aim is for the site to evolve over time as resources and the teaching environment change.

The web address is

http://www.quantitativemethods.ac.uk

Kandy Woofield (Natcen, 7 Feb 2013) writes:

One element of our ESRC funded RDI project includes providing an online space for lecturers to share their ideas, challenges and innovations around QM teaching. The platform is currently being used by participants in our 2012 cohort but we'd love to broaden the group and hear from other lecturers who are interested in the debates around how to engage students with QM. If you'd like to sign up for the platform you can do that here: http://tinyurl.com/ahgjboa

We see this platform as complementary to the new QM portal for resources

See also http://www.esrc.ac.uk/funding-and-guidance/tools-and-resources/undergraduate-QM/index.aspx

A persuasive flyer (from the British Academy) for social scientists to acquire QM skills is:

<a moz-do-not-send="true" href="cid:image003.jpg@01CE052E.6B367A90">cid:image003.jpg@01CE052E.6B367A90]<http://www.britac.ac.uk/policy/Stand_Out_and_Be_Counted.cfm>

John F Hall (Mr)

[retired academic survey researcher]

Email:   [hidden email]

Website: www.surveyresearch.weebly.com

From: SPSSX(r) Discussion [[hidden email]] On Behalf Of J. R. Carroll
Sent: 12 February 2013 19:18
To: [hidden email]
Subject: Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

I second John's comment, but would like to add the following clarifications:

"Syntax is often quicker and easier for data transformations"... **__IF__** you are familiar with SPSS syntax (and the transformations are... more verbose than just replacing "sysmis", for example). If you just need to get your work done without becoming an SPSS wizard or be syntax-literate, the menus are an immediate godsend. NB: the menus can also be a crutch - often causing headaches down the road if you choose to become an "SPSS junkie". Menus-4-newbies, syntax-4-junkies (should we turn that into a listserve t-shirt?)... syntax is also for those who are forced into positions were they sit at the focal point of all things that role down hill - note, the latter must enjoy brutal punishment in order to find SPSS syntax enjoyable =P. Put another way: my calves look fantastic from climbing the SPSS curve and while it's a great exercise, it's not something I would force (or offer as an option) on most people (aka undergraduate students who don't plan on making a career with SPSS, etc).

*puts his two cents on the ground*

-J

----

J. R. Carroll

Independent Researcher through Hurtz Labs

Research Methods, Test Development, and Statistics

www.jrcresearch.net

www.ontvp.com

Cell: (650) 776-6613

Email: [hidden email]

   [hidden email]

   [hidden email]

On Tue, Feb 12, 2013 at 12:58 PM, John F Hall <[hidden email]> wrote:

Syntax is often quicker and easier for data transformations. See the relevant pages in 2.3: Data transformations on my website

http://surveyresearch.weebly.com/23-data-transformations.html .

John F Hall (Mr)

[retired academic survey researcher]

Email:   [hidden email]

Website: www.surveyresearch.weebly.com

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Peter Spangler
Sent: 11 February 2013 21:59
To: [hidden email]

Subject: Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

Absolutely! As a new list serve member I am happy to see such an active group.

Peter

Sent from my iPhone

On Feb 11, 2013, at 12:02 PM, Hong A Ng <[hidden email]> wrote:

Hi J.R. and Peter,

Thank you so much for your speedy response! This worked beautifully. I didn't realize that you could recode so many variables at once ( I tried selecting multiple ones in the transform module and it didn't work so I assumed it was true for every other module). Thank you both so much! This really saved me a lot of time :)

Date: Mon, 11 Feb 2013 11:43:29 -0800
Subject: Re: Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?
From: [hidden email]
To: [hidden email]
CC: [hidden email]

Why not simply Transform, Recode into Same Variables, Select all variables at once, Old and New Values, System Missing --> "0"

On Mon, Feb 11, 2013 at 11:36 AM, Hong A Ng <[hidden email]> wrote:

Hi all,

I have a very long an manual chore ahead of me unless I can find an easier approach to this.

I have many variables that are currently showing missing (.) that I need transformed into 0's instead. Besides doing this manually; how else can I get this done?

I thought of doing it with IF statements but then I would be repeating the below maybe several hundred times (there are a lot of variables that need replacing).

The code would look like this I'd imagine:

if (n15_1_5 is missing ) then n15_1_5=0 if (n15_2_5 is missing ) then n15_2_5=0 if (n15_3_5 is missing ) then n15_3_5=0 ...

I also thought about doing this in excel using the 'Replace' but I honestly don't want to do any merging that might jeopardize moving the data around and potentially making an error that way.

Anyone have a good idea on how to approach this in SPSS?

Merci!

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Art Kendall
Social Research Consultants

Jon K Peck

Re: [Quantitative Methods Teaching] Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?

While I have no quibble with people learning syntax - although the time for that could be debated, I would like to point out that SPSS does have tools that are more focused on the output side that will sometimes be more effective than reviewing the syntax. For example, the CODEBOOK procedure is likely to be a better way to spot errors in the metadata than reading through large volumes of syntax. COMPARE DATASETS can be used to check the data. VALIDATEDATA can show whether the data conform to expected rules; the journal file can show what was done even when using only point and click; custom attributes can be used to record transformation formulas (and there is a Python module that can be used for transformations that will generate these attributes automatically). Notes tables contain the syntax behind point and click. And, of course, reviewing statistics on the data to make sure that they make sense.

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: 720-342-5621

From: Art Kendall <[hidden email]>
To: [hidden email],
Date: 02/15/2013 07:31 AM
Subject: Re: [SPSSX-L] [Quantitative Methods Teaching] Replacing numerous variables that have 'missing values' with 0? Can this be done with IF statements?
Sent by: "SPSSX(r) Discussion" <[hidden email]>