Reverse engineering from *.sav to *.sps

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

Reverse engineering from *.sav to *.sps

John F Hall

I have produced a massive 1,04 gb beta version “mother” file for the 1983-2014 waves of British Social Attitudes.  The variable labels are different for each wave and ADD FILES picks them up from the first file encountered.  I don’t fancy wading through almost 11,000 variables to check/change them.  Is there a way of producing a VARIABLE LABELS list from the *.sav file?  I vaguely remember something from the 1970s called WRITE FILEINFO, but it’s not in the FM.  One alternative is to find the file with the most suitable labels and use that first when adding the other files: another would be to copy the contents of the Name and Labels columns into a syntax file.

 

I need to send the file to Natcen and UKDS.  If it won’t go via Dropbox, can anyone suggest a secure alternative?

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

Jon Peck
It would be possible to extract such a list using programmability, but it would be much easier to just use APPLY DICTIONARY to copy any properties you want from one sav file to others.

On Fri, Apr 15, 2016 at 11:26 AM, John F Hall <[hidden email]> wrote:

I have produced a massive 1,04 gb beta version “mother” file for the 1983-2014 waves of British Social Attitudes.  The variable labels are different for each wave and ADD FILES picks them up from the first file encountered.  I don’t fancy wading through almost 11,000 variables to check/change them.  Is there a way of producing a VARIABLE LABELS list from the *.sav file?  I vaguely remember something from the 1970s called WRITE FILEINFO, but it’s not in the FM.  One alternative is to find the file with the most suitable labels and use that first when adding the other files: another would be to copy the contents of the Name and Labels columns into a syntax file.

 

I need to send the file to Natcen and UKDS.  If it won’t go via Dropbox, can anyone suggest a secure alternative?

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

John F Hall

Jon

 

Thanks for that.  Something else I’ve never used, but I’m always open to new tricks.  The main problem will be to keep the original file structures, but it will probably mean trawling for labels for variables that are not in the first file used.  At a later stage I may need to re-organise the file into topics which will be easier to find for users.   In theory all variables are now numeric and all formats are consistent, otherwise ADD FILES would not have worked.  Measurement levels and missing values all need to be checked and amended before I start.  APPLY DICTIONARY in the FM looks pretty formidable so I’ll have to be careful.  Looks like I’m in for more long, meticulous sessions.  I just wish Natcen had been more consistent when they produced the original files.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Jon Peck
Sent: 15 April 2016 19:28
To: [hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

It would be possible to extract such a list using programmability, but it would be much easier to just use APPLY DICTIONARY to copy any properties you want from one sav file to others.

 

On Fri, Apr 15, 2016 at 11:26 AM, John F Hall <[hidden email]> wrote:

I have produced a massive 1,04 gb beta version “mother” file for the 1983-2014 waves of British Social Attitudes.  The variable labels are different for each wave and ADD FILES picks them up from the first file encountered.  I don’t fancy wading through almost 11,000 variables to check/change them.  Is there a way of producing a VARIABLE LABELS list from the *.sav file?  I vaguely remember something from the 1970s called WRITE FILEINFO, but it’s not in the FM.  One alternative is to find the file with the most suitable labels and use that first when adding the other files: another would be to copy the contents of the Name and Labels columns into a syntax file.

 

I need to send the file to Natcen and UKDS.  If it won’t go via Dropbox, can anyone suggest a secure alternative?

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

Jon Peck
I know you prefer syntax, but I suggest starting out with APPLY DICTIONARY by using the menus: Data > Copy Data Properties

On Fri, Apr 15, 2016 at 12:11 PM, John F Hall <[hidden email]> wrote:

Jon

 

Thanks for that.  Something else I’ve never used, but I’m always open to new tricks.  The main problem will be to keep the original file structures, but it will probably mean trawling for labels for variables that are not in the first file used.  At a later stage I may need to re-organise the file into topics which will be easier to find for users.   In theory all variables are now numeric and all formats are consistent, otherwise ADD FILES would not have worked.  Measurement levels and missing values all need to be checked and amended before I start.  APPLY DICTIONARY in the FM looks pretty formidable so I’ll have to be careful.  Looks like I’m in for more long, meticulous sessions.  I just wish Natcen had been more consistent when they produced the original files.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Jon Peck
Sent: 15 April 2016 19:28
To: [hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

It would be possible to extract such a list using programmability, but it would be much easier to just use APPLY DICTIONARY to copy any properties you want from one sav file to others.

 

On Fri, Apr 15, 2016 at 11:26 AM, John F Hall <[hidden email]> wrote:

I have produced a massive 1,04 gb beta version “mother” file for the 1983-2014 waves of British Social Attitudes.  The variable labels are different for each wave and ADD FILES picks them up from the first file encountered.  I don’t fancy wading through almost 11,000 variables to check/change them.  Is there a way of producing a VARIABLE LABELS list from the *.sav file?  I vaguely remember something from the 1970s called WRITE FILEINFO, but it’s not in the FM.  One alternative is to find the file with the most suitable labels and use that first when adding the other files: another would be to copy the contents of the Name and Labels columns into a syntax file.

 

I need to send the file to Natcen and UKDS.  If it won’t go via Dropbox, can anyone suggest a secure alternative?

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

Albert-Jan Roskam-3
In reply to this post by John F Hall
"At a later stage I may need to re-organise the file into topics which will be easier to find for users." --> you might want to have a look at Variable Sets. These allow you to group vars, which is is handy in the GUI. Afaik, you can not only define sets via syntax.


Date: Fri, 15 Apr 2016 20:11:54 +0200
From: [hidden email]
Subject: Re: [SPSSX-L] Reverse engineering from *.sav to *.sps
To: [hidden email]

Jon

 

Thanks for that.  Something else I’ve never used, but I’m always open to new tricks.  The main problem will be to keep the original file structures, but it will probably mean trawling for labels for variables that are not in the first file used.  At a later stage I may need to re-organise the file into topics which will be easier to find for users.   In theory all variables are now numeric and all formats are consistent, otherwise ADD FILES would not have worked.  Measurement levels and missing values all need to be checked and amended before I start.  APPLY DICTIONARY in the FM looks pretty formidable so I’ll have to be careful.  Looks like I’m in for more long, meticulous sessions.  I just wish Natcen had been more consistent when they produced the original files.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   johnfhall@... 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Jon Peck
Sent: 15 April 2016 19:28
To: [hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

It would be possible to extract such a list using programmability, but it would be much easier to just use APPLY DICTIONARY to copy any properties you want from one sav file to others.

 

On Fri, Apr 15, 2016 at 11:26 AM, John F Hall <johnfhall@...> wrote:

I have produced a massive 1,04 gb beta version “mother” file for the 1983-2014 waves of British Social Attitudes.  The variable labels are different for each wave and ADD FILES picks them up from the first file encountered.  I don’t fancy wading through almost 11,000 variables to check/change them.  Is there a way of producing a VARIABLE LABELS list from the *.sav file?  I vaguely remember something from the 1970s called WRITE FILEINFO, but it’s not in the FM.  One alternative is to find the file with the most suitable labels and use that first when adding the other files: another would be to copy the contents of the Name and Labels columns into a syntax file.

 

I need to send the file to Natcen and UKDS.  If it won’t go via Dropbox, can anyone suggest a secure alternative?

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   johnfhall@... 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to LISTSERV@... (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
jkpeck@...

===================== To manage your subscription to SPSSX-L, send a message to LISTSERV@... (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to LISTSERV@... (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

Jon Peck
It is true that sets can only be defined in the gui, but if you have created such definitions, the SETSMACRO extension command can produce a macro that lists the variables in such sets.

On Sat, Apr 16, 2016 at 2:37 AM, Albert-Jan Roskam <[hidden email]> wrote:
"At a later stage I may need to re-organise the file into topics which will be easier to find for users." --> you might want to have a look at Variable Sets. These allow you to group vars, which is is handy in the GUI. Afaik, you can not only define sets via syntax.


Date: Fri, 15 Apr 2016 20:11:54 +0200
From: [hidden email]
Subject: Re: [SPSSX-L] Reverse engineering from *.sav to *.sps
To: [hidden email]

Jon

 

Thanks for that.  Something else I’ve never used, but I’m always open to new tricks.  The main problem will be to keep the original file structures, but it will probably mean trawling for labels for variables that are not in the first file used.  At a later stage I may need to re-organise the file into topics which will be easier to find for users.   In theory all variables are now numeric and all formats are consistent, otherwise ADD FILES would not have worked.  Measurement levels and missing values all need to be checked and amended before I start.  APPLY DICTIONARY in the FM looks pretty formidable so I’ll have to be careful.  Looks like I’m in for more long, meticulous sessions.  I just wish Natcen had been more consistent when they produced the original files.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Jon Peck
Sent: 15 April 2016 19:28
To: [hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

It would be possible to extract such a list using programmability, but it would be much easier to just use APPLY DICTIONARY to copy any properties you want from one sav file to others.

 

On Fri, Apr 15, 2016 at 11:26 AM, John F Hall <[hidden email]> wrote:

I have produced a massive 1,04 gb beta version “mother” file for the 1983-2014 waves of British Social Attitudes.  The variable labels are different for each wave and ADD FILES picks them up from the first file encountered.  I don’t fancy wading through almost 11,000 variables to check/change them.  Is there a way of producing a VARIABLE LABELS list from the *.sav file?  I vaguely remember something from the 1970s called WRITE FILEINFO, but it’s not in the FM.  One alternative is to find the file with the most suitable labels and use that first when adding the other files: another would be to copy the contents of the Name and Labels columns into a syntax file.

 

I need to send the file to Natcen and UKDS.  If it won’t go via Dropbox, can anyone suggest a secure alternative?

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

John F Hall

You’re both ahead of me at the moment.  I will experiment, but first I have to deal with variable labels which come from different waves depending when they were last used.  As you can see from the table below, SSerial and rentbuy are “clean” because they come from the 2014 wave, but ver  Chfnrl10 unempjob and ChCandte have question numbers at the end which only apply to the year they were first used. 

 

Variable

Position

Label

Level

Width

Format

Missing

SSerial

1

External Serial Number

Scale

12

F12

Lowest through -1

ver

525

Version Number: [S-C]ABC

Nominal

5

F2

Lowest through -1

Chfnrl10

1232

If think it appropriate for children under 12 to attend funerals :Q723

Nominal

10

F2

-9 through -1

rentbuy

1431

Should newly-married young couple, both with steady jobs, buy or rent a home.

Ordinal

9

F2

-9 through -1

unempjob

620

Around here, most unemployed people could find a job if they really wanted one: [S-C]ABC

Ordinal

10

F2

Lowest through -1

ChCandte

1832

Who should have a say in deciding who stands as a party's candidate for MP B2.23C2.22.

Ordinal

7

F2

-9 through -1

 

Hopefully this will only happen in blocks of contingent variables and can be dealt with using <varx> TO <vary> inside APPLY DICTIONARY commands.  The same applies to measurement levels and missing values: values such as 7, 97, and 97, 8, 98 and 998 or 9, 99 and 999 are labelled missing, but need changing to negative to be caught by LO thru -1.  AFAIK widths and formats are consistent throughout.  I need to run some tables to see which year each variable name was first used. 

 

year Year of survey * ver Version Number: [S-C]ABC Crosstabulation

Count 

 

ver Version Number: [S-C]ABC

Total

1 Version A

2 Version B

3 Version C

year Year of survey

2014

799

771

780

2350

Total

799

771

780

2350

 

If necessary I can get rid of [S-C]ABC using Ctrl+H in the Labels column, but since this is used to indicate self-completion questionnaire and there are no actual question numbers it may be useful to leave them in, perhaps just as [S-C] .  Labels with actual question numbers will need the JKP Python treatment, possibly with numbers being replaced by year of first use.  It will take forever, but these can be found via CROSSTABS:

 

 

year Year of interview * SchQual Have you passed any of the examinations on this card?: Q1197 Crosstabulation

Count 

 

SchQual Have you passed any of the examinations on this card?: Q1197

Total

-1

1 Yes

2 No

8 Don't know

9 Refusal

year Year of interview

1990

0

1387

1407

0

3

2797

1991

0

1453

1460

0

5

2918

1993

0

1444

1491

2

8

2945

1994

0

1797

1658

9

5

3469

1995

0

1933

1696

2

2

3633

1996

0

2032

1601

4

25

3662

1997

0

727

618

8

2

1355

1998

0

1786

1317

10

33

3146

1999

0

1766

1343

4

30

3143

2000

0

1989

1423

1

13

3426

2001

0

2022

1239

3

23

3287

2002

0

2118

1296

12

9

3435

2003

0

2740

1664

7

21

4432

2004

0

2086

1105

4

4

3199

2005

0

2695

1552

14

7

4268

2006

0

2803

1467

4

16

4290

2007

0

2680

1430

4

10

4124

2008

0

2988

1477

8

13

4486

2009

0

2381

1029

4

7

3421

2010

0

2162

1113

7

15

3297

2011

0

2241

1045

8

17

3311

2012

0

2154

1082

6

6

3248

2013

0

2269

964

6

5

3244

2014

1

1969

897

7

4

2878

Total

1

49622

31374

134

283

81414

 

. . but it would have helped if there were already a table in the documentation.  As you can see, values -1, 8 and 9 are not treated as missing.

 

Variable labels are inconsistent in how question numbers are appended (with or without question marks and/or spaces):

 

Stolen frm car next year?A240aB234aC249a

Censorship necessary/unnecessary[2]B235h

Rich people get too big sharewealthC250a

Employees need TU protection?      C250b

 

Looks like the vegetable garden will have to wait a while.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Jon Peck
Sent: 16 April 2016 15:49
To: [hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

It is true that sets can only be defined in the gui, but if you have created such definitions, the SETSMACRO extension command can produce a macro that lists the variables in such sets.

 

On Sat, Apr 16, 2016 at 2:37 AM, Albert-Jan Roskam <[hidden email]> wrote:

"At a later stage I may need to re-organise the file into topics which will be easier to find for users." --> you might want to have a look at Variable Sets. These allow you to group vars, which is is handy in the GUI. Afaik, you can not only define sets via syntax.


Date: Fri, 15 Apr 2016 20:11:54 +0200
From: [hidden email]
Subject: Re: [SPSSX-L] Reverse engineering from *.sav to *.sps
To: [hidden email]

Jon

 

Thanks for that.  Something else I’ve never used, but I’m always open to new tricks.  The main problem will be to keep the original file structures, but it will probably mean trawling for labels for variables that are not in the first file used.  At a later stage I may need to re-organise the file into topics which will be easier to find for users.   In theory all variables are now numeric and all formats are consistent, otherwise ADD FILES would not have worked.  Measurement levels and missing values all need to be checked and amended before I start.  APPLY DICTIONARY in the FM looks pretty formidable so I’ll have to be careful.  Looks like I’m in for more long, meticulous sessions.  I just wish Natcen had been more consistent when they produced the original files.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Jon Peck
Sent: 15 April 2016 19:28
To: [hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

It would be possible to extract such a list using programmability, but it would be much easier to just use APPLY DICTIONARY to copy any properties you want from one sav file to others.

 

On Fri, Apr 15, 2016 at 11:26 AM, John F Hall <[hidden email]> wrote:

I have produced a massive 1,04 gb beta version “mother” file for the 1983-2014 waves of British Social Attitudes.  The variable labels are different for each wave and ADD FILES picks them up from the first file encountered.  I don’t fancy wading through almost 11,000 variables to check/change them.  Is there a way of producing a VARIABLE LABELS list from the *.sav file?  I vaguely remember something from the 1970s called WRITE FILEINFO, but it’s not in the FM.  One alternative is to find the file with the most suitable labels and use that first when adding the other files: another would be to copy the contents of the Name and Labels columns into a syntax file.

 

I need to send the file to Natcen and UKDS.  If it won’t go via Dropbox, can anyone suggest a secure alternative?

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

John F Hall
In reply to this post by Jon Peck

Here’s an example of why detective work is needed:

 

year Year of interview * pressrun BQ217 WELL RUN: THE PRESS Crosstabulation

Count 

 

pressrun BQ217 WELL RUN: THE PRESS

Total

1 WELL RUN

2 NOT WELL RUN

3

4

8 DK

9 NA

year Year of interview

1986

632

647

0

0

13

29

1321

1987

25

449

507

185

5

41

1212

Total

657

1096

507

185

18

70

2533

 

Checking on values 3 and 4 for 1987, I found:

 

 

Frequency

Valid

1 VERY    WELL RUN

22

2 WELL RUN

441

3 NOT VRY WELL RUN

495

4 NOT AT ALL WELL RUN

179

8 DK

5

9 NA

40

Total

1181

Missing

0 NO SELF-COMPLETN

194

System

1391

Total

1585

Total

2766

 

Any more like these and gardening begins to be very attractive.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 16 April 2016 18:04
To: '[hidden email]' <[hidden email]>
Subject: RE: Reverse engineering from *.sav to *.sps

 

You’re both ahead of me at the moment.  I will experiment, but first I have to deal with variable labels which come from different waves depending when they were last used.  As you can see from the table below, SSerial and rentbuy are “clean” because they come from the 2014 wave, but ver  Chfnrl10 unempjob and ChCandte have question numbers at the end which only apply to the year they were first used. 

 

Variable

Position

Label

Level

Width

Format

Missing

SSerial

1

External Serial Number

Scale

12

F12

Lowest through -1

ver

525

Version Number: [S-C]ABC

Nominal

5

F2

Lowest through -1

Chfnrl10

1232

If think it appropriate for children under 12 to attend funerals :Q723

Nominal

10

F2

-9 through -1

rentbuy

1431

Should newly-married young couple, both with steady jobs, buy or rent a home.

Ordinal

9

F2

-9 through -1

unempjob

620

Around here, most unemployed people could find a job if they really wanted one: [S-C]ABC

Ordinal

10

F2

Lowest through -1

ChCandte

1832

Who should have a say in deciding who stands as a party's candidate for MP B2.23C2.22.

Ordinal

7

F2

-9 through -1

 

Hopefully this will only happen in blocks of contingent variables and can be dealt with using <varx> TO <vary> inside APPLY DICTIONARY commands.  The same applies to measurement levels and missing values: values such as 7, 97, and 97, 8, 98 and 998 or 9, 99 and 999 are labelled missing, but need changing to negative to be caught by LO thru -1.  AFAIK widths and formats are consistent throughout.  I need to run some tables to see which year each variable name was first used. 

 

year Year of survey * ver Version Number: [S-C]ABC Crosstabulation

Count 

 

ver Version Number: [S-C]ABC

Total

1 Version A

2 Version B

3 Version C

year Year of survey

2014

799

771

780

2350

Total

799

771

780

2350

 

If necessary I can get rid of [S-C]ABC using Ctrl+H in the Labels column, but since this is used to indicate self-completion questionnaire and there are no actual question numbers it may be useful to leave them in, perhaps just as [S-C] .  Labels with actual question numbers will need the JKP Python treatment, possibly with numbers being replaced by year of first use.  It will take forever, but these can be found via CROSSTABS:

 

 

year Year of interview * SchQual Have you passed any of the examinations on this card?: Q1197 Crosstabulation

Count 

 

SchQual Have you passed any of the examinations on this card?: Q1197

Total

-1

1 Yes

2 No

8 Don't know

9 Refusal

year Year of interview

1990

0

1387

1407

0

3

2797

1991

0

1453

1460

0

5

2918

1993

0

1444

1491

2

8

2945

1994

0

1797

1658

9

5

3469

1995

0

1933

1696

2

2

3633

1996

0

2032

1601

4

25

3662

1997

0

727

618

8

2

1355

1998

0

1786

1317

10

33

3146

1999

0

1766

1343

4

30

3143

2000

0

1989

1423

1

13

3426

2001

0

2022

1239

3

23

3287

2002

0

2118

1296

12

9

3435

2003

0

2740

1664

7

21

4432

2004

0

2086

1105

4

4

3199

2005

0

2695

1552

14

7

4268

2006

0

2803

1467

4

16

4290

2007

0

2680

1430

4

10

4124

2008

0

2988

1477

8

13

4486

2009

0

2381

1029

4

7

3421

2010

0

2162

1113

7

15

3297

2011

0

2241

1045

8

17

3311

2012

0

2154

1082

6

6

3248

2013

0

2269

964

6

5

3244

2014

1

1969

897

7

4

2878

Total

1

49622

31374

134

283

81414

 

. . but it would have helped if there were already a table in the documentation.  As you can see, values -1, 8 and 9 are not treated as missing.

 

Variable labels are inconsistent in how question numbers are appended (with or without question marks and/or spaces):

 

Stolen frm car next year?A240aB234aC249a

Censorship necessary/unnecessary[2]B235h

Rich people get too big sharewealthC250a

Employees need TU protection?      C250b

 

Looks like the vegetable garden will have to wait a while.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

 

From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Jon Peck
Sent: 16 April 2016 15:49
To: [hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

It is true that sets can only be defined in the gui, but if you have created such definitions, the SETSMACRO extension command can produce a macro that lists the variables in such sets.

 

On Sat, Apr 16, 2016 at 2:37 AM, Albert-Jan Roskam <[hidden email]> wrote:

"At a later stage I may need to re-organise the file into topics which will be easier to find for users." --> you might want to have a look at Variable Sets. These allow you to group vars, which is is handy in the GUI. Afaik, you can not only define sets via syntax.


Date: Fri, 15 Apr 2016 20:11:54 +0200
From: [hidden email]
Subject: Re: [SPSSX-L] Reverse engineering from *.sav to *.sps
To: [hidden email]

Jon

 

Thanks for that.  Something else I’ve never used, but I’m always open to new tricks.  The main problem will be to keep the original file structures, but it will probably mean trawling for labels for variables that are not in the first file used.  At a later stage I may need to re-organise the file into topics which will be easier to find for users.   In theory all variables are now numeric and all formats are consistent, otherwise ADD FILES would not have worked.  Measurement levels and missing values all need to be checked and amended before I start.  APPLY DICTIONARY in the FM looks pretty formidable so I’ll have to be careful.  Looks like I’m in for more long, meticulous sessions.  I just wish Natcen had been more consistent when they produced the original files.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Jon Peck
Sent: 15 April 2016 19:28
To: [hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

It would be possible to extract such a list using programmability, but it would be much easier to just use APPLY DICTIONARY to copy any properties you want from one sav file to others.

 

On Fri, Apr 15, 2016 at 11:26 AM, John F Hall <[hidden email]> wrote:

I have produced a massive 1,04 gb beta version “mother” file for the 1983-2014 waves of British Social Attitudes.  The variable labels are different for each wave and ADD FILES picks them up from the first file encountered.  I don’t fancy wading through almost 11,000 variables to check/change them.  Is there a way of producing a VARIABLE LABELS list from the *.sav file?  I vaguely remember something from the 1970s called WRITE FILEINFO, but it’s not in the FM.  One alternative is to find the file with the most suitable labels and use that first when adding the other files: another would be to copy the contents of the Name and Labels columns into a syntax file.

 

I need to send the file to Natcen and UKDS.  If it won’t go via Dropbox, can anyone suggest a secure alternative?

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

John F Hall
In reply to this post by John F Hall

I have uploaded two new commentaries to my page Exploring British Social Attitudes http://surveyresearch.weebly.com/exploring-british-social-attitudes.html

 

British Social Attitudes 1983 to 2014: Resolution of type conflicts   

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983_-_2014_resolution_of_type_conflicts.pdf  (11 pp)

and British Social Attitudes 1983-2014:  Beta version of cumulative file

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983-2014__beta_version_of_cumulative_file.pdf  (3pp)  contain sample tables of inconsistencies and incompatibilities and an indication of the process of file-building, including examples of SPSS commands used to resolve them.

 

Serious problems with measurement levels and missing values remain to be resolved.  Incorrect measurement levels may be due to SPSS automatically applying “heuristic” levels to variables depending on the number of values encountered, but they could equally be due to automatic archiving software used at UKDS.  See: British Social Attitudes 2009 - 2014 Measurement levels

Missing values are declared for all variables as negative, almost all in the range Lo thru -1, but the original positive values to be treated as missing (8, 98, 998 and 9, 99, 999) have not been changed and consequently are not treated as missing for analysis purposes.  See: British Social Attitudes 2009 to 2014 _ Missing values.  Checking and correction of these will be routine, but extremely tedious, and, unless I can use some existing syntax from my earlier work or find the “cleanest” individual wave files and copy metadata across using the SPSS command APPLY DICTIONARY, will take some time yet.

 

At 1.04 gb the current file is very large: if it won’t go via Dropbox, can anyone suggest an alternative method of sending it to Natcen (the owners) and UKDS (the distributors)?  If not, I can burn it (encrypted) to DVD and use snail-mail.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

FW: Reverse engineering from *.sav to *.sps

John F Hall
In reply to this post by John F Hall

Apologies for duplicate posting: the original had hyperlinks which the server may reject.  I have replaced them with URLs.

 

I have uploaded two new commentaries to my page Exploring British Social Attitudes http://surveyresearch.weebly.com/exploring-british-social-attitudes.html

 

British Social Attitudes 1983 to 2014: Resolution of type conflicts   

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983_-_2014_resolution_of_type_conflicts.pdf  (11 pp)

and British Social Attitudes 1983-2014:  Beta version of cumulative file

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983-2014__beta_version_of_cumulative_file.pdf  (3pp)  contain sample tables of inconsistencies and incompatibilities and an indication of the process of file-building, including examples of SPSS commands used to resolve them.

 

Serious problems with measurement levels and missing values remain to be resolved.  Incorrect measurement levels may be due to SPSS automatically applying “heuristic” levels to variables depending on the number of values encountered, but they could equally be due to automatic archiving software used at UKDS.  See: British Social Attitudes 2009 - 2014 Measurement levels
http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014_measurement_levels.pdf

 

Missing values are declared for all variables as negative, almost all in the range Lo thru -1, but the original positive values to be treated as missing (8, 98, 998 and 9, 99, 999) have not been changed and consequently are not treated as missing for analysis purposes.  See: British Social Attitudes 2009 to 2014 _ Missing values http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014___missing_values.pdf . 

 

Checking and correction of these will be routine, but extremely tedious, and, unless I can use some existing syntax from my earlier work or find the “cleanest” individual wave files and copy metadata across using the SPSS command APPLY DICTIONARY, will take some time yet.

 

At 1.04 gb the current file is very large: if it won’t go via Dropbox, can anyone suggest an alternative method of sending it to Natcen (the owners) and UKDS (the distributors)?  If not, I can burn it (encrypted) to DVD and use snail-mail.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

John F Hall
In reply to this post by John F Hall

Dropbox has accepted the large *.sav file and created a link for Natcen and UKDS to download it.  Access will be strictly via UKDS and after approval by Natcen.

 

I now have another question.  I can use Ctrl+F to find value labels such as “Strongly

Disagree” in the Values column.  Is it possible to identify all such labels in a single search and produce a list of the variables containing that phrase?  If so this would immensely speed up the correct specification of measurement levels as Ordinal and missing values  8 = Don’t know and 9 = Refused changed to -8 and -9 to be captured by the universal Lo thru -1 in the Missing column.

 

Variables with two digit values will be more difficult to identify unless they have labels such as “Completely satisfied”: some of these are measured on 0-10 scales, others on 1-5.  All two-digit variables have 98 and 99 as missing: they can be changed to -98 and -99.  Some variables have three digits with 998 and 999 as missing, but I can find these myself.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 18 April 2016 17:23
To: '[hidden email]' <[hidden email]>
Cc: '[hidden email]' <[hidden email]>
Subject: FW: Reverse engineering from *.sav to *.sps

 

Apologies for duplicate posting: the original had hyperlinks which the server may reject.  I have replaced them with URLs.

 

I have uploaded two new commentaries to my page Exploring British Social Attitudes http://surveyresearch.weebly.com/exploring-british-social-attitudes.html

 

British Social Attitudes 1983 to 2014: Resolution of type conflicts   

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983_-_2014_resolution_of_type_conflicts.pdf  (11 pp)

and British Social Attitudes 1983-2014:  Beta version of cumulative file

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983-2014__beta_version_of_cumulative_file.pdf  (3pp)  contain sample tables of inconsistencies and incompatibilities and an indication of the process of file-building, including examples of SPSS commands used to resolve them.

 

Serious problems with measurement levels and missing values remain to be resolved.  Incorrect measurement levels may be due to SPSS automatically applying “heuristic” levels to variables depending on the number of values encountered, but they could equally be due to automatic archiving software used at UKDS.  See: British Social Attitudes 2009 - 2014 Measurement levels
http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014_measurement_levels.pdf

 

Missing values are declared for all variables as negative, almost all in the range Lo thru -1, but the original positive values to be treated as missing (8, 98, 998 and 9, 99, 999) have not been changed and consequently are not treated as missing for analysis purposes.  See: British Social Attitudes 2009 to 2014 _ Missing values http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014___missing_values.pdf . 

 

Checking and correction of these will be routine, but extremely tedious, and, unless I can use some existing syntax from my earlier work or find the “cleanest” individual wave files and copy metadata across using the SPSS command APPLY DICTIONARY, will take some time yet.

 

At 1.04 gb the current file is very large: if it won’t go via Dropbox, can anyone suggest an alternative method of sending it to Natcen (the owners) and UKDS (the distributors)?  If not, I can burn it (encrypted) to DVD and use snail-mail.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

Jon Peck
For starters, you could run this code, which will give you a dataset containing an alphabetical list of all the value labels in the dataset along with other properties.  You could then select out the words you want to examine.

dataset declare values.
oms select tables /if subtypes='Variable Values'
/destination outfile=values format=sav viewer=no.
display dict.
omsend.

dataset activate values.
sort cases by Label.


On Mon, Apr 18, 2016 at 11:16 AM, John F Hall <[hidden email]> wrote:

Dropbox has accepted the large *.sav file and created a link for Natcen and UKDS to download it.  Access will be strictly via UKDS and after approval by Natcen.

 

I now have another question.  I can use Ctrl+F to find value labels such as “Strongly

Disagree” in the Values column.  Is it possible to identify all such labels in a single search and produce a list of the variables containing that phrase?  If so this would immensely speed up the correct specification of measurement levels as Ordinal and missing values  8 = Don’t know and 9 = Refused changed to -8 and -9 to be captured by the universal Lo thru -1 in the Missing column.

 

Variables with two digit values will be more difficult to identify unless they have labels such as “Completely satisfied”: some of these are measured on 0-10 scales, others on 1-5.  All two-digit variables have 98 and 99 as missing: they can be changed to -98 and -99.  Some variables have three digits with 998 and 999 as missing, but I can find these myself.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 18 April 2016 17:23
To: '[hidden email]' <[hidden email]>
Cc: '[hidden email]' <[hidden email]>
Subject: FW: Reverse engineering from *.sav to *.sps

 

Apologies for duplicate posting: the original had hyperlinks which the server may reject.  I have replaced them with URLs.

 

I have uploaded two new commentaries to my page Exploring British Social Attitudes http://surveyresearch.weebly.com/exploring-british-social-attitudes.html

 

British Social Attitudes 1983 to 2014: Resolution of type conflicts   

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983_-_2014_resolution_of_type_conflicts.pdf  (11 pp)

and British Social Attitudes 1983-2014:  Beta version of cumulative file

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983-2014__beta_version_of_cumulative_file.pdf  (3pp)  contain sample tables of inconsistencies and incompatibilities and an indication of the process of file-building, including examples of SPSS commands used to resolve them.

 

Serious problems with measurement levels and missing values remain to be resolved.  Incorrect measurement levels may be due to SPSS automatically applying “heuristic” levels to variables depending on the number of values encountered, but they could equally be due to automatic archiving software used at UKDS.  See: British Social Attitudes 2009 - 2014 Measurement levels
http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014_measurement_levels.pdf

 

Missing values are declared for all variables as negative, almost all in the range Lo thru -1, but the original positive values to be treated as missing (8, 98, 998 and 9, 99, 999) have not been changed and consequently are not treated as missing for analysis purposes.  See: British Social Attitudes 2009 to 2014 _ Missing values http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014___missing_values.pdf . 

 

Checking and correction of these will be routine, but extremely tedious, and, unless I can use some existing syntax from my earlier work or find the “cleanest” individual wave files and copy metadata across using the SPSS command APPLY DICTIONARY, will take some time yet.

 

At 1.04 gb the current file is very large: if it won’t go via Dropbox, can anyone suggest an alternative method of sending it to Natcen (the owners) and UKDS (the distributors)?  If not, I can burn it (encrypted) to DVD and use snail-mail.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

John F Hall

Jon

 

Not sure what exactly I will get, so will run your syntax on a copy.  This means the cases will remain in their original order in the source file. 

 

There could well be several thousand varnames thrown up, so if I change my SPSS settings to names only, I should be able to pick up the first and last varnames from the first column of the table and use:

 

var lev <first> to <last> (ord).

recode <first> to <last> (8 = -8)(9 = -9).

missing values <first> to <last> (lo thru -1).

 

Just used Ctrl+F for “strongly disagree” in the Values column and the first block of variables found had values ranging from 1 = “1 Strongly disagree” to 10 = “10 Strongly agree” [sic: must have been written by a complete beginner] with -1 = “Not applicable”, 98 “Don’t know”and 99 “Refused”.   However “10 Strongly agree” could be used to identify items using 1-10 scales.  Lo thru -1 is already declared as missing for most variables and will pick up the -1.  98 and 99 will have to be recoded and given new labels.

 

This creates another problem since there are also other Agree – Disagree items coded in the opposite direction!  One set has five valid values (1 = “Agree strongly” 2 = “Agree” 3 = “Neither agree nor disagree” 4 = “Disagree” 5 = “Disagree strongly”)   Another has only four (1 = “Strongly agree” 2 = “Tend to agree” 3 = “Tend to disagree” 4 = “Strongly disagree”).  Again the unique labels “Neither agree nor disagree” could be used to find the 1-5 items and “Tend to disagree” to find the 1-4 items.

 

Just ran your syntax.  Judging by the new file I think I can identify sets of variables with the same properties from the Var2 column and copy blocks of names from Var1 into a new syntax file.   In fact there are only three variables using the 1 -10 agree - disagree scale, but all the others are listed in blocks so I can use <first> to <last> syntax to set levels and missing values. 

 

Thanks a million: you just saved me days if not weeks.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Jon Peck
Sent: 18 April 2016 19:36
To: [hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

For starters, you could run this code, which will give you a dataset containing an alphabetical list of all the value labels in the dataset along with other properties.  You could then select out the words you want to examine.

 

dataset declare values.

oms select tables /if subtypes='Variable Values'

/destination outfile=values format=sav viewer=no.

display dict.

omsend.

 

dataset activate values.

sort cases by Label.

 

 

On Mon, Apr 18, 2016 at 11:16 AM, John F Hall <[hidden email]> wrote:

Dropbox has accepted the large *.sav file and created a link for Natcen and UKDS to download it.  Access will be strictly via UKDS and after approval by Natcen.

 

I now have another question.  I can use Ctrl+F to find value labels such as “Strongly

Disagree” in the Values column.  Is it possible to identify all such labels in a single search and produce a list of the variables containing that phrase?  If so this would immensely speed up the correct specification of measurement levels as Ordinal and missing values  8 = Don’t know and 9 = Refused changed to -8 and -9 to be captured by the universal Lo thru -1 in the Missing column.

 

Variables with two digit values will be more difficult to identify unless they have labels such as “Completely satisfied”: some of these are measured on 0-10 scales, others on 1-5.  All two-digit variables have 98 and 99 as missing: they can be changed to -98 and -99.  Some variables have three digits with 998 and 999 as missing, but I can find these myself.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 18 April 2016 17:23
To: '[hidden email]' <[hidden email]>
Cc: '[hidden email]' <[hidden email]>
Subject: FW: Reverse engineering from *.sav to *.sps

 

Apologies for duplicate posting: the original had hyperlinks which the server may reject.  I have replaced them with URLs.

 

I have uploaded two new commentaries to my page Exploring British Social Attitudes http://surveyresearch.weebly.com/exploring-british-social-attitudes.html

 

British Social Attitudes 1983 to 2014: Resolution of type conflicts   

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983_-_2014_resolution_of_type_conflicts.pdf  (11 pp)

and British Social Attitudes 1983-2014:  Beta version of cumulative file

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983-2014__beta_version_of_cumulative_file.pdf  (3pp)  contain sample tables of inconsistencies and incompatibilities and an indication of the process of file-building, including examples of SPSS commands used to resolve them.

 

Serious problems with measurement levels and missing values remain to be resolved.  Incorrect measurement levels may be due to SPSS automatically applying “heuristic” levels to variables depending on the number of values encountered, but they could equally be due to automatic archiving software used at UKDS.  See: British Social Attitudes 2009 - 2014 Measurement levels
http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014_measurement_levels.pdf

 

Missing values are declared for all variables as negative, almost all in the range Lo thru -1, but the original positive values to be treated as missing (8, 98, 998 and 9, 99, 999) have not been changed and consequently are not treated as missing for analysis purposes.  See: British Social Attitudes 2009 to 2014 _ Missing values http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014___missing_values.pdf

 

Checking and correction of these will be routine, but extremely tedious, and, unless I can use some existing syntax from my earlier work or find the “cleanest” individual wave files and copy metadata across using the SPSS command APPLY DICTIONARY, will take some time yet.

 

At 1.04 gb the current file is very large: if it won’t go via Dropbox, can anyone suggest an alternative method of sending it to Natcen (the owners) and UKDS (the distributors)?  If not, I can burn it (encrypted) to DVD and use snail-mail.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

John F Hall
In reply to this post by Jon Peck

I have a version of the  BSA cumulative *.sav file created with:

 

SORT VARIABLES by values (d).

 

. . and have been working through looking at value labels beginning with 8, 98, 9, 99 which should be missing, but which are not. 

 

{99, Refusal}...

{99, Not answered}...

{98, Don't   know}...

{8, Don't   know}...

 

Any positive such codes can then be recoded to -8. -9, -98, -99  etc to be caught by (Lo thru -1) .

 

However I have a number of variables which contain 98, 99 etc but which do not display 98 or 99 in the Values column.  For example:

 

VoteEU          {1, Yes: voted}

 

... but the frequency table shows:

 

VotedEU Did Respondent vote in european election 2004? Q1027

 

 

Frequency

Percent

Valid Percent

Cumulative Percent

 

Valid

-2

2083

2.2

19.6

19.6

 

1 Yes: voted

3358

3.5

31.6

51.3

 

2 No

4992

5.2

47.0

98.3

 

97 (Refused to say)

26

.0

.2

98.6

 

98 Don't   know

151

.2

1.4

100.0

 

Total

10610

11.1

100.0

 

 

Missing

System

85020

88.9

 

 

 

 

Total

95630

100.0

 

 

 

I expected SORT VARIABLES by values (d). to display:

 

VoteEU           {98 Don't   know }

 

Is there another way I can force the Values column in the Data Editor to display the highest positive value? 

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 19 April 2016 07:55
To: '[hidden email]' <[hidden email]>
Subject: RE: Reverse engineering from *.sav to *.sps

 

Jon

 

Not sure what exactly I will get, so will run your syntax on a copy.  This means the cases will remain in their original order in the source file. 

 

There could well be several thousand varnames thrown up, so if I change my SPSS settings to names only, I should be able to pick up the first and last varnames from the first column of the table and use:

 

var lev <first> to <last> (ord).

recode <first> to <last> (8 = -8)(9 = -9).

missing values <first> to <last> (lo thru -1).

 

Just used Ctrl+F for “strongly disagree” in the Values column and the first block of variables found had values ranging from 1 = “1 Strongly disagree” to 10 = “10 Strongly agree” [sic: must have been written by a complete beginner] with -1 = “Not applicable”, 98 “Don’t know”and 99 “Refused”.   However “10 Strongly agree” could be used to identify items using 1-10 scales.  Lo thru -1 is already declared as missing for most variables and will pick up the -1.  98 and 99 will have to be recoded and given new labels.

 

This creates another problem since there are also other Agree – Disagree items coded in the opposite direction!  One set has five valid values (1 = “Agree strongly” 2 = “Agree” 3 = “Neither agree nor disagree” 4 = “Disagree” 5 = “Disagree strongly”)   Another has only four (1 = “Strongly agree” 2 = “Tend to agree” 3 = “Tend to disagree” 4 = “Strongly disagree”).  Again the unique labels “Neither agree nor disagree” could be used to find the 1-5 items and “Tend to disagree” to find the 1-4 items.

 

Just ran your syntax.  Judging by the new file I think I can identify sets of variables with the same properties from the Var2 column and copy blocks of names from Var1 into a new syntax file.   In fact there are only three variables using the 1 -10 agree - disagree scale, but all the others are listed in blocks so I can use <first> to <last> syntax to set levels and missing values. 

 

Thanks a million: you just saved me days if not weeks.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Jon Peck
Sent: 18 April 2016 19:36
To:
[hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

For starters, you could run this code, which will give you a dataset containing an alphabetical list of all the value labels in the dataset along with other properties.  You could then select out the words you want to examine.

 

dataset declare values.

oms select tables /if subtypes='Variable Values'

/destination outfile=values format=sav viewer=no.

display dict.

omsend.

 

dataset activate values.

sort cases by Label.

 

 

On Mon, Apr 18, 2016 at 11:16 AM, John F Hall <[hidden email]> wrote:

Dropbox has accepted the large *.sav file and created a link for Natcen and UKDS to download it.  Access will be strictly via UKDS and after approval by Natcen.

 

I now have another question.  I can use Ctrl+F to find value labels such as “Strongly

Disagree” in the Values column.  Is it possible to identify all such labels in a single search and produce a list of the variables containing that phrase?  If so this would immensely speed up the correct specification of measurement levels as Ordinal and missing values  8 = Don’t know and 9 = Refused changed to -8 and -9 to be captured by the universal Lo thru -1 in the Missing column.

 

Variables with two digit values will be more difficult to identify unless they have labels such as “Completely satisfied”: some of these are measured on 0-10 scales, others on 1-5.  All two-digit variables have 98 and 99 as missing: they can be changed to -98 and -99.  Some variables have three digits with 998 and 999 as missing, but I can find these myself.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 18 April 2016 17:23
To: '
[hidden email]' <[hidden email]>
Cc: '
[hidden email]' <[hidden email]>
Subject: FW: Reverse engineering from *.sav to *.sps

 

Apologies for duplicate posting: the original had hyperlinks which the server may reject.  I have replaced them with URLs.

 

I have uploaded two new commentaries to my page Exploring British Social Attitudes http://surveyresearch.weebly.com/exploring-british-social-attitudes.html

 

British Social Attitudes 1983 to 2014: Resolution of type conflicts   

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983_-_2014_resolution_of_type_conflicts.pdf  (11 pp)

and British Social Attitudes 1983-2014:  Beta version of cumulative file

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983-2014__beta_version_of_cumulative_file.pdf  (3pp)  contain sample tables of inconsistencies and incompatibilities and an indication of the process of file-building, including examples of SPSS commands used to resolve them.

 

Serious problems with measurement levels and missing values remain to be resolved.  Incorrect measurement levels may be due to SPSS automatically applying “heuristic” levels to variables depending on the number of values encountered, but they could equally be due to automatic archiving software used at UKDS.  See: British Social Attitudes 2009 - 2014 Measurement levels
http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014_measurement_levels.pdf

 

Missing values are declared for all variables as negative, almost all in the range Lo thru -1, but the original positive values to be treated as missing (8, 98, 998 and 9, 99, 999) have not been changed and consequently are not treated as missing for analysis purposes.  See: British Social Attitudes 2009 to 2014 _ Missing values http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014___missing_values.pdf

 

Checking and correction of these will be routine, but extremely tedious, and, unless I can use some existing syntax from my earlier work or find the “cleanest” individual wave files and copy metadata across using the SPSS command APPLY DICTIONARY, will take some time yet.

 

At 1.04 gb the current file is very large: if it won’t go via Dropbox, can anyone suggest an alternative method of sending it to Natcen (the owners) and UKDS (the distributors)?  If not, I can burn it (encrypted) to DVD and use snail-mail.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

John F Hall
In reply to this post by Jon Peck

An additional problem.  My file has:

 

HipOpA  Scale  how many beans. .  to reduce hospital waiting times for . . a hip operation ?:Q370

 

variable level

HipOpA

(scale).

 

. . but I get:

 

Error # 701 in column 1.  Text: HipOpA

An undefined variable name, or a scratch or system variable was specified in a

variable list which accepts only standard variables.  Check spelling and

verify the existence of this variable.

Execution of this command stops.

 

This occurs with other variables as well.  Can anyone suggest why?

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 22 April 2016 07:48
To: '[hidden email]' <[hidden email]>
Subject: RE: Reverse engineering from *.sav to *.sps

 

I have a version of the  BSA cumulative *.sav file created with:

 

SORT VARIABLES by values (d).

 

. . and have been working through looking at value labels beginning with 8, 98, 9, 99 which should be missing, but which are not. 

 

{99, Refusal}...

{99, Not answered}...

{98, Don't   know}...

{8, Don't   know}...

 

Any positive such codes can then be recoded to -8. -9, -98, -99  etc to be caught by (Lo thru -1) .

 

However I have a number of variables which contain 98, 99 etc but which do not display 98 or 99 in the Values column.  For example:

 

VoteEU           {1, Yes: voted}

 

... but the frequency table shows:

 

VotedEU Did Respondent vote in european election 2004? Q1027

 

 

Frequency

Percent

Valid Percent

Cumulative Percent

 

Valid

-2

2083

2.2

19.6

19.6

 

1 Yes: voted

3358

3.5

31.6

51.3

 

2 No

4992

5.2

47.0

98.3

 

97 (Refused to say)

26

.0

.2

98.6

 

98 Don't   know

151

.2

1.4

100.0

 

Total

10610

11.1

100.0

 

 

Missing

System

85020

88.9

 

 

 

 

Total

95630

100.0

 

 

 

I expected SORT VARIABLES by values (d). to display:

 

VoteEU           {98 Don't   know }

 

Is there another way I can force the Values column in the Data Editor to display the highest positive value? 

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

 

 

From: John F Hall [[hidden email]]
Sent: 19 April 2016 07:55
To: '[hidden email]' <[hidden email]>
Subject: RE: Reverse engineering from *.sav to *.sps

 

Jon

 

Not sure what exactly I will get, so will run your syntax on a copy.  This means the cases will remain in their original order in the source file. 

 

There could well be several thousand varnames thrown up, so if I change my SPSS settings to names only, I should be able to pick up the first and last varnames from the first column of the table and use:

 

var lev <first> to <last> (ord).

recode <first> to <last> (8 = -8)(9 = -9).

missing values <first> to <last> (lo thru -1).

 

Just used Ctrl+F for “strongly disagree” in the Values column and the first block of variables found had values ranging from 1 = “1 Strongly disagree” to 10 = “10 Strongly agree” [sic: must have been written by a complete beginner] with -1 = “Not applicable”, 98 “Don’t know”and 99 “Refused”.   However “10 Strongly agree” could be used to identify items using 1-10 scales.  Lo thru -1 is already declared as missing for most variables and will pick up the -1.  98 and 99 will have to be recoded and given new labels.

 

This creates another problem since there are also other Agree – Disagree items coded in the opposite direction!  One set has five valid values (1 = “Agree strongly” 2 = “Agree” 3 = “Neither agree nor disagree” 4 = “Disagree” 5 = “Disagree strongly”)   Another has only four (1 = “Strongly agree” 2 = “Tend to agree” 3 = “Tend to disagree” 4 = “Strongly disagree”).  Again the unique labels “Neither agree nor disagree” could be used to find the 1-5 items and “Tend to disagree” to find the 1-4 items.

 

Just ran your syntax.  Judging by the new file I think I can identify sets of variables with the same properties from the Var2 column and copy blocks of names from Var1 into a new syntax file.   In fact there are only three variables using the 1 -10 agree - disagree scale, but all the others are listed in blocks so I can use <first> to <last> syntax to set levels and missing values. 

 

Thanks a million: you just saved me days if not weeks.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Jon Peck
Sent: 18 April 2016 19:36
To:
[hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

For starters, you could run this code, which will give you a dataset containing an alphabetical list of all the value labels in the dataset along with other properties.  You could then select out the words you want to examine.

 

dataset declare values.

oms select tables /if subtypes='Variable Values'

/destination outfile=values format=sav viewer=no.

display dict.

omsend.

 

dataset activate values.

sort cases by Label.

 

 

On Mon, Apr 18, 2016 at 11:16 AM, John F Hall <[hidden email]> wrote:

Dropbox has accepted the large *.sav file and created a link for Natcen and UKDS to download it.  Access will be strictly via UKDS and after approval by Natcen.

 

I now have another question.  I can use Ctrl+F to find value labels such as “Strongly

Disagree” in the Values column.  Is it possible to identify all such labels in a single search and produce a list of the variables containing that phrase?  If so this would immensely speed up the correct specification of measurement levels as Ordinal and missing values  8 = Don’t know and 9 = Refused changed to -8 and -9 to be captured by the universal Lo thru -1 in the Missing column.

 

Variables with two digit values will be more difficult to identify unless they have labels such as “Completely satisfied”: some of these are measured on 0-10 scales, others on 1-5.  All two-digit variables have 98 and 99 as missing: they can be changed to -98 and -99.  Some variables have three digits with 998 and 999 as missing, but I can find these myself.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 18 April 2016 17:23
To: '
[hidden email]' <[hidden email]>
Cc: '
[hidden email]' <[hidden email]>
Subject: FW: Reverse engineering from *.sav to *.sps

 

Apologies for duplicate posting: the original had hyperlinks which the server may reject.  I have replaced them with URLs.

 

I have uploaded two new commentaries to my page Exploring British Social Attitudes http://surveyresearch.weebly.com/exploring-british-social-attitudes.html

 

British Social Attitudes 1983 to 2014: Resolution of type conflicts   

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983_-_2014_resolution_of_type_conflicts.pdf  (11 pp)

and British Social Attitudes 1983-2014:  Beta version of cumulative file

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983-2014__beta_version_of_cumulative_file.pdf  (3pp)  contain sample tables of inconsistencies and incompatibilities and an indication of the process of file-building, including examples of SPSS commands used to resolve them.

 

Serious problems with measurement levels and missing values remain to be resolved.  Incorrect measurement levels may be due to SPSS automatically applying “heuristic” levels to variables depending on the number of values encountered, but they could equally be due to automatic archiving software used at UKDS.  See: British Social Attitudes 2009 - 2014 Measurement levels
http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014_measurement_levels.pdf

 

Missing values are declared for all variables as negative, almost all in the range Lo thru -1, but the original positive values to be treated as missing (8, 98, 998 and 9, 99, 999) have not been changed and consequently are not treated as missing for analysis purposes.  See: British Social Attitudes 2009 to 2014 _ Missing values http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014___missing_values.pdf

 

Checking and correction of these will be routine, but extremely tedious, and, unless I can use some existing syntax from my earlier work or find the “cleanest” individual wave files and copy metadata across using the SPSS command APPLY DICTIONARY, will take some time yet.

 

At 1.04 gb the current file is very large: if it won’t go via Dropbox, can anyone suggest an alternative method of sending it to Natcen (the owners) and UKDS (the distributors)?  If not, I can burn it (encrypted) to DVD and use snail-mail.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

Jon Peck
In reply to this post by John F Hall
The values column only shows one value.  There is no mechanism to make it show more.  You have to look at the box under the ellipsis or use CODEBOOK or DISPLAY DICT to see the full list.

The only way I can think of to get more displayed in Variable View would  be to run a little programmability code that examined the value labels and created a custom attribute for each of 99, 98, and 8.  Then these could appear as additional columns in Variable View or to concatenate these and show as a attribute column listing the values of interest.

I don't have time to do this today, but let me know if this would help, and I'll do it over the weekend.

On Thu, Apr 21, 2016 at 11:48 PM, John F Hall <[hidden email]> wrote:

I have a version of the  BSA cumulative *.sav file created with:

 

SORT VARIABLES by values (d).

 

. . and have been working through looking at value labels beginning with 8, 98, 9, 99 which should be missing, but which are not. 

 

{99, Refusal}...

{99, Not answered}...

{98, Don't   know}...

{8, Don't   know}...

 

Any positive such codes can then be recoded to -8. -9, -98, -99  etc to be caught by (Lo thru -1) .

 

However I have a number of variables which contain 98, 99 etc but which do not display 98 or 99 in the Values column.  For example:

 

VoteEU          {1, Yes: voted}

 

... but the frequency table shows:

 

VotedEU Did Respondent vote in european election 2004? Q1027

 

 

Frequency

Percent

Valid Percent

Cumulative Percent

 

Valid

-2

2083

2.2

19.6

19.6

 

1 Yes: voted

3358

3.5

31.6

51.3

 

2 No

4992

5.2

47.0

98.3

 

97 (Refused to say)

26

.0

.2

98.6

 

98 Don't   know

151

.2

1.4

100.0

 

Total

10610

11.1

100.0

 

 

Missing

System

85020

88.9

 

 

 

 

Total

95630

100.0

 

 

 

I expected SORT VARIABLES by values (d). to display:

 

VoteEU           {98 Don't   know }

 

Is there another way I can force the Values column in the Data Editor to display the highest positive value? 

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 19 April 2016 07:55
To: '[hidden email]' <[hidden email]>
Subject: RE: Reverse engineering from *.sav to *.sps

 

Jon

 

Not sure what exactly I will get, so will run your syntax on a copy.  This means the cases will remain in their original order in the source file. 

 

There could well be several thousand varnames thrown up, so if I change my SPSS settings to names only, I should be able to pick up the first and last varnames from the first column of the table and use:

 

var lev <first> to <last> (ord).

recode <first> to <last> (8 = -8)(9 = -9).

missing values <first> to <last> (lo thru -1).

 

Just used Ctrl+F for “strongly disagree” in the Values column and the first block of variables found had values ranging from 1 = “1 Strongly disagree” to 10 = “10 Strongly agree” [sic: must have been written by a complete beginner] with -1 = “Not applicable”, 98 “Don’t know”and 99 “Refused”.   However “10 Strongly agree” could be used to identify items using 1-10 scales.  Lo thru -1 is already declared as missing for most variables and will pick up the -1.  98 and 99 will have to be recoded and given new labels.

 

This creates another problem since there are also other Agree – Disagree items coded in the opposite direction!  One set has five valid values (1 = “Agree strongly” 2 = “Agree” 3 = “Neither agree nor disagree” 4 = “Disagree” 5 = “Disagree strongly”)   Another has only four (1 = “Strongly agree” 2 = “Tend to agree” 3 = “Tend to disagree” 4 = “Strongly disagree”).  Again the unique labels “Neither agree nor disagree” could be used to find the 1-5 items and “Tend to disagree” to find the 1-4 items.

 

Just ran your syntax.  Judging by the new file I think I can identify sets of variables with the same properties from the Var2 column and copy blocks of names from Var1 into a new syntax file.   In fact there are only three variables using the 1 -10 agree - disagree scale, but all the others are listed in blocks so I can use <first> to <last> syntax to set levels and missing values. 

 

Thanks a million: you just saved me days if not weeks.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Jon Peck
Sent: 18 April 2016 19:36
To:
[hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

For starters, you could run this code, which will give you a dataset containing an alphabetical list of all the value labels in the dataset along with other properties.  You could then select out the words you want to examine.

 

dataset declare values.

oms select tables /if subtypes='Variable Values'

/destination outfile=values format=sav viewer=no.

display dict.

omsend.

 

dataset activate values.

sort cases by Label.

 

 

On Mon, Apr 18, 2016 at 11:16 AM, John F Hall <[hidden email]> wrote:

Dropbox has accepted the large *.sav file and created a link for Natcen and UKDS to download it.  Access will be strictly via UKDS and after approval by Natcen.

 

I now have another question.  I can use Ctrl+F to find value labels such as “Strongly

Disagree” in the Values column.  Is it possible to identify all such labels in a single search and produce a list of the variables containing that phrase?  If so this would immensely speed up the correct specification of measurement levels as Ordinal and missing values  8 = Don’t know and 9 = Refused changed to -8 and -9 to be captured by the universal Lo thru -1 in the Missing column.

 

Variables with two digit values will be more difficult to identify unless they have labels such as “Completely satisfied”: some of these are measured on 0-10 scales, others on 1-5.  All two-digit variables have 98 and 99 as missing: they can be changed to -98 and -99.  Some variables have three digits with 998 and 999 as missing, but I can find these myself.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 18 April 2016 17:23
To: '
[hidden email]' <[hidden email]>
Cc: '
[hidden email]' <[hidden email]>
Subject: FW: Reverse engineering from *.sav to *.sps

 

Apologies for duplicate posting: the original had hyperlinks which the server may reject.  I have replaced them with URLs.

 

I have uploaded two new commentaries to my page Exploring British Social Attitudes http://surveyresearch.weebly.com/exploring-british-social-attitudes.html

 

British Social Attitudes 1983 to 2014: Resolution of type conflicts   

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983_-_2014_resolution_of_type_conflicts.pdf  (11 pp)

and British Social Attitudes 1983-2014:  Beta version of cumulative file

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983-2014__beta_version_of_cumulative_file.pdf  (3pp)  contain sample tables of inconsistencies and incompatibilities and an indication of the process of file-building, including examples of SPSS commands used to resolve them.

 

Serious problems with measurement levels and missing values remain to be resolved.  Incorrect measurement levels may be due to SPSS automatically applying “heuristic” levels to variables depending on the number of values encountered, but they could equally be due to automatic archiving software used at UKDS.  See: British Social Attitudes 2009 - 2014 Measurement levels
http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014_measurement_levels.pdf

 

Missing values are declared for all variables as negative, almost all in the range Lo thru -1, but the original positive values to be treated as missing (8, 98, 998 and 9, 99, 999) have not been changed and consequently are not treated as missing for analysis purposes.  See: British Social Attitudes 2009 to 2014 _ Missing values http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014___missing_values.pdf

 

Checking and correction of these will be routine, but extremely tedious, and, unless I can use some existing syntax from my earlier work or find the “cleanest” individual wave files and copy metadata across using the SPSS command APPLY DICTIONARY, will take some time yet.

 

At 1.04 gb the current file is very large: if it won’t go via Dropbox, can anyone suggest an alternative method of sending it to Natcen (the owners) and UKDS (the distributors)?  If not, I can burn it (encrypted) to DVD and use snail-mail.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

John F Hall

 

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

Jon

 

That’s what I do to check.  What I want to know is why the high value 99 does not display instead of the low value -2 or whatever.

 

John

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Jon Peck
Sent: 22 April 2016 15:49
To: [hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

The values column only shows one value.  There is no mechanism to make it show more.  You have to look at the box under the ellipsis or use CODEBOOK or DISPLAY DICT to see the full list.

 

The only way I can think of to get more displayed in Variable View would  be to run a little programmability code that examined the value labels and created a custom attribute for each of 99, 98, and 8.  Then these could appear as additional columns in Variable View or to concatenate these and show as a attribute column listing the values of interest.

 

I don't have time to do this today, but let me know if this would help, and I'll do it over the weekend.

 

On Thu, Apr 21, 2016 at 11:48 PM, John F Hall <[hidden email]> wrote:

I have a version of the  BSA cumulative *.sav file created with:

 

SORT VARIABLES by values (d).

 

. . and have been working through looking at value labels beginning with 8, 98, 9, 99 which should be missing, but which are not. 

 

{99, Refusal}...

{99, Not answered}...

{98, Don't   know}...

{8, Don't   know}...

 

Any positive such codes can then be recoded to -8. -9, -98, -99  etc to be caught by (Lo thru -1) .

 

However I have a number of variables which contain 98, 99 etc but which do not display 98 or 99 in the Values column.  For example:

 

VoteEU          {1, Yes: voted}

 

... but the frequency table shows:

 

VotedEU Did Respondent vote in european election 2004? Q1027

 

 

Frequency

Percent

Valid Percent

Cumulative Percent

 

Valid

-2

2083

2.2

19.6

19.6

 

1 Yes: voted

3358

3.5

31.6

51.3

 

2 No

4992

5.2

47.0

98.3

 

97 (Refused to say)

26

.0

.2

98.6

 

98 Don't   know

151

.2

1.4

100.0

 

Total

10610

11.1

100.0

 

 

Missing

System

85020

88.9

 

 

 

 

Total

95630

100.0

 

 

 

I expected SORT VARIABLES by values (d). to display:

 

VoteEU           {98 Don't   know }

 

Is there another way I can force the Values column in the Data Editor to display the highest positive value? 

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 19 April 2016 07:55
To: '[hidden email]' <[hidden email]>
Subject: RE: Reverse engineering from *.sav to *.sps

 

Jon

 

Not sure what exactly I will get, so will run your syntax on a copy.  This means the cases will remain in their original order in the source file. 

 

There could well be several thousand varnames thrown up, so if I change my SPSS settings to names only, I should be able to pick up the first and last varnames from the first column of the table and use:

 

var lev <first> to <last> (ord).

recode <first> to <last> (8 = -8)(9 = -9).

missing values <first> to <last> (lo thru -1).

 

Just used Ctrl+F for “strongly disagree” in the Values column and the first block of variables found had values ranging from 1 = “1 Strongly disagree” to 10 = “10 Strongly agree” [sic: must have been written by a complete beginner] with -1 = “Not applicable”, 98 “Don’t know”and 99 “Refused”.   However “10 Strongly agree” could be used to identify items using 1-10 scales.  Lo thru -1 is already declared as missing for most variables and will pick up the -1.  98 and 99 will have to be recoded and given new labels.

 

This creates another problem since there are also other Agree – Disagree items coded in the opposite direction!  One set has five valid values (1 = “Agree strongly” 2 = “Agree” 3 = “Neither agree nor disagree” 4 = “Disagree” 5 = “Disagree strongly”)   Another has only four (1 = “Strongly agree” 2 = “Tend to agree” 3 = “Tend to disagree” 4 = “Strongly disagree”).  Again the unique labels “Neither agree nor disagree” could be used to find the 1-5 items and “Tend to disagree” to find the 1-4 items.

 

Just ran your syntax.  Judging by the new file I think I can identify sets of variables with the same properties from the Var2 column and copy blocks of names from Var1 into a new syntax file.   In fact there are only three variables using the 1 -10 agree - disagree scale, but all the others are listed in blocks so I can use <first> to <last> syntax to set levels and missing values. 

 

Thanks a million: you just saved me days if not weeks.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Jon Peck
Sent: 18 April 2016 19:36
To:
[hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

For starters, you could run this code, which will give you a dataset containing an alphabetical list of all the value labels in the dataset along with other properties.  You could then select out the words you want to examine.

 

dataset declare values.

oms select tables /if subtypes='Variable Values'

/destination outfile=values format=sav viewer=no.

display dict.

omsend.

 

dataset activate values.

sort cases by Label.

 

 

On Mon, Apr 18, 2016 at 11:16 AM, John F Hall <[hidden email]> wrote:

Dropbox has accepted the large *.sav file and created a link for Natcen and UKDS to download it.  Access will be strictly via UKDS and after approval by Natcen.

 

I now have another question.  I can use Ctrl+F to find value labels such as “Strongly

Disagree” in the Values column.  Is it possible to identify all such labels in a single search and produce a list of the variables containing that phrase?  If so this would immensely speed up the correct specification of measurement levels as Ordinal and missing values  8 = Don’t know and 9 = Refused changed to -8 and -9 to be captured by the universal Lo thru -1 in the Missing column.

 

Variables with two digit values will be more difficult to identify unless they have labels such as “Completely satisfied”: some of these are measured on 0-10 scales, others on 1-5.  All two-digit variables have 98 and 99 as missing: they can be changed to -98 and -99.  Some variables have three digits with 998 and 999 as missing, but I can find these myself.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 18 April 2016 17:23
To: '
[hidden email]' <[hidden email]>
Cc: '
[hidden email]' <[hidden email]>
Subject: FW: Reverse engineering from *.sav to *.sps

 

Apologies for duplicate posting: the original had hyperlinks which the server may reject.  I have replaced them with URLs.

 

I have uploaded two new commentaries to my page Exploring British Social Attitudes http://surveyresearch.weebly.com/exploring-british-social-attitudes.html

 

British Social Attitudes 1983 to 2014: Resolution of type conflicts   

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983_-_2014_resolution_of_type_conflicts.pdf  (11 pp)

and British Social Attitudes 1983-2014:  Beta version of cumulative file

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983-2014__beta_version_of_cumulative_file.pdf  (3pp)  contain sample tables of inconsistencies and incompatibilities and an indication of the process of file-building, including examples of SPSS commands used to resolve them.

 

Serious problems with measurement levels and missing values remain to be resolved.  Incorrect measurement levels may be due to SPSS automatically applying “heuristic” levels to variables depending on the number of values encountered, but they could equally be due to automatic archiving software used at UKDS.  See: British Social Attitudes 2009 - 2014 Measurement levels
http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014_measurement_levels.pdf

 

Missing values are declared for all variables as negative, almost all in the range Lo thru -1, but the original positive values to be treated as missing (8, 98, 998 and 9, 99, 999) have not been changed and consequently are not treated as missing for analysis purposes.  See: British Social Attitudes 2009 to 2014 _ Missing values http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014___missing_values.pdf

 

Checking and correction of these will be routine, but extremely tedious, and, unless I can use some existing syntax from my earlier work or find the “cleanest” individual wave files and copy metadata across using the SPSS command APPLY DICTIONARY, will take some time yet.

 

At 1.04 gb the current file is very large: if it won’t go via Dropbox, can anyone suggest an alternative method of sending it to Natcen (the owners) and UKDS (the distributors)?  If not, I can burn it (encrypted) to DVD and use snail-mail.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

John F Hall
In reply to this post by Jon Peck

Found the answer.  It’s not the file, it’s SPSS which does not always switch the active file marker when I open the newly created one.  Not quite sure why this happens.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 22 April 2016 12:29
To: '[hidden email]' <[hidden email]>
Subject: RE: Reverse engineering from *.sav to *.sps

 

An additional problem.  My file has:

 

HipOpA  Scale  how many beans. .  to reduce hospital waiting times for . . a hip operation ?:Q370

 

variable level

HipOpA

(scale).

 

. . but I get:

 

Error # 701 in column 1.  Text: HipOpA

An undefined variable name, or a scratch or system variable was specified in a

variable list which accepts only standard variables.  Check spelling and

verify the existence of this variable.

Execution of this command stops.

 

This occurs with other variables as well.  Can anyone suggest why?

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

From: John F Hall [[hidden email]]
Sent: 22 April 2016 07:48
To: '[hidden email]' <
[hidden email]>
Subject: RE: Reverse engineering from *.sav to *.sps

 

I have a version of the  BSA cumulative *.sav file created with:

 

SORT VARIABLES by values (d).

 

. . and have been working through looking at value labels beginning with 8, 98, 9, 99 which should be missing, but which are not. 

 

{99, Refusal}...

{99, Not answered}...

{98, Don't   know}...

{8, Don't   know}...

 

Any positive such codes can then be recoded to -8. -9, -98, -99  etc to be caught by (Lo thru -1) .

 

However I have a number of variables which contain 98, 99 etc but which do not display 98 or 99 in the Values column.  For example:

 

VoteEU           {1, Yes: voted}

 

... but the frequency table shows:

 

VotedEU Did Respondent vote in european election 2004? Q1027

 

 

Frequency

Percent

Valid Percent

Cumulative Percent

 

Valid

-2

2083

2.2

19.6

19.6

 

1 Yes: voted

3358

3.5

31.6

51.3

 

2 No

4992

5.2

47.0

98.3

 

97 (Refused to say)

26

.0

.2

98.6

 

98 Don't   know

151

.2

1.4

100.0

 

Total

10610

11.1

100.0

 

 

Missing

System

85020

88.9

 

 

 

 

Total

95630

100.0

 

 

 

I expected SORT VARIABLES by values (d). to display:

 

VoteEU           {98 Don't   know }

 

Is there another way I can force the Values column in the Data Editor to display the highest positive value? 

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

 

 

From: John F Hall [[hidden email]]
Sent: 19 April 2016 07:55
To: '[hidden email]' <
[hidden email]>
Subject: RE: Reverse engineering from *.sav to *.sps

 

Jon

 

Not sure what exactly I will get, so will run your syntax on a copy.  This means the cases will remain in their original order in the source file. 

 

There could well be several thousand varnames thrown up, so if I change my SPSS settings to names only, I should be able to pick up the first and last varnames from the first column of the table and use:

 

var lev <first> to <last> (ord).

recode <first> to <last> (8 = -8)(9 = -9).

missing values <first> to <last> (lo thru -1).

 

Just used Ctrl+F for “strongly disagree” in the Values column and the first block of variables found had values ranging from 1 = “1 Strongly disagree” to 10 = “10 Strongly agree” [sic: must have been written by a complete beginner] with -1 = “Not applicable”, 98 “Don’t know”and 99 “Refused”.   However “10 Strongly agree” could be used to identify items using 1-10 scales.  Lo thru -1 is already declared as missing for most variables and will pick up the -1.  98 and 99 will have to be recoded and given new labels.

 

This creates another problem since there are also other Agree – Disagree items coded in the opposite direction!  One set has five valid values (1 = “Agree strongly” 2 = “Agree” 3 = “Neither agree nor disagree” 4 = “Disagree” 5 = “Disagree strongly”)   Another has only four (1 = “Strongly agree” 2 = “Tend to agree” 3 = “Tend to disagree” 4 = “Strongly disagree”).  Again the unique labels “Neither agree nor disagree” could be used to find the 1-5 items and “Tend to disagree” to find the 1-4 items.

 

Just ran your syntax.  Judging by the new file I think I can identify sets of variables with the same properties from the Var2 column and copy blocks of names from Var1 into a new syntax file.   In fact there are only three variables using the 1 -10 agree - disagree scale, but all the others are listed in blocks so I can use <first> to <last> syntax to set levels and missing values. 

 

Thanks a million: you just saved me days if not weeks.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Jon Peck
Sent: 18 April 2016 19:36
To:
[hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

For starters, you could run this code, which will give you a dataset containing an alphabetical list of all the value labels in the dataset along with other properties.  You could then select out the words you want to examine.

 

dataset declare values.

oms select tables /if subtypes='Variable Values'

/destination outfile=values format=sav viewer=no.

display dict.

omsend.

 

dataset activate values.

sort cases by Label.

 

 

On Mon, Apr 18, 2016 at 11:16 AM, John F Hall <[hidden email]> wrote:

Dropbox has accepted the large *.sav file and created a link for Natcen and UKDS to download it.  Access will be strictly via UKDS and after approval by Natcen.

 

I now have another question.  I can use Ctrl+F to find value labels such as “Strongly

Disagree” in the Values column.  Is it possible to identify all such labels in a single search and produce a list of the variables containing that phrase?  If so this would immensely speed up the correct specification of measurement levels as Ordinal and missing values  8 = Don’t know and 9 = Refused changed to -8 and -9 to be captured by the universal Lo thru -1 in the Missing column.

 

Variables with two digit values will be more difficult to identify unless they have labels such as “Completely satisfied”: some of these are measured on 0-10 scales, others on 1-5.  All two-digit variables have 98 and 99 as missing: they can be changed to -98 and -99.  Some variables have three digits with 998 and 999 as missing, but I can find these myself.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 18 April 2016 17:23
To: '
[hidden email]' <[hidden email]>
Cc: '
[hidden email]' <[hidden email]>
Subject: FW: Reverse engineering from *.sav to *.sps

 

Apologies for duplicate posting: the original had hyperlinks which the server may reject.  I have replaced them with URLs.

 

I have uploaded two new commentaries to my page Exploring British Social Attitudes http://surveyresearch.weebly.com/exploring-british-social-attitudes.html

 

British Social Attitudes 1983 to 2014: Resolution of type conflicts   

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983_-_2014_resolution_of_type_conflicts.pdf  (11 pp)

and British Social Attitudes 1983-2014:  Beta version of cumulative file

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983-2014__beta_version_of_cumulative_file.pdf  (3pp)  contain sample tables of inconsistencies and incompatibilities and an indication of the process of file-building, including examples of SPSS commands used to resolve them.

 

Serious problems with measurement levels and missing values remain to be resolved.  Incorrect measurement levels may be due to SPSS automatically applying “heuristic” levels to variables depending on the number of values encountered, but they could equally be due to automatic archiving software used at UKDS.  See: British Social Attitudes 2009 - 2014 Measurement levels
http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014_measurement_levels.pdf

 

Missing values are declared for all variables as negative, almost all in the range Lo thru -1, but the original positive values to be treated as missing (8, 98, 998 and 9, 99, 999) have not been changed and consequently are not treated as missing for analysis purposes.  See: British Social Attitudes 2009 to 2014 _ Missing values http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014___missing_values.pdf

 

Checking and correction of these will be routine, but extremely tedious, and, unless I can use some existing syntax from my earlier work or find the “cleanest” individual wave files and copy metadata across using the SPSS command APPLY DICTIONARY, will take some time yet.

 

At 1.04 gb the current file is very large: if it won’t go via Dropbox, can anyone suggest an alternative method of sending it to Natcen (the owners) and UKDS (the distributors)?  If not, I can burn it (encrypted) to DVD and use snail-mail.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

Jon Peck
In reply to this post by John F Hall
AFAIK, the list of values is maintained in numerical or alpha order, and just the first one is shown in the DE.  You will see that if you turn on the value label display in Data View, the list order there is the same.

On Fri, Apr 22, 2016 at 11:21 AM, John F Hall <[hidden email]> wrote:

 

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

Jon

 

That’s what I do to check.  What I want to know is why the high value 99 does not display instead of the low value -2 or whatever.

 

John

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Jon Peck
Sent: 22 April 2016 15:49
To: [hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

The values column only shows one value.  There is no mechanism to make it show more.  You have to look at the box under the ellipsis or use CODEBOOK or DISPLAY DICT to see the full list.

 

The only way I can think of to get more displayed in Variable View would  be to run a little programmability code that examined the value labels and created a custom attribute for each of 99, 98, and 8.  Then these could appear as additional columns in Variable View or to concatenate these and show as a attribute column listing the values of interest.

 

I don't have time to do this today, but let me know if this would help, and I'll do it over the weekend.

 

On Thu, Apr 21, 2016 at 11:48 PM, John F Hall <[hidden email]> wrote:

I have a version of the  BSA cumulative *.sav file created with:

 

SORT VARIABLES by values (d).

 

. . and have been working through looking at value labels beginning with 8, 98, 9, 99 which should be missing, but which are not. 

 

{99, Refusal}...

{99, Not answered}...

{98, Don't   know}...

{8, Don't   know}...

 

Any positive such codes can then be recoded to -8. -9, -98, -99  etc to be caught by (Lo thru -1) .

 

However I have a number of variables which contain 98, 99 etc but which do not display 98 or 99 in the Values column.  For example:

 

VoteEU          {1, Yes: voted}

 

... but the frequency table shows:

 

VotedEU Did Respondent vote in european election 2004? Q1027

 

 

Frequency

Percent

Valid Percent

Cumulative Percent

 

Valid

-2

2083

2.2

19.6

19.6

 

1 Yes: voted

3358

3.5

31.6

51.3

 

2 No

4992

5.2

47.0

98.3

 

97 (Refused to say)

26

.0

.2

98.6

 

98 Don't   know

151

.2

1.4

100.0

 

Total

10610

11.1

100.0

 

 

Missing

System

85020

88.9

 

 

 

 

Total

95630

100.0

 

 

 

I expected SORT VARIABLES by values (d). to display:

 

VoteEU           {98 Don't   know }

 

Is there another way I can force the Values column in the Data Editor to display the highest positive value? 

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 19 April 2016 07:55
To: '[hidden email]' <[hidden email]>
Subject: RE: Reverse engineering from *.sav to *.sps

 

Jon

 

Not sure what exactly I will get, so will run your syntax on a copy.  This means the cases will remain in their original order in the source file. 

 

There could well be several thousand varnames thrown up, so if I change my SPSS settings to names only, I should be able to pick up the first and last varnames from the first column of the table and use:

 

var lev <first> to <last> (ord).

recode <first> to <last> (8 = -8)(9 = -9).

missing values <first> to <last> (lo thru -1).

 

Just used Ctrl+F for “strongly disagree” in the Values column and the first block of variables found had values ranging from 1 = “1 Strongly disagree” to 10 = “10 Strongly agree” [sic: must have been written by a complete beginner] with -1 = “Not applicable”, 98 “Don’t know”and 99 “Refused”.   However “10 Strongly agree” could be used to identify items using 1-10 scales.  Lo thru -1 is already declared as missing for most variables and will pick up the -1.  98 and 99 will have to be recoded and given new labels.

 

This creates another problem since there are also other Agree – Disagree items coded in the opposite direction!  One set has five valid values (1 = “Agree strongly” 2 = “Agree” 3 = “Neither agree nor disagree” 4 = “Disagree” 5 = “Disagree strongly”)   Another has only four (1 = “Strongly agree” 2 = “Tend to agree” 3 = “Tend to disagree” 4 = “Strongly disagree”).  Again the unique labels “Neither agree nor disagree” could be used to find the 1-5 items and “Tend to disagree” to find the 1-4 items.

 

Just ran your syntax.  Judging by the new file I think I can identify sets of variables with the same properties from the Var2 column and copy blocks of names from Var1 into a new syntax file.   In fact there are only three variables using the 1 -10 agree - disagree scale, but all the others are listed in blocks so I can use <first> to <last> syntax to set levels and missing values. 

 

Thanks a million: you just saved me days if not weeks.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Jon Peck
Sent: 18 April 2016 19:36
To:
[hidden email]
Subject: Re: Reverse engineering from *.sav to *.sps

 

For starters, you could run this code, which will give you a dataset containing an alphabetical list of all the value labels in the dataset along with other properties.  You could then select out the words you want to examine.

 

dataset declare values.

oms select tables /if subtypes='Variable Values'

/destination outfile=values format=sav viewer=no.

display dict.

omsend.

 

dataset activate values.

sort cases by Label.

 

 

On Mon, Apr 18, 2016 at 11:16 AM, John F Hall <[hidden email]> wrote:

Dropbox has accepted the large *.sav file and created a link for Natcen and UKDS to download it.  Access will be strictly via UKDS and after approval by Natcen.

 

I now have another question.  I can use Ctrl+F to find value labels such as “Strongly

Disagree” in the Values column.  Is it possible to identify all such labels in a single search and produce a list of the variables containing that phrase?  If so this would immensely speed up the correct specification of measurement levels as Ordinal and missing values  8 = Don’t know and 9 = Refused changed to -8 and -9 to be captured by the universal Lo thru -1 in the Missing column.

 

Variables with two digit values will be more difficult to identify unless they have labels such as “Completely satisfied”: some of these are measured on 0-10 scales, others on 1-5.  All two-digit variables have 98 and 99 as missing: they can be changed to -98 and -99.  Some variables have three digits with 998 and 999 as missing, but I can find these myself.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

From: John F Hall [mailto:[hidden email]]
Sent: 18 April 2016 17:23
To: '
[hidden email]' <[hidden email]>
Cc: '
[hidden email]' <[hidden email]>
Subject: FW: Reverse engineering from *.sav to *.sps

 

Apologies for duplicate posting: the original had hyperlinks which the server may reject.  I have replaced them with URLs.

 

I have uploaded two new commentaries to my page Exploring British Social Attitudes http://surveyresearch.weebly.com/exploring-british-social-attitudes.html

 

British Social Attitudes 1983 to 2014: Resolution of type conflicts   

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983_-_2014_resolution_of_type_conflicts.pdf  (11 pp)

and British Social Attitudes 1983-2014:  Beta version of cumulative file

http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_1983-2014__beta_version_of_cumulative_file.pdf  (3pp)  contain sample tables of inconsistencies and incompatibilities and an indication of the process of file-building, including examples of SPSS commands used to resolve them.

 

Serious problems with measurement levels and missing values remain to be resolved.  Incorrect measurement levels may be due to SPSS automatically applying “heuristic” levels to variables depending on the number of values encountered, but they could equally be due to automatic archiving software used at UKDS.  See: British Social Attitudes 2009 - 2014 Measurement levels
http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014_measurement_levels.pdf

 

Missing values are declared for all variables as negative, almost all in the range Lo thru -1, but the original positive values to be treated as missing (8, 98, 998 and 9, 99, 999) have not been changed and consequently are not treated as missing for analysis purposes.  See: British Social Attitudes 2009 to 2014 _ Missing values http://surveyresearch.weebly.com/uploads/2/9/9/8/2998485/british_social_attitudes_2009_-_2014___missing_values.pdf

 

Checking and correction of these will be routine, but extremely tedious, and, unless I can use some existing syntax from my earlier work or find the “cleanest” individual wave files and copy metadata across using the SPSS command APPLY DICTIONARY, will take some time yet.

 

At 1.04 gb the current file is very large: if it won’t go via Dropbox, can anyone suggest an alternative method of sending it to Natcen (the owners) and UKDS (the distributors)?  If not, I can burn it (encrypted) to DVD and use snail-mail.

 

John F Hall (Mr)

[Retired academic survey researcher]

 

Email:   [hidden email] 

Website: www.surveyresearch.weebly.com

SPSS start page:  www.surveyresearch.weebly.com/1-survey-analysis-workshop

 

 

 

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



 

--

Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Reverse engineering from *.sav to *.sps

David Marso
Administrator
In reply to this post by John F Hall
John,
PLEASE TRIM YOUR POSTS!!!!
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me.
---
"Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis."
Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?"