DATA LIST FILE Command

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

DATA LIST FILE Command

Zachary Feinstein
I have to read in some very large sets of data.  Currently I am doing something like the following:
 
DATA LIST FILE = xyz RECORDS = 4
 /resp = 1- 4
 /
 /mostoften 7 - 9
 /current1 TO current31 7 - 99
...
And so on.
 
The text files that have the data are arranged by card.  Columns 5 - 6 let me know the exact card for each respondent.  So, in the above example, a 01, 02, 03, or 04 would appear in columns 5 - 6.
 
Is there a way in SPSS for me to read the card number and let DATA LIST FILE know to put the relevant card information in for each respondent?  I greatly oversimplified the issue I am encountering here and wish to remove any thought that goes into the ordering of the subcommands above.
 
Thank you.
 
Zachary
 

Reply | Threaded
Open this post in threaded view
|

Re: DATA LIST FILE Command

Marks, Jim

Zachary:

 

Does the card number change the record layout?

 

I had a similar task using a text file—different record length and variables depending on an identifier in each row.

 

I read in all the data after the identifier as a long string.

 

I used dataset copy to create versions of the data based on the identifier, and SELECT IF to limit each dataset to one record layout.

 

I then used NUMERIC and STRING to define the record layout, and a series of commands like:

 

COMPUTE var_nbr = NUMBER(SUBSTR(longstring,1,4),F8.0)

COMPUTE var_str = SUBSTR(longstring,5,10)

 

Does this help as an approach? It might be cumbersome if you have a lot of variables to parse, but all the computes can be run on the same pass.

 

Jim Marks

Director, Market Research

x1616

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Zachary Feinstein
Sent: Wednesday, March 31, 2010 4:58 PM
To: [hidden email]
Subject: DATA LIST FILE Command

 

I have to read in some very large sets of data.  Currently I am doing something like the following:

 

DATA LIST FILE = xyz RECORDS = 4

 /resp = 1- 4

 /

 /mostoften 7 - 9

 /current1 TO current31 7 - 99

...

And so on.

 

The text files that have the data are arranged by card.  Columns 5 - 6 let me know the exact card for each respondent.  So, in the above example, a 01, 02, 03, or 04 would appear in columns 5 - 6.

 

Is there a way in SPSS for me to read the card number and let DATA LIST FILE know to put the relevant card information in for each respondent?  I greatly oversimplified the issue I am encountering here and wish to remove any thought that goes into the ordering of the subcommands above.

 

Thank you.

 

Zachary

 

 

Reply | Threaded
Open this post in threaded view
|

Re: DATA LIST FILE Command

Jon K Peck
In reply to this post by Zachary Feinstein

From your description, this sounds like a case for
FILE TYPE MIXED.  That allows you to specify different layouts for input records based on a record type id.
Here is an example from the CSR.
FILE TYPE MIXED FILE='/data/treatmnt.txt' RECORD=RECID 1-2.
+ RECORD TYPE 21,22,23,24.
+ DATA LIST /SEX 5 AGE 6-7 DOSAGE 8-10 RESULT 12.
+ RECORD TYPE 25.
+ DATA LIST /SEX 5 AGE 6-7 DOSAGE 10-12 RESULT 15.
END FILE TYPE.

If your scheme doesn't fit this model, you can always write an INPUT PROGRAM to build the cases

HTH,

Jon Peck
SPSS, an IBM Company
[hidden email]
312-651-3435



From: Zachary Feinstein <[hidden email]>
To: [hidden email]
Date: 03/31/2010 04:01 PM
Subject: [SPSSX-L] DATA LIST FILE Command
Sent by: "SPSSX(r) Discussion" <[hidden email]>





I have to read in some very large sets of data.  Currently I am doing something like the following:
 
DATA LIST FILE = xyz RECORDS = 4
 /resp = 1- 4
 /
 /mostoften 7 - 9
 /current1 TO current31 7 - 99
...
And so on.
 
The text files that have the data are arranged by card.  Columns 5 - 6 let me know the exact card for each respondent.  So, in the above example, a 01, 02, 03, or 04 would appear in columns 5 - 6.
 
Is there a way in SPSS for me to read the card number and let DATA LIST FILE know to put the relevant card information in for each respondent?  I greatly oversimplified the issue I am encountering here and wish to remove any thought that goes into the ordering of the subcommands above.
 
Thank you.
 
Zachary
zsfeinstein@...
 


Reply | Threaded
Open this post in threaded view
|

Re: DATA LIST FILE Command

Bruce Weaver
Administrator
Here are a couple more examples using FILE TYPE MIXED.

  www.angelfire.com/wv/bwhomedir/spss/read_data_from_mixed_file.txt

Bruce

Jon K Peck wrote
From your description, this sounds like a case for
FILE TYPE MIXED.  That allows you to specify different layouts for input
records based on a record type id.
Here is an example from the CSR.
FILE TYPE MIXED FILE='/data/treatmnt.txt' RECORD=RECID 1-2.
+ RECORD TYPE 21,22,23,24.
+ DATA LIST /SEX 5 AGE 6-7 DOSAGE 8-10 RESULT 12.
+ RECORD TYPE 25.
+ DATA LIST /SEX 5 AGE 6-7 DOSAGE 10-12 RESULT 15.
END FILE TYPE.

If your scheme doesn't fit this model, you can always write an INPUT
PROGRAM to build the cases

HTH,

Jon Peck
SPSS, an IBM Company
peck@us.ibm.com
312-651-3435



From:
Zachary Feinstein <zsfeinstein@yahoo.com>
To:
SPSSX-L@LISTSERV.UGA.EDU
Date:
03/31/2010 04:01 PM
Subject:
[SPSSX-L] DATA LIST FILE Command
Sent by:
"SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>



I have to read in some very large sets of data.  Currently I am doing
something like the following:

DATA LIST FILE = xyz RECORDS = 4
 /resp = 1- 4
 /
 /mostoften 7 - 9
 /current1 TO current31 7 - 99
...
And so on.

The text files that have the data are arranged by card.  Columns 5 - 6 let
me know the exact card for each respondent.  So, in the above example, a
01, 02, 03, or 04 would appear in columns 5 - 6.

Is there a way in SPSS for me to read the card number and let DATA LIST
FILE know to put the relevant card information in for each respondent?  I
greatly oversimplified the issue I am encountering here and wish to remove
any thought that goes into the ordering of the subcommands above.

Thank you.

Zachary
zsfeinstein@yahoo.com


--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

PLEASE NOTE THE FOLLOWING: 
1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above.
2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/).
Reply | Threaded
Open this post in threaded view
|

Re: DATA LIST FILE Command

John F Hall
In reply to this post by Zachary Feinstein
Zachary
 
It's not exactly clear from your message what your data actually look like, but if it's a straightforward rectangular matrix (ie 80-column Hollerith cards) there's a long step-by-step tutorial using DATA LIST on:
 
 
(item 1.3.3) which takes you from raw data to SPSS saved file.  Line by line syntax with screenshots at each step of syntax file and data editor.
 
This example is for one card per case (24 variables) but there's another tutorial using data from the British Social Attitudes Survey  (23 records per case) on:
 
 
This only extracts 2 variables, but the logic is very clear and can be extended to your own data.  I can send you the syntax for all 900+ variables if you like!
 
It shouldn't really matter if your cards are of different lengths.  If you get stuck, send me an extract from your data so I can better help you out.
 
John Hall
 
 
 
 
 
 
 
 
Sent: Wednesday, March 31, 2010 11:57 PM
Subject: DATA LIST FILE Command

I have to read in some very large sets of data.  Currently I am doing something like the following:
 
DATA LIST FILE = xyz RECORDS = 4
 /resp = 1- 4
 /
 /mostoften 7 - 9
 /current1 TO current31 7 - 99
...
And so on.
 
The text files that have the data are arranged by card.  Columns 5 - 6 let me know the exact card for each respondent.  So, in the above example, a 01, 02, 03, or 04 would appear in columns 5 - 6.
 
Is there a way in SPSS for me to read the card number and let DATA LIST FILE know to put the relevant card information in for each respondent?  I greatly oversimplified the issue I am encountering here and wish to remove any thought that goes into the ordering of the subcommands above.
 
Thank you.
 
Zachary