How can I rea a SPSSX file from a IBM 360 file system format

[Julio-21]

Dear List Members

I just recover from an old tape, a series of SPPSSX dat files.

The files are written in EBCDIC in an IBM 360 file system format. The
internal file headers begin with "FL2SPSS  SYSTEM  FILE  IBM
360  OS SPSS RELEASE 1". I need to convert this files to a readable format.

Can somebody help me out

Best

Julio

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Richard Ristow]

At 01:42 AM 5/10/2008, Julio wrote:

>I just recover from an old tape, a series of SPPSSX dat files.
>
>The files are written in EBCDIC in an IBM 360 file system format.
>The internal file headers begin with "FL2SPSS  SYSTEM  FILE  IBM
>360  OS SPSS RELEASE 1".

I assume these really are SPSS-X, not earlier SPSS; the first Release
1 must have been around 1970; SPSS-X, in the early 1980s

>I need to convert this files to a readable format.

You could try loading them as disk files and trying to read them with
GET FILE. I don't know how far back SPSS is cognizant of its own file
formats, but this might work. Don't get your hopes too high, though.

Or, you may have to use low-level techniques. How do you have the
files now? That is, are they on 9-track tape and you have a tape
reader, or are they disk files in some form? You apparently have some
form of EBCDIC to ASCII conversion, or you couldn't read and print
the header as you have. Unfortunately, if the file's been converted
from EBCDIC to ASCII, that could mess up the numerical values.

And, if the files are on disk, do the records represent (physical)
tape 'blocks' or the files 'logical records'? If you have logical
records, I believe the SPSS system files (usually called '.sav files'
now) included header (data-dictionary) records and data records; and
all data records were the same length. Remove the header lines, and
you should be able to read the data lines with DATA LIST. Numeric
values were in System/360 single-precision floating point
('hexfloat'), and should be readable with format RB4. As I said,
numeric values could be a problem if the file's been converted to
ASCII; and string values (including variable names) could be
difficult, if it hasn't.

If your disk records correspond to (physical) tape blocks, you'll
have to separate out the logical records; but I won't go into that,
unless you need it.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Oliver, Richard]

Look at the FILE HANDLE command.

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Richard Ristow
Sent: Wednesday, May 14, 2008 8:17 AM
To: [hidden email]
Subject: Re: How can I rea a SPSSX file from a IBM 360 file system format

At 01:42 AM 5/10/2008, Julio wrote:

>I just recover from an old tape, a series of SPPSSX dat files.
>
>The files are written in EBCDIC in an IBM 360 file system format.
>The internal file headers begin with "FL2SPSS  SYSTEM  FILE  IBM
>360  OS SPSS RELEASE 1".

I assume these really are SPSS-X, not earlier SPSS; the first Release
1 must have been around 1970; SPSS-X, in the early 1980s

>I need to convert this files to a readable format.

You could try loading them as disk files and trying to read them with
GET FILE. I don't know how far back SPSS is cognizant of its own file
formats, but this might work. Don't get your hopes too high, though.

Or, you may have to use low-level techniques. How do you have the
files now? That is, are they on 9-track tape and you have a tape
reader, or are they disk files in some form? You apparently have some
form of EBCDIC to ASCII conversion, or you couldn't read and print
the header as you have. Unfortunately, if the file's been converted
from EBCDIC to ASCII, that could mess up the numerical values.

And, if the files are on disk, do the records represent (physical)
tape 'blocks' or the files 'logical records'? If you have logical
records, I believe the SPSS system files (usually called '.sav files'
now) included header (data-dictionary) records and data records; and
all data records were the same length. Remove the header lines, and
you should be able to read the data lines with DATA LIST. Numeric
values were in System/360 single-precision floating point
('hexfloat'), and should be readable with format RB4. As I said,
numeric values could be a problem if the file's been converted to
ASCII; and string values (including variable names) could be
difficult, if it hasn't.

If your disk records correspond to (physical) tape blocks, you'll
have to separate out the logical records; but I won't go into that,
unless you need it.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Richard Ristow]

At 12:27 PM 5/14/2008, Oliver, Richard wrote:

>Look at the FILE HANDLE command.

Thank you! Excellent advice. I hadn't kept conscious of all that you
can do with FILE HANDLE.

It looks like you'd use MODE=360 RECFORM=VARIABLE LRECL=??????.

I assume you'd still have to strip off, or ignore garbage output
from, the file header records.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Julio-21]

To both Richard, thanks a lot for your helpful comments.
As I understand from your comments, in principle the file "as is" can be
read directly and easily in the windows version of SPPS  with a code
similar to the following:

FILE HANDLE TEST / NAME='file0001.dat'/ MODE=360 /RECFORM=FIXED / LRECL=??.
DATA LIST FILE=TEST /   VAR1 (Fx.0) VAR2 (Fx.0) ... VARn (Fx.0).
LIST.

I am afraid however that I do not know the format of the variables
included in the tape!

Does this mean I cannot recover the data... or is something that I can do?

Thanks again for your help!!!

Julio

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Oliver, Richard]

First, unless you think the record (row) length is more than 8,192 characters, you don't need the LRECL subcommand.

Second, you probably need a fully qualified path as part of the Name specification.

Finally, you could read the data into SPSS as one big string, writing it out as a new text file, and then use the Text Wizard to help you read the new text file:

FILE HANDLE TEST / NAME='c:\filepath\file0001.dat'/ MODE=360 /RECFORM=FIXED.
DATA LIST FILE=TEST /VAR1 (a8192). /*or whatever length you think is enough.
write outfile='c:\filepath\newdata.txt'
  /var1.
execute.
*Now use the Text Wizard to help you read the data.

And this is one of the cases where you really do need the Execute command.




-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Julio
Sent: Thursday, May 15, 2008 12:55 AM
To: [hidden email]
Subject: Re: How can I rea a SPSSX file from a IBM 360 file system format

To both Richard, thanks a lot for your helpful comments.
As I understand from your comments, in principle the file "as is" can be
read directly and easily in the windows version of SPPS  with a code
similar to the following:

FILE HANDLE TEST / NAME='file0001.dat'/ MODE=360 /RECFORM=FIXED / LRECL=??.
DATA LIST FILE=TEST /   VAR1 (Fx.0) VAR2 (Fx.0) ... VARn (Fx.0).
LIST.

I am afraid however that I do not know the format of the variables
included in the tape!

Does this mean I cannot recover the data... or is something that I can do?

Thanks again for your help!!!

Julio

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Oliver, Richard]

Oops. Wait a minute. I just re-read the original message. If the original data files are binary SPSS data files, DATA LIST won't work.

I don't know if GET FILE with FILE HANDLE will work in this case, but you could try:

FILE HANDLE TEST / NAME='c:\filepath\file0001.dat'/ MODE=360 /RECFORM=FIXED.
GET FILE=TEST.


-----Original Message-----
From: Oliver, Richard
Sent: Thursday, May 15, 2008 9:46 AM
To: 'Julio'; [hidden email]
Subject: RE: Re: How can I rea a SPSSX file from a IBM 360 file system format

First, unless you think the record (row) length is more than 8,192 characters, you don't need the LRECL subcommand.

Second, you probably need a fully qualified path as part of the Name specification.

Finally, you could read the data into SPSS as one big string, writing it out as a new text file, and then use the Text Wizard to help you read the new text file:

FILE HANDLE TEST / NAME='c:\filepath\file0001.dat'/ MODE=360 /RECFORM=FIXED.
DATA LIST FILE=TEST /VAR1 (a8192). /*or whatever length you think is enough.
write outfile='c:\filepath\newdata.txt'
  /var1.
execute.
*Now use the Text Wizard to help you read the data.

And this is one of the cases where you really do need the Execute command.




-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Julio
Sent: Thursday, May 15, 2008 12:55 AM
To: [hidden email]
Subject: Re: How can I rea a SPSSX file from a IBM 360 file system format

To both Richard, thanks a lot for your helpful comments.
As I understand from your comments, in principle the file "as is" can be
read directly and easily in the windows version of SPPS  with a code
similar to the following:

FILE HANDLE TEST / NAME='file0001.dat'/ MODE=360 /RECFORM=FIXED / LRECL=??.
DATA LIST FILE=TEST /   VAR1 (Fx.0) VAR2 (Fx.0) ... VARn (Fx.0).
LIST.

I am afraid however that I do not know the format of the variables
included in the tape!

Does this mean I cannot recover the data... or is something that I can do?

Thanks again for your help!!!

Julio

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Oliver, Richard]

Wait another minute. My brain isn't firing on all cylinders. Have you tried just using GET FILE to read the data? If the data files really are binary SPSS data files, the advice I provided on using FILE HANDLE doens't make any sense.

-----Original Message-----
From: Oliver, Richard
Sent: Thursday, May 15, 2008 9:50 AM
To: 'Julio'; '[hidden email]'
Subject: RE: Re: How can I rea a SPSSX file from a IBM 360 file system format

Oops. Wait a minute. I just re-read the original message. If the original data files are binary SPSS data files, DATA LIST won't work.

I don't know if GET FILE with FILE HANDLE will work in this case, but you could try:

FILE HANDLE TEST / NAME='c:\filepath\file0001.dat'/ MODE=360 /RECFORM=FIXED.
GET FILE=TEST.


-----Original Message-----
From: Oliver, Richard
Sent: Thursday, May 15, 2008 9:46 AM
To: 'Julio'; [hidden email]
Subject: RE: Re: How can I rea a SPSSX file from a IBM 360 file system format

First, unless you think the record (row) length is more than 8,192 characters, you don't need the LRECL subcommand.

Second, you probably need a fully qualified path as part of the Name specification.

Finally, you could read the data into SPSS as one big string, writing it out as a new text file, and then use the Text Wizard to help you read the new text file:

FILE HANDLE TEST / NAME='c:\filepath\file0001.dat'/ MODE=360 /RECFORM=FIXED.
DATA LIST FILE=TEST /VAR1 (a8192). /*or whatever length you think is enough.
write outfile='c:\filepath\newdata.txt'
  /var1.
execute.
*Now use the Text Wizard to help you read the data.

And this is one of the cases where you really do need the Execute command.




-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Julio
Sent: Thursday, May 15, 2008 12:55 AM
To: [hidden email]
Subject: Re: How can I rea a SPSSX file from a IBM 360 file system format

To both Richard, thanks a lot for your helpful comments.
As I understand from your comments, in principle the file "as is" can be
read directly and easily in the windows version of SPPS  with a code
similar to the following:

FILE HANDLE TEST / NAME='file0001.dat'/ MODE=360 /RECFORM=FIXED / LRECL=??.
DATA LIST FILE=TEST /   VAR1 (Fx.0) VAR2 (Fx.0) ... VARn (Fx.0).
LIST.

I am afraid however that I do not know the format of the variables
included in the tape!

Does this mean I cannot recover the data... or is something that I can do?

Thanks again for your help!!!

Julio

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Richard Ristow]

At 10:50 AM 5/15/2008, Oliver, Richard wrote:

>I don't know if GET FILE with FILE HANDLE will work in this case,
>but you could try:
>
>FILE HANDLE TEST / NAME='c:\filepath\file0001.dat'/ MODE=360 /RECFORM=FIXED.
>GET FILE=TEST.

That's definitely the first thing to try. However, I remember pretty
strongly that system files were RECFORM=VARIABLE (unless you know
something I don't, Richard). The data records were all the same
length, but the header data-dictionary records were of several lengths.

Going on (backward in time sequence), at 10:45 AM 5/15/2008, Oliver,
Richard wrote:

>First, unless you think the record (row) length is more than 8,192
>characters, you don't need the LRECL subcommand.

Depends on the number of variables. Data records were 4 bytes per
numeric variable, I suppose the number of bytes rounded up to a
multiple of 4 for string variables, plus 2 bytes overhead.

>Second, you probably need a fully qualified path as part of the Name
>specification.

Yes.

>Finally, you could read the data into SPSS as one big string,
>writing it out as a new text file

Because the numbers are binary, the text wizard wouldn't work, as
Richard wrote later. However, writing the text file still might be
useful to hand-strip the header records before reading the data records.

For reading the records, at 01:55 AM 5/15/2008, Julio asked:

>I am afraid however that I do not know the format of the variables
>included in the tape!

SPSS system files of those days stored all variables as IBM 360
single-precision floating-point numbers. If you inspect them with a
hex editor, the first byte will usually be 40x, or not far from it.
They should be readable with format RB4. Depending on how the file
was read, there MAY be a two-byte length value (format IB2) before
the data values start.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Richard Ristow]

At 11:27 AM 5/15/2008, Oliver, Richard wrote:

>Wait another minute. My brain isn't firing on all cylinders. Have
>you tried just using GET FILE to read the data? If the data files
>really are binary SPSS data files, the advice I provided on using
>FILE HANDLE doesn't make any sense.

Right. The very first suggestion. But don't get your hopes too high.
The format of system/save files has changed a lot since then. This
strategy depends on SPSS's having enough logic in GET FILE to parse
the file headers and deduce which of the 'save' formats the file is in.

(Richard, I'll bet it would be an adventure prowling around the
Development section, or whomever, trying to find someone who knows
GET FILE that well. They'd either growl at you, or be fascinated by
the question.)

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Julio-21]

Dear Richard

I tried "FILE HANDLE"  with both "DATA LIST FILE" and "GET FILE". In both
cases did not work

1.  for "GET FILE" it says "The specified input data file is neither an
SPSS data file nor an SPSS/PC+ data file".

Remember that my file headers begin with "FL2SPSS SYSTEM FILE.  IBM 360
OS, SPSSX RELEASE 1"

Could the problem be with differences between SPSS-X, abd earlier SPSS? I
know that the files were used in the early 80s but I am not totally sure
if they were saved in SPSS-X and not in earlier SPSS Release.

 2. In the case of "FILE HANDLE" combined with "DATA LIST" it did not work
using the RECFORM=FIXED option as SPSS ask me for LRECL.

When I run:

FILE HANDLE TEST/ NAME='c:\file0001.dat'/
MODE=360/RECFORM=FIXED /LRECL=8192.
DATA LIST FILE=TEST /VAR1 (a8192).
write outfile='c:\newdata.txt'
  /var1.
execute.

The SPPSS pocesor collapses due to a catastrophic error. Further a number
of warnings associated to invalid characters encountered in a field read
under an A format. HOWEVER this time the program does produce a TXT file
that shows the following Heading:

FL2SPSS SYSTEM FILE.  IBM 360 OS, SPSSX RELEASE 1

Further, it shows (in the middle of lots of "garbage" the names of the
variables that were originally saved in the SPSS file.


Any ideas about what else I can do??

I do not want to push my luck too far but if you will like to look first
hand to one of my files (the third one has a size of just 56Kb) I can send
it directly to you.

Thanks again for all the help

Best

Julio

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Richard Ristow]

Dear Richard,

I hope it's OK to reply on-list -- this is an on-list discussion.

At 08:39 PM 5/16/2008, Oliver, Richard wrote:

>According to one of the guys who knows way more than I do (Tex Hull)

Is this Ted Hull the Hull who's coauthor of the 'Maroon Manual' and all? Wow!

>, you can't get there from here.

I was afraid that was only too likely. The system file format was
completely changed; retaining backward compatibility would have
meant, essentially, retaining both sets of code, with logic to choose
which. It could have been done, but I sure see why not.

That leaves us with reading the file as a binary file, as we've been
discussing. I still think that should be possible. You'll do a lot
better if you have a hex editor to look at it with. If nothing else,
I'd be willing to look at it, if it's
small enough to E-mail. But not until the end of this week, at the earliest.

-Good luck and good wishes!
  Richard, who's on the road thinking mostly of other things

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Richard Ristow]

At 11:51 AM 5/16/2008, Julio wrote:

>I tried "FILE HANDLE"  with both "DATA LIST FILE" and "GET FILE". In both
>cases did not work
>
>1.  for "GET FILE" it says "The specified input data file is neither an
>SPSS data file nor an SPSS/PC+ data file". Remember that my file
>headers begin with "FL2SPSS SYSTEM FILE.  IBM 360 OS, SPSSX RELEASE 1"

Yeah, I know. It was an off-chance, but no, it won't work. See
Richard Oliver's note, per my reply.

>Could the problem be with differences between SPSS-X, abd earlier SPSS?

Probably not. I think those were backward-compatible.

>  2. In the case of "FILE HANDLE" combined with "DATA LIST" it did not work
>using the RECFORM=FIXED option as SPSS ask me for LRECL.
>
>When I run:
>
>FILE HANDLE TEST/ NAME='c:\file0001.dat'/
>MODE=360/RECFORM=FIXED /LRECL=8192.
>DATA LIST FILE=TEST /VAR1 (a8192).
>write outfile='c:\newdata.txt'
>   /var1.
>execute.
>
>The SPSS pocesor collapses due to a catastrophic error.

I think it's doable, but as I wrote before, I can't look at it this
week. See previous note.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD