ver. 17 problem reading in decimals
Locked
ver. 17 problem reading in decimals
 
|
Good Morning Listers, Have a condition not encountered before,
and wonder if anyone else has – my data, a tab delimited text file has
several numeric fields with about 12 decimals. When reading in these data if I set the
decimals value (Fx.x) to over 7 I get errors. Setting the value to 7 or less
works. Is this a glitch or
intentional, and if intentional, how can I get around it? TIA Mike |
Re: ver. 17 problem reading in decimals
|
|
What kind of errors? What does the syntax
look like? I can read a tab-delimited file that contains values with 12
decimals without any problems. From: SPSSX(r)
Discussion [mailto:[hidden email]] On
Behalf Of Roberts, Michael Good Morning Listers, Have a condition not encountered before,
and wonder if anyone else has – my data, a tab delimited text file has
several numeric fields with about 12 decimals. When reading in these data
if I set the decimals value (Fx.x) to over 7 I get errors. Setting the
value to 7 or less works. Is this a glitch or intentional, and if intentional,
how can I get around it? TIA Mike |
Re: ver. 17 problem reading in decimals
|
|
Hi Richard, I have run a few tests varying the number of decimals and the
following is what seems to be the case: Up to 16 decimals there are no problems; I varied the decimals
from F20.19 through F20.15, and at F20.16 there were no errors. Syntax is generated by the dialogue box
(wanted to see what SPSS was seeing) – the bit with the relevant
variables is included: “... GET DATA /TYPE=TXT /FILE='<<filepath>>’ /DELCASE=LINE
/DELIMITERS=","
/ARRANGEMENT=DELIMITED /FIRSTCASE=2 /IMPORTCASE=FIRST
9000 /VARIABLES= . . . on_risk A1 demoscore F21.19 demofact F1.0 rxscore F20.17 rxfact F1.0 totscore F21.19 adjscore F21.19 CACHE. EXECUTE. DATASET NAME DataSet3 WINDOW=FRONT. “ The error I get
when running the code as is – “ Error. Command name:
GET DATA (2265) Unrecognized or invalid variable definition. This command not executed. CACHE. EXECUTE. Error # 105. Command
name: EXECUTE This command is not valid before a working file has been defined. This command not executed. DATASET NAME DataSet3 WINDOW=FRONT.” When changing the decimal value to 16 (F21.16), the file reads in OK. Is this a documented limitation of the
decimals, or just a bug? TIA Mike From: SPSSX(r) Discussion [mailto:[hidden email]] On
Behalf Of Oliver, Richard What kind of errors? What does the
syntax look like? I can read a tab-delimited file that contains values with 12
decimals without any problems. From: SPSSX(r) Discussion
[mailto:[hidden email]] On Behalf Of Roberts, Michael Good Morning Listers, Have a condition not encountered before,
and wonder if anyone else has – my data, a tab delimited text file has
several numeric fields with about 12 decimals. When reading in these data
if I set the decimals value (Fx.x) to over 7 I get errors. Setting the
value to 7 or less works. Is this a glitch or intentional, and if intentional,
how can I get around it? TIA Mike |
Re: ver. 17 problem reading in decimals
|
|
Right. There was a known issue with the
Text Wizard, which I think has been resolved for the next release. You cannot
specify a format value that contains more than 16 decimal positions, but the
Text Wizard was attempting to do so. Keep in mind that double-precision
floating point values are basically limited to 16 digits of precision. So if
you really have 21-digit values with 19 decimals, these values cannot be represented
precisely in SPSS (except as strings). All the folks out there who actually know
something about computers will correct me if I’m wrong. From: Roberts, Michael
[mailto:[hidden email]] Hi
Richard, I have run
a few tests varying the number of decimals and the following is what seems to
be the case: Up to 16
decimals there are no problems; I varied the decimals from F20.19 through
F20.15, and at F20.16 there were no errors. Syntax is generated by the
dialogue box (wanted to see what SPSS was seeing) – the bit with the
relevant variables is included: “... GET DATA
/TYPE=TXT
/FILE='<<filepath>>’
/DELCASE=LINE
/DELIMITERS=","
/ARRANGEMENT=DELIMITED
/FIRSTCASE=2
/IMPORTCASE=FIRST 9000
/VARIABLES= . . .
on_risk A1 demoscore
F21.19
demofact F1.0
rxscore F20.17
rxfact F1.0
totscore F21.19
adjscore F21.19 CACHE. EXECUTE. DATASET
NAME DataSet3 WINDOW=FRONT. “ The error I get when running the code as is – “ Error.
Command name: GET DATA (2265) Unrecognized or invalid variable definition. This command not executed. CACHE. EXECUTE. Error # 105. Command name: EXECUTE This command is not valid before a working file has been defined. This command not executed. DATASET NAME DataSet3 WINDOW=FRONT.” When changing the decimal value to 16 (F21.16), the file reads in
OK. Is this a documented limitation of the decimals, or just a bug? TIA Mike From: SPSSX(r)
Discussion [mailto:[hidden email]] On
Behalf Of Oliver, Richard What kind of errors? What
does the syntax look like? I can read a tab-delimited file that contains values
with 12 decimals without any problems. From: SPSSX(r)
Discussion [mailto:[hidden email]] On
Behalf Of Roberts, Michael Good Morning Listers, Have a condition not
encountered before, and wonder if anyone else has – my data, a tab
delimited text file has several numeric fields with about 12 decimals.
When reading in these data if I set the decimals value (Fx.x) to over 7 I get
errors. Setting the value to 7 or less works. Is this a glitch or
intentional, and if intentional, how can I get around it? TIA Mike |
Re: ver. 17 problem reading in decimals
|
|
Richard, Thank you for your help with this. Do you know whether SAS would be subject
to a similar limitation? The
problem was originally discovered with that app. Sincerely Mike From: Oliver, Richard [mailto:[hidden email]] Right. There was a known issue
with the Text Wizard, which I think has been resolved for the next release. You
cannot specify a format value that contains more than 16 decimal positions, but
the Text Wizard was attempting to do so. Keep in mind that double-precision
floating point values are basically limited to 16 digits of precision. So if
you really have 21-digit values with 19 decimals, these values cannot be
represented precisely in SPSS (except as strings). All the folks out there who
actually know something about computers will correct me if I’m wrong. From: Roberts, Michael
[mailto:[hidden email]] Hi Richard, I have run a few tests varying the number of decimals and the
following is what seems to be the case: Up to 16 decimals there are no problems; I varied the decimals
from F20.19 through F20.15, and at F20.16 there were no errors. Syntax is
generated by the dialogue box (wanted to see what SPSS was seeing) – the
bit with the relevant variables is included: “... GET DATA /TYPE=TXT /FILE='<<filepath>>’ /DELCASE=LINE /DELIMITERS="," /ARRANGEMENT=DELIMITED /FIRSTCASE=2 /IMPORTCASE=FIRST 9000 /VARIABLES= . . . on_risk A1 demoscore F21.19 demofact F1.0 rxscore F20.17 rxfact F1.0 totscore F21.19 adjscore F21.19 CACHE. EXECUTE. DATASET NAME DataSet3 WINDOW=FRONT. “ The error I get
when running the code as is – “ Error.
Command name: GET DATA (2265)
Unrecognized or invalid variable definition. This command not
executed. CACHE. EXECUTE. Error #
105. Command name: EXECUTE This command is
not valid before a working file has been defined. This command not
executed. DATASET NAME
DataSet3 WINDOW=FRONT.” When changing
the decimal value to 16 (F21.16), the file reads in OK. Is this a
documented limitation of the decimals, or just a bug? TIA Mike From: SPSSX(r) Discussion
[mailto:[hidden email]] On Behalf Of Oliver, Richard What kind of errors? What does the
syntax look like? I can read a tab-delimited file that contains values with 12
decimals without any problems. From: SPSSX(r) Discussion
[mailto:[hidden email]] On Behalf Of Roberts, Michael Good Morning Listers, Have a condition not encountered before,
and wonder if anyone else has – my data, a tab delimited text file has
several numeric fields with about 12 decimals. When reading in these data
if I set the decimals value (Fx.x) to over 7 I get errors. Setting the
value to 7 or less works. Is this a glitch or intentional, and if
intentional, how can I get around it? TIA Mike |
Re: ver. 17 problem reading in decimals
|
|
In reply to this post by Oliver, Richard
Richard, One more question – my original uncompressed text file was
about 5gb. After I saved it as a
.SAV file it grew to 10gb. Is this
normal? Thanking You Sincerely Mike From: Oliver, Richard [mailto:[hidden email]] Right. There was a known issue
with the Text Wizard, which I think has been resolved for the next release. You
cannot specify a format value that contains more than 16 decimal positions, but
the Text Wizard was attempting to do so. Keep in mind that double-precision
floating point values are basically limited to 16 digits of precision. So if
you really have 21-digit values with 19 decimals, these values cannot be
represented precisely in SPSS (except as strings). All the folks out there who actually
know something about computers will correct me if I’m wrong. From: Roberts, Michael
[mailto:[hidden email]] Hi Richard, I have run a few tests varying the number of decimals and the
following is what seems to be the case: Up to 16 decimals there are no problems; I varied the decimals
from F20.19 through F20.15, and at F20.16 there were no errors. Syntax is
generated by the dialogue box (wanted to see what SPSS was seeing) – the
bit with the relevant variables is included: “... GET DATA /TYPE=TXT /FILE='<<filepath>>’ /DELCASE=LINE /DELIMITERS="," /ARRANGEMENT=DELIMITED /FIRSTCASE=2 /IMPORTCASE=FIRST 9000 /VARIABLES= . . . on_risk A1 demoscore F21.19 demofact F1.0 rxscore F20.17 rxfact F1.0 totscore F21.19 adjscore F21.19 CACHE. EXECUTE. DATASET NAME DataSet3 WINDOW=FRONT. “ The error I get
when running the code as is – “ Error.
Command name: GET DATA (2265)
Unrecognized or invalid variable definition. This command not
executed. CACHE. EXECUTE. Error #
105. Command name: EXECUTE This command is
not valid before a working file has been defined. This command not
executed. DATASET NAME
DataSet3 WINDOW=FRONT.” When changing
the decimal value to 16 (F21.16), the file reads in OK. Is this a
documented limitation of the decimals, or just a bug? TIA Mike From: SPSSX(r) Discussion
[mailto:[hidden email]] On Behalf Of Oliver, Richard What kind of errors? What does the
syntax look like? I can read a tab-delimited file that contains values with 12
decimals without any problems. From: SPSSX(r) Discussion [mailto:[hidden email]]
On Behalf Of Roberts, Michael Good Morning Listers, Have a condition not encountered before,
and wonder if anyone else has – my data, a tab delimited text file has
several numeric fields with about 12 decimals. When reading in these data
if I set the decimals value (Fx.x) to over 7 I get errors. Setting the
value to 7 or less works. Is this a glitch or intentional, and if intentional,
how can I get around it? TIA Mike |
Re: ver. 17 problem reading in decimals
|
|
You have a 5 GB text file from which read
only the first 9,000 cases, and you end up with a 10 GB SAV file? From: SPSSX(r)
Discussion [mailto:[hidden email]] On
Behalf Of Roberts, Michael Richard, One more
question – my original uncompressed text file was about 5gb. After
I saved it as a .SAV file it grew to 10gb. Is this normal? Thanking
You Sincerely Mike From: Oliver,
Richard [mailto:[hidden email]] Right. There was a known
issue with the Text Wizard, which I think has been resolved for the next
release. You cannot specify a format value that contains more than 16 decimal
positions, but the Text Wizard was attempting to do so. Keep in mind that
double-precision floating point values are basically limited to 16 digits of
precision. So if you really have 21-digit values with 19 decimals, these values
cannot be represented precisely in SPSS (except as strings). All the folks out there
who actually know something about computers will correct me if I’m wrong. From: Roberts,
Michael [mailto:[hidden email]] Hi Richard, I have run a few tests varying the number of decimals and the
following is what seems to be the case: Up to 16 decimals there are no problems; I varied the decimals
from F20.19 through F20.15, and at F20.16 there were no errors. Syntax is
generated by the dialogue box (wanted to see what SPSS was seeing) – the
bit with the relevant variables is included: “... GET DATA /TYPE=TXT /FILE='<<filepath>>’ /DELCASE=LINE /DELIMITERS="," /ARRANGEMENT=DELIMITED /FIRSTCASE=2 /IMPORTCASE=FIRST 9000 /VARIABLES= . . . on_risk A1 demoscore F21.19 demofact F1.0 rxscore F20.17 rxfact F1.0 totscore F21.19 adjscore F21.19 CACHE. EXECUTE. DATASET NAME DataSet3 WINDOW=FRONT. “ The error I get when running the
code as is – “ Error. Command name: GET DATA (2265) Unrecognized or invalid variable definition. This command not executed. CACHE. EXECUTE. Error # 105. Command name: EXECUTE This command is not valid before a working file has been defined. This command not executed. DATASET NAME DataSet3 WINDOW=FRONT.” When changing the decimal value to 16 (F21.16), the file reads in
OK. Is this a documented limitation of the decimals, or just a bug? TIA Mike From: SPSSX(r)
Discussion [mailto:[hidden email]] On
Behalf Of Oliver, Richard What
kind of errors? What does the syntax look like? I can read a tab-delimited file
that contains values with 12 decimals without any problems. From: SPSSX(r)
Discussion [mailto:[hidden email]] On
Behalf Of Roberts, Michael Good Morning Listers, Have a condition not
encountered before, and wonder if anyone else has – my data, a tab
delimited text file has several numeric fields with about 12 decimals.
When reading in these data if I set the decimals value (Fx.x) to over 7 I get
errors. Setting the value to 7 or less works. Is this a glitch or
intentional, and if intentional, how can I get around it? TIA Mike |
Re: ver. 17 problem reading in decimals
|
|
In reply to this post by Roberts, Michael
If SAS stores numbers as double-precision
floating point values, then you should have issues with values that exceed 16
digits of precision. From: Roberts, Michael
[mailto:[hidden email]] Richard, Thank you
for your help with this. Do you know whether SAS would be subject to a
similar limitation? The problem was originally discovered with that app. Sincerely Mike
From: Oliver,
Richard [mailto:[hidden email]] Right. There was a known
issue with the Text Wizard, which I think has been resolved for the next
release. You cannot specify a format value that contains more than 16 decimal
positions, but the Text Wizard was attempting to do so. Keep in mind that double-precision
floating point values are basically limited to 16 digits of precision. So if
you really have 21-digit values with 19 decimals, these values cannot be
represented precisely in SPSS (except as strings). All the folks out there
who actually know something about computers will correct me if I’m wrong. From: Roberts,
Michael [mailto:[hidden email]] Hi Richard, I have run a few tests varying the number of decimals and the
following is what seems to be the case: Up to 16 decimals there are no problems; I varied the decimals
from F20.19 through F20.15, and at F20.16 there were no errors. Syntax is
generated by the dialogue box (wanted to see what SPSS was seeing) – the
bit with the relevant variables is included: “... GET DATA /TYPE=TXT /FILE='<<filepath>>’ /DELCASE=LINE /DELIMITERS="," /ARRANGEMENT=DELIMITED /FIRSTCASE=2 /IMPORTCASE=FIRST 9000 /VARIABLES= . . . on_risk A1 demoscore F21.19 demofact F1.0 rxscore F20.17 rxfact F1.0 totscore F21.19 adjscore F21.19 CACHE. EXECUTE. DATASET NAME DataSet3 WINDOW=FRONT. “ The error I get when running the
code as is – “ Error. Command name: GET DATA (2265) Unrecognized or invalid variable definition. This command not executed. CACHE. EXECUTE. Error # 105. Command name: EXECUTE This command is not valid before a working file has been defined. This command not executed. DATASET NAME DataSet3 WINDOW=FRONT.” When changing the decimal value to 16 (F21.16), the file reads in
OK. Is this a documented limitation of the decimals, or just a bug? TIA Mike From: SPSSX(r)
Discussion [mailto:[hidden email]] On
Behalf Of Oliver, Richard What
kind of errors? What does the syntax look like? I can read a tab-delimited file
that contains values with 12 decimals without any problems. From: SPSSX(r)
Discussion [mailto:[hidden email]] On
Behalf Of Roberts, Michael Good Morning Listers, Have a condition not
encountered before, and wonder if anyone else has – my data, a tab
delimited text file has several numeric fields with about 12 decimals.
When reading in these data if I set the decimals value (Fx.x) to over 7 I get
errors. Setting the value to 7 or less works. Is this a glitch or
intentional, and if intentional, how can I get around it? TIA Mike |
Re: ver. 17 problem reading in decimals
|
|
Thank you again, Richard. This resolves our mystery! Regards Mike From: Oliver, Richard [mailto:[hidden email]] If SAS stores numbers as
double-precision floating point values, then you should have issues with values
that exceed 16 digits of precision. From: Roberts, Michael
[mailto:[hidden email]] Richard, Thank you for your help with this. Do you know whether SAS
would be subject to a similar limitation? The problem was originally
discovered with that app. Sincerely Mike From: Oliver, Richard [mailto:[hidden email]] Right. There was a known issue
with the Text Wizard, which I think has been resolved for the next release. You
cannot specify a format value that contains more than 16 decimal positions, but
the Text Wizard was attempting to do so. Keep in mind that double-precision
floating point values are basically limited to 16 digits of precision. So if
you really have 21-digit values with 19 decimals, these values cannot be represented
precisely in SPSS (except as strings). All the folks out there who
actually know something about computers will correct me if I’m wrong. From: Roberts, Michael
[mailto:[hidden email]] Hi Richard, I have run a few tests varying the number of decimals and the
following is what seems to be the case: Up to 16 decimals there are no problems; I varied the decimals
from F20.19 through F20.15, and at F20.16 there were no errors. Syntax is
generated by the dialogue box (wanted to see what SPSS was seeing) – the
bit with the relevant variables is included: “... GET DATA /TYPE=TXT /FILE='<<filepath>>’ /DELCASE=LINE /DELIMITERS="," /ARRANGEMENT=DELIMITED /FIRSTCASE=2 /IMPORTCASE=FIRST 9000 /VARIABLES= . . . on_risk A1 demoscore F21.19 demofact F1.0 rxscore F20.17 rxfact F1.0 totscore F21.19 adjscore F21.19 CACHE. EXECUTE. DATASET NAME DataSet3 WINDOW=FRONT. “ The error I get
when running the code as is – “ Error. Command
name: GET DATA (2265)
Unrecognized or invalid variable definition. This command not
executed. CACHE. EXECUTE. Error #
105. Command name: EXECUTE This command is
not valid before a working file has been defined. This command not
executed. DATASET NAME
DataSet3 WINDOW=FRONT.” When changing
the decimal value to 16 (F21.16), the file reads in OK. Is this a
documented limitation of the decimals, or just a bug? TIA Mike From: SPSSX(r) Discussion
[mailto:[hidden email]] On Behalf Of Oliver, Richard What kind of errors? What does the
syntax look like? I can read a tab-delimited file that contains values with 12
decimals without any problems. From: SPSSX(r) Discussion
[mailto:[hidden email]] On Behalf Of Roberts, Michael Good Morning Listers, Have a condition not encountered before,
and wonder if anyone else has – my data, a tab delimited text file has
several numeric fields with about 12 decimals. When reading in these data
if I set the decimals value (Fx.x) to over 7 I get errors. Setting the
value to 7 or less works. Is this a glitch or intentional, and if
intentional, how can I get around it? TIA Mike |
Re: ver. 17 problem reading in decimals
|
In reply to this post by Roberts, Michael
At 03:29 PM 5/21/2009, Oliver, Richard wrote:
>Double-precision floating point values are basically limited to 16 >digits of precision. So if you really have 21-digit values with 19 >decimals, these values cannot be represented precisely in SPSS At 03:33 PM 5/21/2009, Roberts, Michael wrote: >Do you know whether SAS would be subject to a similar limitation? This comes up from time to time. SPSS, SAS, and many other modern systems use what are called "IEEE-754 64-bit numbers"(*); SAS's limitations will be exactly the same as SPSS's. It is commonly stated that the precision is 16 digits. It's actually 53 bits, which is close but not quite identical. The 53 bits can represent all integers from 0 through 9,007,199,254,740,992; that is, all 15-digit integers and most, BUT NOT ALL, 16-digit integers. As Richard Oliver wrote, numbers with 17 or more decimal digits can be represented only to a close approximation. ........................ (*)IEEE is the Institute of Electrical and Electronic Engineers. 'IEEE 754 numbers' are numbers in a format described in 'IEEE 754', i.e the 754th standard in the IEEE's ongoing program of developing relevant standards. IEEE 754 is widely used, first, because it's a pretty good standard. And second, most modern computers have hardware to do arithmetic with IEEE 754 numbers, so calculations are fast and easy to program. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Re: ver. 17 problem reading in decimals
|
|
Hi Richard,
Thank you very much for the clear explanation. Not sure why our data has these decimals, but this is a good occasion to revisit the data! Sincerely Mike -----Original Message----- From: Richard Ristow [mailto:[hidden email]] Sent: Friday, May 22, 2009 6:22 PM To: Roberts, Michael; [hidden email] Cc: Oliver, Richard Subject: Re: ver. 17 problem reading in decimals At 03:29 PM 5/21/2009, Oliver, Richard wrote: >Double-precision floating point values are basically limited to 16 >digits of precision. So if you really have 21-digit values with 19 >decimals, these values cannot be represented precisely in SPSS At 03:33 PM 5/21/2009, Roberts, Michael wrote: >Do you know whether SAS would be subject to a similar limitation? This comes up from time to time. SPSS, SAS, and many other modern systems use what are called "IEEE-754 64-bit numbers"(*); SAS's limitations will be exactly the same as SPSS's. It is commonly stated that the precision is 16 digits. It's actually 53 bits, which is close but not quite identical. The 53 bits can represent all integers from 0 through 9,007,199,254,740,992; that is, all 15-digit integers and most, BUT NOT ALL, 16-digit integers. As Richard Oliver wrote, numbers with 17 or more decimal digits can be represented only to a close approximation. ........................ (*)IEEE is the Institute of Electrical and Electronic Engineers. 'IEEE 754 numbers' are numbers in a format described in 'IEEE 754', i.e the 754th standard in the IEEE's ongoing program of developing relevant standards. IEEE 754 is widely used, first, because it's a pretty good standard. And second, most modern computers have hardware to do arithmetic with IEEE 754 numbers, so calculations are fast and easy to program. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
| Free forum by Nabble | Edit this page |