I can't believe there's a limit on datafile size - bug in input dialogue v 18.1

[mcunix]

Hi List

 

Long time reader, 1^st time poster.

 

SPSS 18.1, XP 32bit:

 

I have a 4.5 million row, 5-field, csv text file. I cannot read it in (file>open>data). It doesn’t show any errors but goes right into the input window dialogue.

 

It looks like it’s trying to input binary data, but it never gives me any indication it cannot do this.

 

If I cut the file down to 2+ million rows, it reads it in just fine.

 

I consider this to be a bug.

 

Thanks,

-greg

 

Research Analyst

Colorado Springs School District 11

 

[Richard Ristow]

At 02:28 PM 1/28/2010, CHESNEY, GREG wrote:

>I have a 4.5 million row, 5-field, csv text file. I cannot read it
>in (file>open>data). It doesn't show any errors but goes right into
>the input window dialogue.

SPSS's actual limit is huge, many times what you're reading, if you
have the disk space.

But that's the SPSS engine itself, the part of SPSS that's run by
syntax. The interactive portion of SPSS hits lower limits, under
circumstances that are hard to quantify.

>If I cut the file down to 2+ million rows, it reads it in just fine.

I recommend doing that; then, pasting rather than running the
resulting syntax. Then, run that syntax from a syntax file, reading
the whole of your data.

I expect this will do it. If it doesn't, the next step is to look for
oddities in your data.

>I consider this to be a bug.

No argument. A limitation, anyhow; with the documentation not saying
much about the relative limits of the menus and the SPSS engine itself.

-Good luck with this,
  Richard

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD