spss limit

[jimjohn]

does anyone know what the maximum number of cases an spss file can load? thanks!

[willers]

>
> does anyone know what the maximum number of cases an spss file can load?
> thanks!
No limit.
Reinhart Willers

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Lemon, John S.]

Has this changed then ?

I always thought the limits was 1 less than the maximum number that could be stored on the machine / operating system and for PCs it was (2*10**32)-1 or something like that

Best Wishes

John S. Lemon
Student Liaison Officer
Directorate of Information Technology (DIT) - University of Aberdeen
Edward Wright Building: Room G51
Tel:  [phone]
Fax: [phone]

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Reinhart Willers
Sent: Tuesday, May 06, 2008 5:58 PM
To: [hidden email]
Subject: AW: spss limit

>
> does anyone know what the maximum number of cases an spss file can load?
> thanks!
No limit.
Reinhart Willers

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


The University of Aberdeen is a charity registered in Scotland, No SC013683.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[jimjohn]

thanks guys. ill actually be having a huge dataset (about 4 million), so hopefully it will in spss

jimjohn wrote

does anyone know what the maximum number of cases an spss file can load? thanks!

[Art Kendall]

The limit is as much as you can put in a file on the storage (disks) of
*your* system.
Some OS's can not span multiple volumes for a single file so you may
need to have all of a file on a single disk.
Figure about 8 bytes per numeric variable per case  + the number of
bytes in string variables.
say you have 4 million cases with 2 thousand bytes of data each that
gives about 8 Gigabytes. your system must allow an 8 G file. and about
twice that for additional scratch space.



Art Kendall
Social research Consultants

jimjohn wrote:

> thanks guys. ill actually be having a huge dataset (about 4 million), so
> hopefully it will in spss
>
>
>
> jimjohn wrote:
>
>> does anyone know what the maximum number of cases an spss file can load?
>> thanks!
>>
>>
>
> --
> View this message in context: http://www.nabble.com/spss-limit-tp17087209p17091075.html
> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>
>
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Johnny Amora]

What about the limit in terms of number of variables or columns?


Art Kendall <[hidden email]> wrote: The limit is as much as you can put in a file on the storage (disks) of
*your* system.
Some OS's can not span multiple volumes for a single file so you may
need to have all of a file on a single disk.
Figure about 8 bytes per numeric variable per case  + the number of
bytes in string variables.
say you have 4 million cases with 2 thousand bytes of data each that
gives about 8 Gigabytes. your system must allow an 8 G file. and about
twice that for additional scratch space.



Art Kendall
Social research Consultants

jimjohn wrote:

> thanks guys. ill actually be having a huge dataset (about 4 million), so
> hopefully it will in spss
>
>
>
> jimjohn wrote:
>
>> does anyone know what the maximum number of cases an spss file can load?
>> thanks!
>>
>>
>
> --
> View this message in context: http://www.nabble.com/spss-limit-tp17087209p17091075.html
> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>
>
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD



Johnny T. Amora
Center for Learning and Performance Assessment
De La Salle-College of Saint Benilde
Manila, Philippines

---------------------------------
Be a better friend, newshound, and know-it-all with Yahoo! Mobile.  Try it now.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Victor Kogler]

Our friend Raynald has the answer to that question -

http://www.spsstools.net/FAQ.htm#HowManyVariables


Johnny Amora wrote:

> What about the limit in terms of number of variables or columns?
>
>
> Art Kendall <[hidden email]> wrote: The limit is as much as you can put in a file on the storage (disks) of
> *your* system.
> Some OS's can not span multiple volumes for a single file so you may
> need to have all of a file on a single disk.
> Figure about 8 bytes per numeric variable per case  + the number of
> bytes in string variables.
> say you have 4 million cases with 2 thousand bytes of data each that
> gives about 8 Gigabytes. your system must allow an 8 G file. and about
> twice that for additional scratch space.
>
>
>
> Art Kendall
> Social research Consultants
>
> jimjohn wrote:
>
>> thanks guys. ill actually be having a huge dataset (about 4 million), so
>> hopefully it will in spss
>>
>>
>>
>> jimjohn wrote:
>>
>>
>>> does anyone know what the maximum number of cases an spss file can load?
>>> thanks!
>>>
>>>
>>>
>> --
>> View this message in context: http://www.nabble.com/spss-limit-tp17087209p17091075.html
>> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>>
>> =====================
>> To manage your subscription to SPSSX-L, send a message to
>> [hidden email] (not to SPSSX-L), with no body text except the
>> command. To leave the list, send the command
>> SIGNOFF SPSSX-L
>> For a list of commands to manage subscriptions, send the command
>> INFO REFCARD
>>
>>
>>
>>
>
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Richard Ristow]

At 09:19 PM 5/6/2008, Johnny Amora wrote:

>What about the limit in terms of number of variables or columns?

Since this comes up from time to time --

FAQ: How many variables and cases are allowed in SPSS?

Below is a discussion by Jon Peck of SPSS, Inc., which applies to all
SPSS versions that are likely to be in use - back to release 9, at least.

I add to what Jon wrote,

. For most operations, increasing the number of cases will increase
the running time about in proportion. Usually, SPSS can handle a
great many cases gracefully. Jon, below, notes some operations for
which many cases may slow SPSS badly, but often work-around can be
found even for these.

. Increasing the number of variables will generally increase the
running time about in proportion, even if you're not using them all,
because the running time is dominated by the time to read the file
from disk, i.e. the total file size

. After some point hard to estimate (though larger if the machine has
more RAM), increasing the number of variables will increase the
running time out of all proportion, because putting the whole
dictionary and data for one case in RAM may require paging.

. I emphasize Jon's point that "modern database practice would be to
break up your variables into cohesive subsets", i.e. to restructure
with more cases and fewer variables. A typical example is changing
from one record per entity with data for many years, to one record
per entity per year. I've posted a number of solutions in which data
is given such a 'long' representation with many cases, instead of a
'wide' representation with many variables.

At 10:25 AM 6/5/2003, Peck, Jon [of SPSS, Inc.] wrote:

>There are several points to making regarding very wide files and
>huge datasets.
>
>First, the theoretical SPSS limits are
>
>Number of variables: (2**31) -1
>Number of cases: (2**31) - 1
>
>In calculating these limits, count one for each 8 bytes or part
>thereof of a string variable.  An a10 variable counts as two
>variables, for example.
>
>Approaching the theoretical limit on the number of variables,
>however, is a very bad idea in practice for several reasons.
>
>1. These are the theoretical limits in that you absolutely cannot go
>beyond them.  But there are other environmentally imposed limits
>that you will surely hit first.  For example, Windows applications
>are absolutely limited to 2GB of addressable memory, and 1GB is a
>more practical limit.  Each dictionary entry requires about 100
>bytes of memory, because in addition to the variable name, other
>variable properties also have to be stored.  (On non-Windows
>platforms, SPSS Server could, of course, face different
>environmental limits.)  Numerical variable values take 8 bytes as
>they are held as double precision floating point values.
>
>2. The overhead of reading and writing extremely wide cases when you
>are doubtless not using more than a small fraction of them will
>limit performance.  And you don't want to be paging the variable
>dictionary.  If you have lots of RAM, you can probably reach between
>32,000 and 100,000 variables before memory paging degrades
>performance seriously.
>
>3. Dialog boxes cannot display very large variable lists.  You can
>use variable sets to restrict the lists to the variables you are
>really using, but lists with thousands of variables will always be awkward.
>
>4. Memory usage is not just about the dictionary.  The operating
>system will almost always be paging code and data between memory and
>disk.  (You can look at paging rates via the Windows Task
>Manager).  The more you page, the slower things get, but the
>variable dictionary is only one among many objects that the
>operating system is juggling.  However, there is another effect.  On
>NT and later, Windows automatically caches files (code or data) in
>memory so that it can retrieve it quickly.  This cache occupies
>memory that is otherwise surplus, so if any application needs it,
>portions of the cache are discarded to make room.  You can see this
>effect quite clearly if you start SPSS or any other large
>application; then shut it down and start it again.  It will load
>much more quickly the second time, because it is retrieving the code
>modules needed at startup from memory rather than disk.  The Windows
>cache, unfortunately, will not help data access very much unless
>most of the dataset stays in memory, because the cache will
>generally hold the most recently accessed data.  If you are reading
>cases sequentially, the one you just finished with is the LAST one
>you will want again.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD