sort cases memory problem

[Nico Peruzzi]

I am getting the following error when trying to run sort cases:

>Warning # 44
>The operating system could not allocate a memory segment of the size
>requested by SPSS.  Therefore, SPSS can only use the largest block
>available in memory already allocated.  Use SHOW WORKSPACE to determine the
>size of the request and SET WORKSPACE to change it.

>Error # 5817.  Command name: SORT CASES BY
>The SPSS file sort has failed.  The file remains unsorted.  The specific
>problem is printed below.
>This command not executed.

>Error # 5828.  Command name: SORT CASES BY
>Insufficient memory in which to sort the file.  Increase workspace with SET
>WORKSPACE command .

So, I use: SET WORKSPACE=2000000 and try again, and get the same error
meesage, and ti does not sort.  Even when I run some simple frequencies, it
works, but I get this error:

"The operating system could not allocate a memory segment of the size
requested by SPSS.  Therefore, SPSS can only use the largest block available
in memory already allocated.  Use SHOW WORKSPACE to determine the size of
the request and SET WORKSPACE to change it."

This past weekend, I was working with a very very large dataset that taxed
my system to the limit - perhaps this problem is related, as I had not
encountered it before then.

I have rebooted a few times, cleaned up my hard disk, defragmented, etc.
System has 1.5 GB of RAM, 1.6 MHz Centrino.  Everything else is working
fine.  I'm using SPSS 13.

Any ideas would be greatly appreciated.

Thanks, Nico

--
Nico Peruzzi, Ph.D.

[Oliver, Richard]

Well, you've allocated a really, really large workspace, and when I set my workspace to that size, I get the same errror, even with a relatively simple sort. You might trying knocking off a couple digits:

input program.
loop #i=1 to 1000000.
compute x=trunc(uniform(10)).
end case.
end loop.
end file.
end input program.
*this results in a workspace error on sort.
set workspace 2000000.
sort cases by x.
*this works.
set workspace 20000.
sort cases by x.

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Nico Peruzzi
Sent: Thursday, September 14, 2006 12:07 PM
To: [hidden email]
Subject: sort cases memory problem

I am getting the following error when trying to run sort cases:

>Warning # 44
>The operating system could not allocate a memory segment of the size
>requested by SPSS.  Therefore, SPSS can only use the largest block
>available in memory already allocated.  Use SHOW WORKSPACE to determine
>the size of the request and SET WORKSPACE to change it.

>Error # 5817.  Command name: SORT CASES BY The SPSS file sort has
>failed.  The file remains unsorted.  The specific problem is printed
>below.
>This command not executed.

>Error # 5828.  Command name: SORT CASES BY Insufficient memory in which
>to sort the file.  Increase workspace with SET WORKSPACE command .

So, I use: SET WORKSPACE=2000000 and try again, and get the same error meesage, and ti does not sort.  Even when I run some simple frequencies, it works, but I get this error:

"The operating system could not allocate a memory segment of the size requested by SPSS.  Therefore, SPSS can only use the largest block available in memory already allocated.  Use SHOW WORKSPACE to determine the size of the request and SET WORKSPACE to change it."

This past weekend, I was working with a very very large dataset that taxed my system to the limit - perhaps this problem is related, as I had not encountered it before then.

I have rebooted a few times, cleaned up my hard disk, defragmented, etc.
System has 1.5 GB of RAM, 1.6 MHz Centrino.  Everything else is working fine.  I'm using SPSS 13.

Any ideas would be greatly appreciated.

Thanks, Nico

--
Nico Peruzzi, Ph.D.

[Nico Peruzzi]

Richard,

That's great - thanks - it works with a smaller workspace.  I got so caught
up in having a huge workspace for the multi-million case file I was working
on, that I didn't realize it would have a negative effect on the more basic
things.

Nico

On 9/14/06, Oliver, Richard <[hidden email]> wrote:

>
> Well, you've allocated a really, really large workspace, and when I set my
> workspace to that size, I get the same errror, even with a relatively simple
> sort. You might trying knocking off a couple digits:
>
> input program.
> loop #i=1 to 1000000.
> compute x=trunc(uniform(10)).
> end case.
> end loop.
> end file.
> end input program.
> *this results in a workspace error on sort.
> set workspace 2000000.
> sort cases by x.
> *this works.
> set workspace 20000.
> sort cases by x.
>
> -----Original Message-----
> From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
> Nico Peruzzi
> Sent: Thursday, September 14, 2006 12:07 PM
> To: [hidden email]
> Subject: sort cases memory problem
>
> I am getting the following error when trying to run sort cases:
>
> >Warning # 44
> >The operating system could not allocate a memory segment of the size
> >requested by SPSS.  Therefore, SPSS can only use the largest block
> >available in memory already allocated.  Use SHOW WORKSPACE to determine
> >the size of the request and SET WORKSPACE to change it.
>
> >Error # 5817.  Command name: SORT CASES BY The SPSS file sort has
> >failed.  The file remains unsorted.  The specific problem is printed
> >below.
> >This command not executed.
>
> >Error # 5828.  Command name: SORT CASES BY Insufficient memory in which
> >to sort the file.  Increase workspace with SET WORKSPACE command .
>
> So, I use: SET WORKSPACE=2000000 and try again, and get the same error
> meesage, and ti does not sort.  Even when I run some simple frequencies, it
> works, but I get this error:
>
> "The operating system could not allocate a memory segment of the size
> requested by SPSS.  Therefore, SPSS can only use the largest block available
> in memory already allocated.  Use SHOW WORKSPACE to determine the size of
> the request and SET WORKSPACE to change it."
>
> This past weekend, I was working with a very very large dataset that taxed
> my system to the limit - perhaps this problem is related, as I had not
> encountered it before then.
>
> I have rebooted a few times, cleaned up my hard disk, defragmented, etc.
> System has 1.5 GB of RAM, 1.6 MHz Centrino.  Everything else is working
> fine.  I'm using SPSS 13.
>
> Any ideas would be greatly appreciated.
>
> Thanks, Nico
>
> --
> Nico Peruzzi, Ph.D.
>

--
Nico Peruzzi, Ph.D.

[Barnett, Adrian (HEALTH)-2]

Hi Nico
As Richard Oliver has already pointed out, your WORKSPACE request is
more than SPSS can handle.

This is why: The units that WORKSPACE is specified in are kilobytes, so
by requesting 2000000, you are requesting 2GB, whereas you say your
system only has 1.5 GB, so you could never get that much.

But there is, I believe, another issue at work here. Even if you'd asked
for substantially less than that, even if the amount requested was
within the capacity of your system, there's no guarantee you'd have
avoided the problem. I've found out in my own tests, that requesting 600
megabytes of workspace on a 2GB system will give you the same message,
even though the Windows System monitor is showing 900 MB free. It shows
the SPSS process using 600MB though. I've found that requesting anything
more than about 200 MB can generate these messages, again even when
there is plenty of free RAM. I'm pretty sure that SPSS just does not do
memory management well in current (and earlier) versions.

As I mentioned in an earlier posting, I'm hoping that V15 , seeing it no
longer supports Win95 and 98, will no longer be saddled with this
limitation and just leave memory management to the operating system,
which ought to have a better idea of what is going on and be able to
give SPSS whatever RAM is free when it asks for it.

I'd be interested to hear from someone at SPSS comment on this topic.

Regards


Adrian

--
Adrian Barnett
Research & Information Officer
Research, Analysis and Evaluation
Strategic Planning and Research Branch
Strategic Planning and Population Health Division
Department of Health


This e-mail may contain confidential information, which also may be
legally privileged.  Only the intended  recipient(s) may access, use,
distribute or copy this e-mail.  If this e-mail is received in error,
please inform the sender by return e-mail and delete the original.  If
there are doubts about the validity of this message, please contact the
sender by telephone.  It is the recipient's responsibility to check the
e-mail and any attached files for viruses.


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Nico Peruzzi
Sent: Friday, 15 September 2006 2:37
To: [hidden email]
Subject: sort cases memory problem

I am getting the following error when trying to run sort cases:

>Warning # 44
>The operating system could not allocate a memory segment of the size
>requested by SPSS.  Therefore, SPSS can only use the largest block
>available in memory already allocated.  Use SHOW WORKSPACE to determine

>the size of the request and SET WORKSPACE to change it.

>Error # 5817.  Command name: SORT CASES BY The SPSS file sort has
>failed.  The file remains unsorted.  The specific problem is printed
>below.
>This command not executed.

>Error # 5828.  Command name: SORT CASES BY Insufficient memory in which

>to sort the file.  Increase workspace with SET WORKSPACE command .

So, I use: SET WORKSPACE=2000000 and try again, and get the same error
meesage, and ti does not sort.  Even when I run some simple frequencies,
it works, but I get this error:

"The operating system could not allocate a memory segment of the size
requested by SPSS.  Therefore, SPSS can only use the largest block
available in memory already allocated.  Use SHOW WORKSPACE to determine
the size of the request and SET WORKSPACE to change it."

This past weekend, I was working with a very very large dataset that
taxed my system to the limit - perhaps this problem is related, as I had
not encountered it before then.

I have rebooted a few times, cleaned up my hard disk, defragmented, etc.
System has 1.5 GB of RAM, 1.6 MHz Centrino.  Everything else is working
fine.  I'm using SPSS 13.

Any ideas would be greatly appreciated.

Thanks, Nico

--
Nico Peruzzi, Ph.D.