SPSS memory/cache
Locked
 
|
Hi All,
I'm no expert on computing, so bear with me. I have a very complicated syntax running that involves using CREATE a lot (I have to do 120 leads over about 12 variables). Anyway, it typically takes around a couple of minutes to run on the first occasion. But every subsequent run is slower and slower again until it pretty much takes forever. Presumably SPSS is storing something in memory which is slowing everything down. Anyone have any idea how to improve this situation? I don't want to have to restart the PC after every run! Cheers Dan |
|
|
I don't know what is happening with the job on your computer. However, I wonder if the job would run faster if you reverse sorted the dataset and used lags. In most instances, lead is the inverse of lag but that may not work in your specific situation.
Gene Maguin -----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of dp1g08 Sent: Wednesday, May 06, 2015 5:37 AM To: [hidden email] Subject: SPSS memory/cache Hi All, I'm no expert on computing, so bear with me. I have a very complicated syntax running that involves using CREATE a lot (I have to do 120 leads over about 12 variables). Anyway, it typically takes around a couple of minutes to run on the first occasion. But every subsequent run is slower and slower again until it pretty much takes forever. Presumably SPSS is storing something in memory which is slowing everything down. Anyone have any idea how to improve this situation? I don't want to have to restart the PC after every run! Cheers Dan -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-memory-cache-tp5729494.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
In reply to this post by dp1g08
CREATE, unlike most procedures, keeps the
data in memory, because it often needs to use multiple leads and lags.
However, it should delete the data from memory once the computation
is completed. Watch the job run using the Task Manager (if on Windows).
Look at the spssengine process memory usage. You may need to
add these fields via the TM View menu: Working Set, Memory, Page Faults.
If these number grow continuously as the job runs, then memory is not being released, and eventually the page fault rate will rise, which would slow the job down. If your computer has a small amount of memory and the series are large, this could account for the slowdown. Restarting Statistics would free up the memory, but if CREATE is not releasing its memory, a bug should be reported. Jon Peck (no "h") aka Kim Senior Software Engineer, IBM [hidden email] phone: 720-342-5621 From: dp1g08 <[hidden email]> To: [hidden email] Date: 05/06/2015 03:37 AM Subject: [SPSSX-L] SPSS memory/cache Sent by: "SPSSX(r) Discussion" <[hidden email]> Hi All, I'm no expert on computing, so bear with me. I have a very complicated syntax running that involves using CREATE a lot (I have to do 120 leads over about 12 variables). Anyway, it typically takes around a couple of minutes to run on the first occasion. But every subsequent run is slower and slower again until it pretty much takes forever. Presumably SPSS is storing something in memory which is slowing everything down. Anyone have any idea how to improve this situation? I don't want to have to restart the PC after every run! Cheers Dan -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-memory-cache-tp5729494.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
Hi,
I've had a look on Task Manager whilst running the long series of CREATEs and all of the memory indicators are rising throughout. Then when the syntax is finished running they remain static at that high level. If I then run the syntax again, the memory indicators rise again from their already-high level. I have 8GB memory, so wouldn't have thought it'd be a problem. But obviously it is. I will try and give the reversal/LAG idea a go. Dan |
|
|
See also SHIFT VALUES, which allows you
to construct lead values without having to reverse sort.
Jon Peck (no "h") aka Kim Senior Software Engineer, IBM [hidden email] phone: 720-342-5621 From: dp1g08 <[hidden email]> To: [hidden email] Date: 05/06/2015 08:31 AM Subject: Re: [SPSSX-L] SPSS memory/cache Sent by: "SPSSX(r) Discussion" <[hidden email]> Hi, I've had a look on Task Manager whilst running the long series of CREATEs and all of the memory indicators are rising throughout. Then when the syntax is finished running they remain static at that high level. If I then run the syntax again, the memory indicators rise again from their already-high level. I have 8GB memory, so wouldn't have thought it'd be a problem. But obviously it is. I will try and give the reversal/LAG idea a go. Dan -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-memory-cache-tp5729494p5729502.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
«
Return to SPSSX Discussion
|
1 view|%1 views
| Free forum by Nabble | Edit this page |