insufficient memory Generalized Linear Mixed Models
Locked
insufficient memory Generalized Linear Mixed Models
 
|
Dear list,
I am trying to do a logistic multilevel regression using Generalized Linear Mixed Models in SPSS but I get an error message that there is insufficient memory. The data set is huge (it contains the data from a complete population) so that may explain why the analysis doesn't run. But does anyone know how to resolve this problem? I have already set the Workspace at its maximum level. Any help would be highly appreciated! Best regards, Joost van Ginkel ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Re: insufficient memory Generalized Linear Mixed Models
Administrator
|
Maybe provide a little 411 on your computing environment?
I'll bet you are running on a 32 bit doze system (limiting you to about 3G). That's likely the root of the problem!
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me. --- "Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis." Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?" |
Re: insufficient memory Generalized Linear Mixed Models
|
|
GLMM does not use the static workspace,
I believe. It gets its memory dynamically. In fact, setting
the workspace to a large values takes memory away from procedures that
allocate dynamically. However, a gigantic problem might still run
out of memory on a 32-bit system. Note that some (many?) users are
on a 64-bit environment but actually running 32-bit Statistics, which can't
take advantage of the extra memory possibilities.
Jon Peck (no "h") aka Kim Senior Software Engineer, IBM [hidden email] phone: 720-342-5621 From: David Marso <[hidden email]> To: [hidden email], Date: 05/14/2014 09:06 AM Subject: Re: [SPSSX-L] insufficient memory Generalized Linear Mixed Models Sent by: "SPSSX(r) Discussion" <[hidden email]> Maybe provide a little 411 on your computing environment? I'll bet you are running on a 32 bit doze system (limiting you to about 3G). That's likely the root of the problem! Joost van Ginkel wrote > Dear list, > > I am trying to do a logistic multilevel regression using Generalized > Linear Mixed Models in SPSS but I get an error message that there is > insufficient memory. The data set is huge (it contains the data from a > complete population) so that may explain why the analysis doesn't run. But > does anyone know how to resolve this problem? I have already set the > Workspace at its maximum level. Any help would be highly appreciated! > > Best regards, > > Joost van Ginkel > > ===================== > To manage your subscription to SPSSX-L, send a message to > LISTSERV@.UGA > (not to SPSSX-L), with no body text except the > command. To leave the list, send the command > SIGNOFF SPSSX-L > For a list of commands to manage subscriptions, send the command > INFO REFCARD ----- Please reply to the list and not to my personal email. Those desiring my consulting or training services please feel free to email me. --- "Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis." Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?" -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/insufficient-memory-Generalized-Linear-Mixed-Models-tp5726028p5726030.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Re: insufficient memory Generalized Linear Mixed Models
|
|
In reply to this post by David Marso
The computer is indeed a 32 bit system but I run the analyses from a server with remote access to a much bigger computer which performs all the calculations. So can't we exclude this possibility then?
Best, Joost -----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of David Marso Sent: woensdag 14 mei 2014 17:05 To: [hidden email] Subject: Re: insufficient memory Generalized Linear Mixed Models Maybe provide a little 411 on your computing environment? I'll bet you are running on a 32 bit doze system (limiting you to about 3G). That's likely the root of the problem! Joost van Ginkel wrote > Dear list, > > I am trying to do a logistic multilevel regression using Generalized > Linear Mixed Models in SPSS but I get an error message that there is > insufficient memory. The data set is huge (it contains the data from a > complete population) so that may explain why the analysis doesn't run. > But does anyone know how to resolve this problem? I have already set > the Workspace at its maximum level. Any help would be highly appreciated! > > Best regards, > > Joost van Ginkel > > ===================== > To manage your subscription to SPSSX-L, send a message to > LISTSERV@.UGA > (not to SPSSX-L), with no body text except the command. To leave the > list, send the command SIGNOFF SPSSX-L For a list of commands to > manage subscriptions, send the command INFO REFCARD ----- Please reply to the list and not to my personal email. Those desiring my consulting or training services please feel free to email me. --- "Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis." Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?" -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/insufficient-memory-Generalized-Linear-Mixed-Models-tp5726028p5726030.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Re: insufficient memory Generalized Linear Mixed Models
|
|
Can you post the syntax you're running
to the list?
|
Re: insufficient memory Generalized Linear Mixed Models
|
|
Unfortunately I can’t. I work from a server from a company that gathers data from the complete Dutch population. Everything I do on that computer
is not allowed to leave the computer without going through a very time consuming and expensive bureaucratic procedure first. Another problem is that the software that gives me access to the server only runs on a 32 bit system. I think the best thing to do
would be to ask the company if they can improve their software so that it runs on 64 bit systems too.
L From: SPSSX(r) Discussion [mailto:[hidden email]]
On Behalf Of Alex Reutter Can you post the syntax you're running to the list?
|
Re: insufficient memory Generalized Linear Mixed Models
|
|
Understood. Tens of millions of records
is a lot, but if you're able to build other models normally on the same
system, you might want to take a careful look at the number of parameters
in your model's G matrix; i.e., if you're trying to fit a 100x100
unstructured matrix, there's your culprit, and should consider fitting
a more structured and/or smaller matrix.
Running on 64-bit systems would probably help, too. Alex From: "Ginkel, Joost van" <[hidden email]> To: Alex Reutter/Burlington/IBM@IBMUS, "[hidden email]" <[hidden email]>, Date: 05/15/2014 04:04 AM Subject: RE: insufficient memory Generalized Linear Mixed Models Unfortunately I can’t. I work from a server from a company that gathers data from the complete Dutch population. Everything I do on that computer is not allowed to leave the computer without going through a very time consuming and expensive bureaucratic procedure first. Another problem is that the software that gives me access to the server only runs on a 32 bit system. I think the best thing to do would be to ask the company if they can improve their software so that it runs on 64 bit systems too. L From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Alex Reutter Sent: woensdag 14 mei 2014 19:37 To: [hidden email] Subject: Re: insufficient memory Generalized Linear Mixed Models Can you post the syntax you're running to the list? |
|
|
In reply to this post by Joost van Ginkel
I'm sorry to interject, but one ought to be able to change variable names to something generic ("a", "b" etc.) and post the code. If what you are working on is so highly confidential that you cannot even provide code with variables represented by letters to the list, then I would suggest you contact IBM SPSS tech support directly. In addition to the code, I would need to know the approximate number of units at each level. That would help us determine the true computational intensiveness of your model. If that too is confidential (I don't see how that's possible), then again I would suggest your best bet would be to contact tech support. Ryan On Thu, May 15, 2014 at 4:03 AM, Ginkel, Joost van <[hidden email]> wrote:
|
Re: insufficient memory Generalized Linear Mixed Models
|
|
It’s not so much confidential but for everything that leaves the computer you would have to ask permission. For a syntax file it would undoubtedly
be easier to get permission than for a data file or output. The point is that I would have to ask for permission in the first place and going through a procedure of filling out a form and so on. For the moment I think that’s too much trouble, especially since
I may have already found a way to carry out the analysis by deleting the cases in the data file that I don’t need for the analysis. From: SPSSX(r) Discussion [mailto:[hidden email]]
On Behalf Of Ryan Black I'm sorry to interject, but one ought to be able to change variable names to something generic ("a", "b" etc.) and post the code. If what you are working on is so highly confidential that you cannot even provide code with variables represented
by letters to the list, then I would suggest you contact IBM SPSS tech support directly. In addition to the code, I would need to know the approximate number of units at each level. That would help us determine the true computational intensiveness of your
model. If that too is confidential (I don't see how that's possible), then again I would suggest your best bet would be to contact tech support. Ryan On Thu, May 15, 2014 at 4:03 AM, Ginkel, Joost van <[hidden email]> wrote: Unfortunately I can’t. I work from a server from a company that gathers data from the
complete Dutch population. Everything I do on that computer is not allowed to leave the computer without going through a very time consuming and expensive bureaucratic procedure first. Another problem is that the software that gives me access to the server
only runs on a 32 bit system. I think the best thing to do would be to ask the company if they can improve their software so that it runs on 64 bit systems too.
L From: SPSSX(r)
Discussion [mailto:[hidden email]]
On Behalf Of Alex Reutter
Can you post the syntax you're running to the list?
|
|
|
I see. Well, I hope the solution works. Best of luck! Ryan On Thu, May 15, 2014 at 9:50 AM, Ginkel, Joost van <[hidden email]> wrote:
|
| Free forum by Nabble | Edit this page |