R for SPSS crashes using large dataset

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

R for SPSS crashes using large dataset

Van Overschelde, Jim
My SPSS syntax calls R code to run propensity score matching (PSM) routine (Thoemmes, 2012).
PSM runs fine with under 20,000 records but Startx32.exe crashes every time when 150,000 or more records included. Full file has 250,000.
I am assuming it is a memory-related error but can't find anything in my searches.  Crash report shows Fault Module name is MSVCR90.dll

System is Windows 2008 Server R2 64-bit, 8GB memory, dual quad-core system.
SPSS v20.

SUGGESTIONS???














































    Reply

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: R for SPSS crashes using large dataset

Bruce Weaver
Administrator
"SUGGESTIONS???"

Have you tried running PSM in R directly (i.e., without going through SPSS)?


Van Overschelde, Jim wrote
My SPSS syntax calls R code to run propensity score matching (PSM) routine (Thoemmes, 2012).
PSM runs fine with under 20,000 records but Startx32.exe crashes every time when 150,000 or more records included. Full file has 250,000.
I am assuming it is a memory-related error but can't find anything in my searches.  Crash report shows Fault Module name is MSVCR90.dll

System is Windows 2008 Server R2 64-bit, 8GB memory, dual quad-core system.
SPSS v20.

SUGGESTIONS???














































    Reply

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

PLEASE NOTE THE FOLLOWING: 
1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above.
2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/).
Reply | Threaded
Open this post in threaded view
|

Re: R for SPSS crashes using large dataset

Jim Van Overschelde
I would if I knew R.

I figured out a work around.  I modified the PSM routine so that it only
output the treatment and matched control records (<20,000) to the new
dataset and that worked.  If I tried to output the unmatched records also,
it crashed every time.


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Bruce Weaver
Sent: Wednesday, October 10, 2012 6:16 AM
To: [hidden email]
Subject: Re: R for SPSS crashes using large dataset

"SUGGESTIONS???"

Have you tried running PSM in R directly (i.e., without going through SPSS)?



Van Overschelde, Jim wrote

> My SPSS syntax calls R code to run propensity score matching (PSM)
> routine (Thoemmes, 2012).
> PSM runs fine with under 20,000 records but Startx32.exe crashes every
> time when 150,000 or more records included. Full file has 250,000.
> I am assuming it is a memory-related error but can't find anything in
> my searches.  Crash report shows Fault Module name is MSVCR90.dll
>
> System is Windows 2008 Server R2 64-bit, 8GB memory, dual quad-core
> system.
> SPSS v20.
>
> SUGGESTIONS???
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>     Reply
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the command. To leave the
> list, send the command SIGNOFF SPSSX-L For a list of commands to
> manage subscriptions, send the command INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/R-for-SPSS-crashes-using-large
-dataset-tp5715560p5715569.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command SIGNOFF SPSSX-L For a list of
commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD