SPSSINC RanFor

[pspangler1]

When running the following code using the SPSSINC Random forest classifier I receive the following error,
"Error: Error in matrix(0, n, n): too many elements specified".  I have 6 scale predictor that include 0s for a 5 level variable in a 10k case data set.  It appears this has to do with building the proximity matrix but this algorithm is described as fast with many predictors and case volume. I have the R package set to a my computer's maximum of 16GB of RAM.

See notes here http://www.alanfielding.co.uk/multivar/rf.htm

SPSSINC RANFOR DEPENDENT=BuyerCluster
ENTER=log_BabyandKids log_CollectingArt log_CraftsHobbiesToys log_Entertainment log_Fashion log_HomeOutdoor log_JewelryWatches log_SportingGoodsFitness log_TechGadgets UNSUPERVISED=NO
/OPTIONS MISSING=ROUGH NUMTREES=500
/SAVE PREDVALUES=predicted_cluster
/PLOT VARIABLEIMPORTANCE  MDSPLOT=YES MDSPLOTDIM=2.
randomForest 4.6-7
Type rfNews() to see new features/changes/bug fixes.
Error: Error in matrix(0, n, n): too many elements specified
In addition: Warning message:
package 'randomForest' was built under R version 2.15.3

Any idea how I can avoid this error?

[David Marso]

Peter,
Given the fact that I can successfully execute the command
x<-matrix(1,20000,20000) under my 64 bit version of R I confidently conclude you are attempting to do this on a 32 bit system.  That ain't gonna fly!
It doesn't matter how much RAM you have on the box. Neither 32 bit OS nor 32 bit software can grab more than a paltry 4G or so.

pspangler1 wrote

When running the following code using the SPSSINC Random forest classifier I receive the following error,
"Error: Error in matrix(0, n, n): too many elements specified".  I have 6 scale predictor that include 0s for a 5 level variable in a 10k case data set.  It appears this has to do with building the proximity matrix but this algorithm is described as fast with many predictors and case volume. I have the R package set to a my computer's maximum of 16GB of RAM.

See notes here http://www.alanfielding.co.uk/multivar/rf.htm

SPSSINC RANFOR DEPENDENT=BuyerCluster
ENTER=log_BabyandKids log_CollectingArt log_CraftsHobbiesToys log_Entertainment log_Fashion log_HomeOutdoor log_JewelryWatches log_SportingGoodsFitness log_TechGadgets UNSUPERVISED=NO
/OPTIONS MISSING=ROUGH NUMTREES=500
/SAVE PREDVALUES=predicted_cluster
/PLOT VARIABLEIMPORTANCE  MDSPLOT=YES MDSPLOTDIM=2.
randomForest 4.6-7
Type rfNews() to see new features/changes/bug fixes.
Error: Error in matrix(0, n, n): too many elements specified
In addition: Warning message:
package 'randomForest' was built under R version 2.15.3

Any idea how I can avoid this error?

[Jon K Peck]

You are using a 64-bit OS, but are you using 64- or 32-bit Statistics (check the About box).  How did you set the R memory limit?

Run this code to see what the R code being used with Statistics actually has as a memory limit and the R configuration being used.

begin program r.
print(memory.limit())
print(sessionInfo())
end program.



Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
phone: [phone]




From:        pspangler1 <[hidden email]>
To:        [hidden email]
Date:        01/07/2015 04:56 PM
Subject:        [SPSSX-L] SPSSINC RanFor
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

When running the following code using the SPSSINC Random forest classifier I
receive the following error,
"Error: Error in matrix(0, n, n): too many elements specified".  I have 6
scale predictor that include 0s for a 5 level variable in a 10k case data
set.  It appears this has to do with building the proximity matrix but this
algorithm is described as fast with many predictors and case volume. I have
the R package set to a my computer's maximum of 16GB of RAM.

See notes here http://www.alanfielding.co.uk/multivar/rf.htm

SPSSINC RANFOR DEPENDENT=BuyerCluster
ENTER=log_BabyandKids log_CollectingArt log_CraftsHobbiesToys
log_Entertainment log_Fashion log_HomeOutdoor log_JewelryWatches
log_SportingGoodsFitness log_TechGadgets UNSUPERVISED=NO
/OPTIONS MISSING=ROUGH NUMTREES=500
/SAVE PREDVALUES=predicted_cluster
/PLOT VARIABLEIMPORTANCE  MDSPLOT=YES MDSPLOTDIM=2.
randomForest 4.6-7
Type rfNews() to see new features/changes/bug fixes.
Error: Error in matrix(0, n, n): too many elements specified
In addition: Warning message:
package 'randomForest' was built under R version 2.15.3

Any idea how I can avoid this error?




--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSSINC-RanFor-tp5728377.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[David Marso]

That would be my conclusion.

When I submit the following I receive enlightening output:

BEGIN PROGRAM R.

sessionInfo()

END PROGRAM.

-----------------------------------------

R version 2.15.2 (2012-10-26)

Platform: x86_64-w64-mingw32/x64 (64-bit)

On Wed, Jan 7, 2015 at 6:40 PM, Peter Spangler <[hidden email]> wrote:

I appreciate the conclusion, however, I am operating on a 3.00GHz, 64-bit Windows box using Windows 7.

I have R x64 2.15.0 on this machine. Could this be calling on my other R version that is not 64bit?

On Wed, Jan 7, 2015 at 3:25 PM, David Marso <[hidden email]> wrote:

Peter,
Given the fact that I can successfully execute the command
x<-matrix(1,20000,20000) under my 64 bit version of R I confidently conclude
you are attempting to do this on a 32 bit system.  That ain't gonna fly!
It doesn't matter how much RAM you have on the box. Neither 32 bit OS nor 32
bit software can grab more than a paltry 4G or so.


pspangler1 wrote
> When running the following code using the SPSSINC Random forest classifier
> I receive the following error,
> "Error: Error in matrix(0, n, n): too many elements specified".  I have 6
> scale predictor that include 0s for a 5 level variable in a 10k case data
> set.  It appears this has to do with building the proximity matrix but
> this algorithm is described as fast with many predictors and case volume.
> I have the R package set to a my computer's maximum of 16GB of RAM.
>
> See notes here http://www.alanfielding.co.uk/multivar/rf.htm
>
> SPSSINC RANFOR DEPENDENT=BuyerCluster
> ENTER=log_BabyandKids log_CollectingArt log_CraftsHobbiesToys
> log_Entertainment log_Fashion log_HomeOutdoor log_JewelryWatches
> log_SportingGoodsFitness log_TechGadgets UNSUPERVISED=NO
> /OPTIONS MISSING=ROUGH NUMTREES=500
> /SAVE PREDVALUES=predicted_cluster
> /PLOT VARIABLEIMPORTANCE  MDSPLOT=YES MDSPLOTDIM=2.
> randomForest 4.6-7
> Type rfNews() to see new features/changes/bug fixes.
> Error: Error in matrix(0, n, n): too many elements specified
> In addition: Warning message:
> package 'randomForest' was built under R version 2.15.3
>
> Any idea how I can avoid this error?

-----
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me.
---
"Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis."
Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?"
--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSSINC-RanFor-tp5728377p5728379.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

This email may contain confidential information for the sole use of the intended recipient(s). If you are not an intended recipient, please notify the sender and delete all copies immediately.

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[David Marso]

Run
BEGIN PROGRAM R.
help (memory.size)
END PROGRAM.

For further insight.