SPSS 20 + 64 bit OS

[drfg2008]

Does SPSS 20 comply with a Windows 64 bit OS ?

Frank

[Jon K Peck]

The 32-bit Statistics V20 runs under Win7 64-bit and other 64-bit Windows system just fine, but in order to take advantage of the larger address space available in those environments, you need the 64-bit version of Statistics.

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: [phone]




From:        drfg2008 <[hidden email]>
To:        [hidden email]
Date:        01/23/2012 07:38 AM
Subject:        [SPSSX-L] SPSS 20 + 64 bit OS
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

Does SPSS 20 comply with a Windows 64 bit OS ?

Frank

-----
Dr. Frank Gaeth
FU-Berlin

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-20-64-bit-OS-tp5166219p5166219.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[drfg2008]

We downloaded SPSS 20 from IBM Server (don't have the details right now), it runs on our 32 bit machine. Is it possible to use the same downloaded Software also for 64 bit, or is the 64 bit version a different program that has to be purchased separately.

[J. R. Carroll]

It is my understanding that 32bit SPSS will work on both win32 and win64 OSs.

However, you can not use a 64bit SPSS package on a 32bit machine.

All else fails - try to install it. The SPSS installer will let you know if you meet the minimum system requirements.

J. R. Carroll (sent from my phone)
www.jrcresearch.net
Email: [hidden email]
           [hidden email]
           [hidden email]

On Jan 26, 2012, at 10:00 PM, drfg2008 <[hidden email]> wrote:

> We downloaded SPSS 20 from IBM Server (don't have the details right now), it
> runs on our 32 bit machine. Is it possible to use the same downloaded
> Software also for 64 bit, or is the 64 bit version a different program that
> has to be purchased separately.
>
> -----
> Dr. Frank Gaeth
> FU-Berlin
>
> --
> View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-20-64-bit-OS-tp5166219p5434647.html
> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[drfg2008]

Thanks J. R. Carroll, now I probably begin to understand Jon K Peck's answer:

The 32 bit version of SPSS 20 runs on a 64 bit OS but does NOT profit from the larger workspace (over the 4Gig limit of 32bit OS). So it does not run faster or can not compute greater ammounts of data just because it is installed on a 64 bit OS more than 4 Gig RAM.

Bad news.

[drfg2008]

Thanks also for the information about 64 bit specials (memory pointer sizes, runtime libraries).

How can one speed up a 32 bit version as much as possible: Probably only by using a machine with more kernals (eg. six core)? Would linux be an advantage to windows?

[J. R. Carroll]

Frank,

 

Thanks also for the information about 64 bit specials (memory pointer sizes,
runtime libraries).
How can one speed up a 32 bit version as much as possible: Probably only by
using a machine with more kernals (eg. six core)? Would linux be an
advantage to windows? 

There are a number of solutions that I have worked with in the past.  Hardware and software specifics notwithstanding, consider the following:

1.  Last I heard, individual lic of SPSS 32bit/64bit cannot utilize multi-core technology; if you want to utilize multiple cores, I believe you have to get the server edition (and the server - it's not cheap), and run SPSS "remotely" (or on the server itself).

2.  Running a Linux distribution won't necessarily help you.  It's more about how much 'crap' you have running on your PC that will contribute to slowing down SPSS.  For instance, if you are browsing the internet or listening to streaming music you will be potentially taking away from available processing power.  Shut down extra applications (and services running in the background) to avoid any potential resource hogging issues.

3.  My home PC (which is also my backup work PC) is tailored/custom-built (I built it, but I also paid for premium parts) - things that I have found over the years that help SPSS that are hardware related are the following:  faster RPM harddrive (I have a 10,000 RPM raptor HD that I use to run my OS and as a scratch disk for writing data like SPSS) - better yet, if you can get a solid-state HD, you might see improvements, but I do not know this for certain; faster RAM (1066, 1333, 1666, etc) - the bad part here is that you need to make sure your motherboard (MOBO) supports faster RAM before you buy it (and try not to mix RAM speeds).  There are other concerns regarding RAM that you'll want to consider, but this should be discussed with your IT support (if you have them); another thing to consider is proper cooling!  I have seen considerably faster results in computing with PC's that ran at colder temperatures (each processor/PC has its optimal temp range - follow that!).  I use a custom designed case with intake and outtake fans that draw/pull air in a systematic fashion, and I use a liquid cooled radiator on my processor.  This keeps my PC running around 27-32 degrees Celsius (if I am remembering correctly).  My friends PC without proper air flow and a standard fan/copper radiator cooling his processor runs closer to 45-50 deg Celsius.  

4.  How you run your SPSS will also contribute to performance (and Jon Peck and David Marso can speak on this issue better than I, I am sure).  I have done several Monte Carlo simulations in the past and I've used Python to run a set of simulations, save the data/output, close it all, and start the next series of simulations.  By introducing/adapting my syntax to consider different ways to processing my data, I had better results than if I had just done it all with syntax (chances are that I wrote the syntax poorly and the same job could have been optimized in SPSS without Python - but the point still stands - optimize your scripts!).

I'd start with those considerations and improve where you can.  But, upgrading your PC in both hardware/software in only one domain may prove futile - if you are upgrading (and you can afford it) you may just want to get the best-of-the-best PC + 64bit windows (or the equiv of Linux/Apple), server edition of SPSS, etc... but, that costs some serious $$.

HTH,

 

----


J. R. Carroll

On Fri, Jan 27, 2012 at 8:02 AM, drfg2008 <[hidden email]> wrote:

Thanks also for the information about 64 bit specials (memory pointer sizes,
runtime libraries).

How can one speed up a 32 bit version as much as possible: Probably only by
using a machine with more kernals (eg. six core)? Would linux be an
advantage to windows?

-----
Dr. Frank Gaeth
FU-Berlin

--

View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-20-64-bit-OS-tp5166219p5435831.html

Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Jon K Peck]

More cores will help in some cases.  Faster disks will always help.  Being sure that you have enough memory so that the system is not paging helps.  And get rid of most EXECUTE statements, which force generally needless data passes.

You can measure Statistics performance with the STATS BENCHMRK extension command available from the SPSS Community.

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: [phone]




From:        drfg2008 <[hidden email]>
To:        [hidden email]
Date:        01/27/2012 09:06 AM
Subject:        Re: [SPSSX-L] SPSS 20 + 64 bit OS
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

Thanks also for the information about 64 bit specials (memory pointer sizes,
runtime libraries).

How can one speed up a 32 bit version as much as possible: Probably only by
using a machine with more kernals (eg. six core)? Would linux be an
advantage to windows?



-----
Dr. Frank Gaeth
FU-Berlin

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-20-64-bit-OS-tp5166219p5435831.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[drfg2008]

Our IT checked the download options, would that be the 64 bit SPSS version ?



Download details
IBM SPSS Statistics Client 64-Bit 20.0 Windows Multilingual
________________________________________
Product description
File name
SPSS_Statistics_20_win64_.exe
________________________________________
Platform(s)
Windows 7 Enterprise, Windows 7 Professional, Windows Vista Business, Windows Vista Enterprise, Windows XP Professional

[J. R. Carroll]

Frank,

I'm sure someone responded off-list, but to formally respond:  yes - the snip-it you posted about the download is the 64bit version.  

win32 = 32bit version

win64 = 64bit version

Just a bit of nomenclature =P

-J

----


J. R. Carroll

Independent Researcher through Hurtz Labs

Research Methods, Test Development, and Statistics

www.jrcresearch.net

Cell:  [phone]

Email: [hidden email]

          [hidden email]

          [hidden email]

On Mon, Jan 30, 2012 at 4:35 AM, drfg2008 <[hidden email]> wrote:

Our IT checked the download options, would that be the 64 bit SPSS version ?



Download details
IBM SPSS Statistics Client 64-Bit 20.0 Windows Multilingual
________________________________________
Product description
File name
SPSS_Statistics_20_win64_.exe
________________________________________
Platform(s)
Windows 7 Enterprise, Windows 7 Professional, Windows Vista Business,
Windows Vista Enterprise, Windows XP Professional

-----
Dr. Frank Gaeth
FU-Berlin

--

View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-20-64-bit-OS-tp5166219p5441306.html

Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[drfg2008]

Our IT just told me that our Windows only accepts max. 16 Gig RAM. Does Linux accept more than 16? Would SPSS 20 run on a linux machine with more than 16 Gig RAM and make use of it?

Sorry for the somewhat basic questions.

[David Marso]

What for crying out loud are you doing which would conceivably require more than 16G.
AFAIK, Win64 can address MUCH more than 16G.  It must be your specific computers which will accept no more than 16G.
I remember when PCs shipped with only 16MB.
Before that I recall the old Mac in the lab which had 512K memory.

drfg2008 wrote

Our IT just told me that our Windows only accepts max. 16 Gig RAM. Does Linux accept more than 16? Would SPSS 20 run on a linux machine with more than 16 Gig RAM and make use of it?

Sorry for the somewhat basic questions.

[Jon K Peck]

I suppose that you are referring to physical memory.  The Win7 virtual memory limit would be much higher.  Statistics can use more than the physical memory, but swapping will increase.  However, it would be very unusual for Statistics to consume memory in such quantities.  Most procedures do not keep the case data in memory.  Time series procedures are an exception, but they don't usually involve huge quantities of data.

If you really think that Statistics is thrashing due to physical memory limits, try observing the Statistics processes in the Task Manager, watching the page fault rate or use the new STATS BENCHMRK extension command to measure time, memory, and i/o statistics.

HTH,

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: [phone]




From:        drfg2008 <[hidden email]>
To:        [hidden email]
Date:        02/06/2012 05:38 AM
Subject:        Re: [SPSSX-L] SPSS 20 + 64 bit OS
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

Our IT just told me that our Windows only accepts max. 16 Gig RAM. Does Linux
accept more than 16? Would SPSS 20 run on a linux machine with more than 16
Gig RAM and make use of it?

Sorry for the somewhat basic questions.

-----
Dr. Frank Gaeth
FU-Berlin

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-20-64-bit-OS-tp5166219p5459989.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Rich Ulrich]

Memory lane?  A competition?

> Date: Mon, 6 Feb 2012 06:44:22 -0800
> From: [hidden email]
> Subject: Re: SPSS 20 + 64 bit OS
> To: [hidden email]
>
> What for crying out loud are you doing which would conceivably require more
> than 16G.
> AFAIK, Win64 can address MUCH more than 16G. It must be your specific
> computers which will accept no more than 16G.
> I remember when PCs shipped with only 16MB.

The first MAINFRAME that I worked on, at my first job (1968), had 32K
of memory, measured in 16-bit words.  The project did not pay
extra to get the 64K version.  This was an IBM 1130. 

It came with 2 disks drives that used platters of about 12 inch diameter,
with plastic cases about 2 inches tall.  Storage per disk was about 5 MB,
as I calculated years later.  There was no internal disk.  There was a
card reader, a paper-tape reader, and a printer (upper-case only) that
ran at 60 lines per minute.  Lower-case did not become widely available
on terminals, etc., until about 1980.

> Before that I recall the old Mac in the lab which had 512K memory.

My first PC was a TI, "IBM-compatible" that cost about $2000.  It came
with 64KB of memory, which I later expanded to 640 KB.  I wrote a
terminal emulator program in Basic in order to connect to a DEC-10
mainframe which allowed 256K (36-bit words) for its maximum job size.
Submitting jobs of 32K assured faster turn-around. 
[snip, previous]

--
Rich Ulrich

[drfg2008]

What for crying out loud are you doing which would conceivably require more than 16G.

David,
68.774.810 rows / 4798 variables -> GLM

Bye,
Frank

[David Marso]

I smell something fishy!
The number of cases (rows) is likely irrelevant.
4798 (are you sure there aren't 4797) is a rather odd request to submit to GLM.
Have you experimented to see at what point the thing crumbles?
4798*4798=23,020,804 covariance elements.  I suspect that would fit into about 200M.
If these are all continuous (scale) variables then who knows what the limit would be.
If they are categorical then the problem size grows exponentially if a full factorial (the default) model is inadvertently specified.
So, in your court.
MATRIX.
COMPUTE FATTY=MAKE(4798,4798,1).
END MATRIX.
No problem.  I am running 11.5 under Vista (win32) 4G RAM.


 

drfg2008 wrote

What for crying out loud are you doing which would conceivably require more than 16G.

David,
68.774.810 rows / 4798 variables -> GLM

Bye,
Frank

[Albert-Jan Roskam]

If you ditch all but the 10-or-so vars you need, your data is about 15.7Gb (assuming it's all numeric) uncompressed. Enough to even keep the whole thing in core.

I've processed 500M datasets with much less memory. Ok, it takes a while (I speak fluent Mandarine Chinese now), but it works.

 

Cheers!!
Albert-Jan

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a
fresh water system, and public health, what have the Romans ever done for us?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

---

From: drfg2008 <[hidden email]>
To: [hidden email]
Sent: Wednesday, February 8, 2012 6:03 PM
Subject: Re: [SPSSX-L] SPSS 20 + 64 bit OS

/What for crying out loud are you doing which would conceivably require more
than 16G. /

David,
68.774.810 rows / 4798 variables -> GLM

Bye,
Frank

-----
Dr. Frank Gaeth
FU-Berlin

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-20-64-bit-OS-tp5166219p5467049.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

 

[David Marso]

I'm proceeding under the (INSANE) assumption that Frank is trying to analyze *ALL* 4798 variables at the same time in a single GLM ;-)

Albert-Jan Roskam wrote

If you ditch all but the 10-or-so vars you need, your data is about 15.7Gb (assuming it's all numeric) uncompressed. Enough to even keep the whole thing in core.

I've processed 500M datasets with much less memory. Ok, it takes a while (I speak fluent Mandarine Chinese now), but it works.

 
Cheers!!
Albert-Jan

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a
fresh water system, and public health, what have the Romans ever done for us?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


>________________________________
> From: drfg2008 <[hidden email]>
>To: [hidden email] 
>Sent: Wednesday, February 8, 2012 6:03 PM
>Subject: Re: [SPSSX-L] SPSS 20 + 64 bit OS
>
>/What for crying out loud are you doing which would conceivably require more
>than 16G. /
>
>David,
>68.774.810 rows / 4798 variables -> GLM
>
>Bye,
>Frank
>
>-----
>Dr. Frank Gaeth
>FU-Berlin
>
>--
>View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-20-64-bit-OS-tp5166219p5467049.html
>Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
>=====================
>To manage your subscription to SPSSX-L, send a message to
>[hidden email] (not to SPSSX-L), with no body text except the
>command. To leave the list, send the command
>SIGNOFF SPSSX-L
>For a list of commands to manage subscriptions, send the command
>INFO REFCARD
>
>
> 

[drfg2008]

I'm proceeding under the (INSANE) assumption that Frank is trying to analyze *ALL* 4798 variables at the same time in a single GLM ;-)

David,
it's a live-game model with all teams / players as dummys and results (for example goals) in a minutes- / seconds - timeframe. It's (of course) not about checking, what variable is stat. relevant, but to estimate the mean expected value (goal) as precise as possible and as fast as possible for the time-frame of the specific game. Since the teams / players play in different 'leagues' and the combination of plays and leagues is hard to differentiate (at least for a programmer) and not 'closed' (not limited to a closed number of leagues), all teams / players should be in one (super-) model.

Plus, the model should run as fast as possible.

Hence my question of max. Gig.

(And a bivariate Poisson- distribution (Team1 - Team2), by the way. Found a solution?)

Frank

[David Marso]

Frank,
Looks like you will have a serious problem with independence, clustering/hierarchy etc which GLM cannot easily address.  I suspect the design matrix might be permutable to something close to a block diagonal which would allow you to partition the "super-model" into several smaller submodels. Doesn't seem that this is likely to be an easy model to formulate much or less estimate quickly.  I don't have any references handy, but you should look into sparse matrices, updating formulas for sparse matrix inverses, what have others who have researched this area used?  SPSS doesn't readily lend itself to 'real-time' analysis.
HTH, David
--

drfg2008 wrote

I'm proceeding under the (INSANE) assumption that Frank is trying to analyze *ALL* 4798 variables at the same time in a single GLM ;-)

David,
it's a live-game model with all teams / players as dummys and results (for example goals) in a minutes- / seconds - timeframe. It's (of course) not about checking, what variable is stat. relevant, but to estimate the mean expected value (goal) as precise as possible and as fast as possible for the time-frame of the specific game. Since the teams / players play in different 'leagues' and the combination of plays and leagues is hard to differentiate (at least for a programmer) and not 'closed' (not limited to a closed number of leagues), all teams / players should be in one (super-) model.

Plus, the model should run as fast as possible.

Hence my question of max. Gig.

(And a bivariate Poisson- distribution (Team1 - Team2), by the way. Found a solution?)

Frank