SPSSX Discussion

Compressing SPSS data file

Classic

List

Threaded

15 messages Options

Thien Hop

Compressing SPSS data file

Hi all,

I'm working on tracking study and the SPSS data is expanding in size over
time. May I know if there is any way to effectively compress the file. I
have hassle working with file above 20MB.

Thank you so much and hearing your advices,

TH

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Albert-Jan Roskam

Re: Compressing SPSS data file

Hi!
if you use '/compressed' while saving, it should save some space, but in my experience that's not very much. I often zip/rar various data file versions. That makes a huge diffence.
save outfile 'd:/temp/myfile.sav' /compressed.

Albert-Jan

--- On Thu, 7/30/09, Thien Hop <[hidden email]> wrote:

> From: Thien Hop <[hidden email]>
> Subject: Compressing SPSS data file
> To: [hidden email]
> Date: Thursday, July 30, 2009, 10:20 AM
> Hi all,
>
> I'm working on tracking study and the SPSS data is
> expanding in size over
> time. May I know if there is any way to effectively
> compress the file. I
> have hassle working with file above 20MB.
>
> Thank you so much and hearing your advices,
>
> TH
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email]
> (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the
> command
> INFO REFCARD
>

Richard Ristow

Re: Compressing SPSS data file

In reply to this post by Thien Hop

At 04:20 AM 7/30/2009, Thien Hop wrote:

>I'm working on tracking study and the SPSS data is expanding in size
>over time. May I know if there is any way to effectively compress
>the file. I have hassle working with file above 20MB.

This may be a time to look at your hardware. 20MB isn't a very big
dataset these days; SPSS should handle it easily.

What version of SPSS are you using, under what operating system?

How much main memory (RAM) do you have?

How much free disk space to you have? (I recommend at least triple
the size of your file, ideally 5-6 times.)

-Good luck,\
Richard

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Thien Hop

Re: Compressing SPSS data file

Thank you Richard & Albert for your reply!

You are right, the funcion "compress" in SPPSS is not really helpful.

Sorry Richard as I'm using version 12 which cannot run 2 processors at the
same time hence I have to set affinity to 1 CPU only. I think the RAM
(500MB) & free disk space (20GBs) shouldn't be a problem.

For now I still can handle the processing but As the file is getting larger
and larger after every month, I wonder if there is any work-around solution
or I have to switch to other software ie. survey craft?

Thank you so much,

TH
----- Original Message -----
From: "Richard Ristow" <[hidden email]>
To: "Thien Hop" <[hidden email]>; <[hidden email]>
Sent: Friday, July 31, 2009 12:50 AM
Subject: Re: Compressing SPSS data file

> At 04:20 AM 7/30/2009, Thien Hop wrote:
>
>>I'm working on tracking study and the SPSS data is expanding in size over
>>time. May I know if there is any way to effectively compress the file. I
>>have hassle working with file above 20MB.
>
> This may be a time to look at your hardware. 20MB isn't a very big dataset
> these days; SPSS should handle it easily.
>
> What version of SPSS are you using, under what operating system?
>
> How much main memory (RAM) do you have?
>
> How much free disk space to you have? (I recommend at least triple the
> size of your file, ideally 5-6 times.)
>
> -Good luck,\
> Richard
>

Lemon, John S.

Re: Compressing SPSS data file

At the risk of starting a ‘bragging war’ of “my data set is bigger than yours” – I think you should consider upgrading your version of SPSS as I’m currently working on a data set with 17.0.2 which has 15 variables, 35,000,000+ cases which takes up about 1.5 Gbytes. This is on two year old 1 Gbyte memory dual core machine which your reply indicates you have and although it works , it can be slow !!

I’m not sure whether SurveyCraft will help as the description of it indicates it exports the data to SPSS !!

Best Wishes

John S. Lemon

Student Liaison Officer

Directorate of Information Technology (DIT) - University of Aberdeen

Edward Wright Building: Room G51

Tel: +44 1224 273350

Fax: +44 1224 273372

Diary ( Free / Busy )

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Thien Hop
Sent: 31 July 2009 09:11
To: [hidden email]
Subject: Re: Compressing SPSS data file

Thank you Richard & Albert for your reply!

You are right, the funcion "compress" in SPPSS is not really helpful.

Sorry Richard as I'm using version 12 which cannot run 2 processors at the

same time hence I have to set affinity to 1 CPU only. I think the RAM

(500MB) & free disk space (20GBs) shouldn't be a problem.

For now I still can handle the processing but As the file is getting larger

and larger after every month, I wonder if there is any work-around solution

or I have to switch to other software ie. survey craft?

Thank you so much,

The University of Aberdeen is a charity registered in Scotland, No SC013683.

Albert-Jan Roskam

Re: Compressing SPSS data file

I understood that the Spss WORKSPACE is set quite conservatively, so one could increase it to boost performance (depending on RAM, probably). Does anybody know a rule-of-thumb for this? I used trial and error in the past and saw some performance improvements using SET WORKSPACE, but setting the amount of workspace too high causes crashes.

Cheers!!
Albert-Jan

--- On Fri, 7/31/09, Lemon, John S. <[hidden email]> wrote:

> From: Lemon, John S. <[hidden email]>
> Subject: Re: Compressing SPSS data file
> To: [hidden email]
> Date: Friday, July 31, 2009, 10:36 AM
>
>
>
>
>
>
>
>
>
>
> At
> the risk of starting a ‘bragging war’ of
> “my data set is bigger than yours” – I
> think you should consider upgrading your version of SPSS as
> I’m currently working on
> a data set with 17.0.2 which has 15 variables,
> 35,000,000+ cases which takes up about 1.5 Gbytes. This
> is on two year old 1 Gbyte memory dual core machine which
> your reply indicates you have and although it works , it can
> be slow !!
>
> �
> I’m not sure whether
> SurveyCraft will help as the description of it indicates it
> exports the data to SPSS !!
>
>
> �
> Best
> Wishes
> �
> John S.
> Lemon
> Student Liaison
> Officer
> Directorate of Information
> Technology (DIT) - University of
> Aberdeen
> Edward Wright Building:
> Room� G51
> �
> Tel:� +44
> 1224 273350
> Fax: +44 1224
> 273372
>
> �
> Diary ( Free / Busy
> )
>
> �
> �
> -----Original
> Message-----
>
> From: SPSSX(r) Discussion [mailto:[hidden email]]
> On Behalf Of Thien Hop
>
> Sent: 31 July 2009 09:11
>
> To: [hidden email]
>
> Subject: Re: Compressing SPSS data file
> �
> Thank you Richard & Albert for
> your reply!
> �
> You are right, the funcion
> "compress" in SPPSS is not really helpful.
> �
> Sorry Richard as I'm using
> version 12 which cannot run 2 processors at the
> same time hence I have to set
> affinity to 1 CPU only. I think the RAM
> (500MB) & free disk space
> (20GBs) shouldn't be a problem.
> �
> For now I still can handle the
> processing but As the file is getting larger
> and larger after every month, I
> wonder if there is any work-around solution
> or I have to switch to other
> software ie. survey craft?
> �
> Thank you so much,
> �
> TH
>
>
>
>
>
> The University of Aberdeen is a charity registered in
> Scotland, No SC013683.
>
>
>
>
>

Paul Thoresen

PASW Data Collection Interviewer Web and PASW Data Collection Author

Hi Everyone,

I am wondering if anyone uses the SPPS products for web data collection (AKA the former mrInterview products)?

We are considering the use of the SPSS Data Collection product for the web using the Data Collection Author to build the surveys. I am hoping there may be a few users in this forum who have experiences with it and could provide me some feedback. From what I see in comparison to a few other products, it has some nice advantages such as:

• Ability to incorporate into full SPSS suite
• Integration with other SPSS products providing for unified way to distribute surveys from one application
• Very user friendly interface i.e. Drag/drop questions, Create survey by uploading Word document, pull in questions from old sav files with variable and value labels.
• Export directly into SPSS data files

It also has some big disadvantages:

• Highest start up cost
• Highest ongoing cost
• Additional Training may be beneficial to fully take advantage of the functionality
• Most of our surveys are mixed method/Hybrid and we use a different software for paper distribution.

Please feel free to email me offline, or to discuss in the forum. Any feedback is appreciated.
Thank you!

Best regards,

Paul

Paul Thoresen
Manager, Surveys and Research
American Academy of Neurology
1080 Montreal Ave., St. Paul, MN 55116
Ph: 651.695.2803 Fax: 651.361.4903
[hidden email] www.aan.com

2009 AAN Fall Conference, November 6 – 8, 2009, in Las Vegas. Early registration deadline is October 5. Register today at www.aan.com/view/Fall2009.

AAN Vision: To be indispensable to our members.
AAN Mission: To promote the highest quality patient-centered neurologic care and enhance member career satisfaction.

Legal Notice: This message may contain confidential or legally privileged information intended only for the use of the addressee(s) named above. Unauthorized use, disclosure, distribution, or copying is prohibited. If you have received this message in error, please reply to the sender and delete the original message. Thank you.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Peck, Jon

Re: Compressing SPSS data file

In reply to this post by Albert-Jan Roskam

The workspace is a mixed bag. While some procedures use it, many do not - allocating memory dynamically as needed. So making it too large can deprive those procedures of the use of that memory.

Note this comment from the CSR:
Do not increase the workspace memory allocation unless the program issues a message that
there is not enough memory to complete a procedure.

HTH,
Jon Peck

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Albert-Jan Roskam
Sent: Friday, July 31, 2009 10:06 AM
To: [hidden email]
Subject: Re: [SPSSX-L] Compressing SPSS data file

I understood that the Spss WORKSPACE is set quite conservatively, so one could increase it to boost performance (depending on RAM, probably). Does anybody know a rule-of-thumb for this? I used trial and error in the past and saw some performance improvements using SET WORKSPACE, but setting the amount of workspace too high causes crashes.

Cheers!!
Albert-Jan

--- On Fri, 7/31/09, Lemon, John S. <[hidden email]> wrote:

> From: Lemon, John S. <[hidden email]>
> Subject: Re: Compressing SPSS data file
> To: [hidden email]
> Date: Friday, July 31, 2009, 10:36 AM
>
>
>
>
>
>
>
>
>
>
> At
> the risk of starting a ‘bragging war’ of
> “my data set is bigger than yours” – I
> think you should consider upgrading your version of SPSS as
> I’m currently working on
> a data set with 17.0.2 which has 15 variables,
> 35,000,000+ cases which takes up about 1.5 Gbytes. This
> is on two year old 1 Gbyte memory dual core machine which
> your reply indicates you have and although it works , it can
> be slow !!
>
>
> I’m not sure whether
> SurveyCraft will help as the description of it indicates it
> exports the data to SPSS !!
>
>
>
> Best
> Wishes
>
> John S.
> Lemon
> Student Liaison
> Officer
> Directorate of Information
> Technology (DIT) - University of
> Aberdeen
> Edward Wright Building:
> Room G51
>
> Tel: +44
> 1224 273350
> Fax: +44 1224
> 273372
>
>
> Diary ( Free / Busy
> )
>
>
>
> -----Original
> Message-----
>
> From: SPSSX(r) Discussion [mailto:[hidden email]]
> On Behalf Of Thien Hop
>
> Sent: 31 July 2009 09:11
>
> To: [hidden email]
>
> Subject: Re: Compressing SPSS data file
>
> Thank you Richard & Albert for
> your reply!
>
> You are right, the funcion
> "compress" in SPPSS is not really helpful.
>
> Sorry Richard as I'm using
> version 12 which cannot run 2 processors at the
> same time hence I have to set
> affinity to 1 CPU only. I think the RAM
> (500MB) & free disk space
> (20GBs) shouldn't be a problem.
>
> For now I still can handle the
> processing but As the file is getting larger
> and larger after every month, I
> wonder if there is any work-around solution
> or I have to switch to other
> software ie. survey craft?
>
> Thank you so much,
>
> TH
>
>
>
>
>
> The University of Aberdeen is a charity registered in
> Scotland, No SC013683.
>
>
>
>
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

John McConnell-2

Re: PASW Data Collection Interviewer Web and PASW Data Collection Author

In reply to this post by Paul Thoresen

Hi Paul

If you haven't done so already you might want to consider posting the same question to the "SPSS Dimensions Users" Group on Linkedin

"Dimensions" as you are probably aware was the former name for PASW Data Collection.

Best regards

John

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Paul Thoresen
Sent: 31 July 2009 16:32
To: [hidden email]
Subject: PASW Data Collection Interviewer Web and PASW Data Collection Author

Hi Everyone,

I am wondering if anyone uses the SPPS products for web data collection (AKA the former mrInterview products)?

We are considering the use of the SPSS Data Collection product for the web using the Data Collection Author to build the surveys. I am hoping there may be a few users in this forum who have experiences with it and could provide me some feedback. From what I see in comparison to a few other products, it has some nice advantages such as:

• Ability to incorporate into full SPSS suite
• Integration with other SPSS products providing for unified way to distribute surveys from one application
• Very user friendly interface i.e. Drag/drop questions, Create survey by uploading Word document, pull in questions from old sav files with variable and value labels.
• Export directly into SPSS data files

It also has some big disadvantages:

• Highest start up cost
• Highest ongoing cost
• Additional Training may be beneficial to fully take advantage of the functionality
• Most of our surveys are mixed method/Hybrid and we use a different software for paper distribution.

Please feel free to email me offline, or to discuss in the forum. Any feedback is appreciated.
Thank you!

Best regards,

Paul

Paul Thoresen
Manager, Surveys and Research
American Academy of Neurology
1080 Montreal Ave., St. Paul, MN 55116
Ph: 651.695.2803 Fax: 651.361.4903
[hidden email] www.aan.com

2009 AAN Fall Conference, November 6 – 8, 2009, in Las Vegas. Early registration deadline is October 5. Register today at www.aan.com/view/Fall2009.

AAN Vision: To be indispensable to our members.
AAN Mission: To promote the highest quality patient-centered neurologic care and enhance member career satisfaction.

Legal Notice: This message may contain confidential or legally privileged information intended only for the use of the addressee(s) named above. Unauthorized use, disclosure, distribution, or copying is prohibited. If you have received this message in error, please reply to the sender and delete the original message. Thank you.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Thien Hop

Re: Compressing SPSS data file

In reply to this post by Lemon, John S.

Thank you John and Jon Peck!

It's worthy considering version 17 however, I need to use the autoscript.sbs which is no longer supported in newer version. I tried to save my current scripts in the name StartClient_.wwd and put it in the scripts subdirectory of your SPSS Statistics installation as what Jon Peck advised last time but it cannot auto start with SPSS v17.

Jon Peck also highly recommended using Python, I wonder if there is any tool that help convert all current scripts to Python (as the number of scripts are a lot and the language is different from normal syntax/ script, it's good to have a conversion).

Thank you very much and have a nice week ahead,

----- Original Message -----

From: [hidden email]

To: [hidden email] ; [hidden email]

Sent: Friday, July 31, 2009 4:36 PM

Subject: RE: Compressing SPSS data file

At the risk of starting a bragging war of my data set is bigger than yours I think you should consider upgrading your version of SPSS as Im currently working on a data set with 17.0.2 which has 15 variables, 35,000,000+ cases which takes up about 1.5 Gbytes. This is on two year old 1 Gbyte memory dual core machine which your reply indicates you have and although it works , it can be slow !!

Im not sure whether SurveyCraft will help as the description of it indicates it exports the data to SPSS !!

Best Wishes

John S. Lemon

Student Liaison Officer

Directorate of Information Technology (DIT) - University of Aberdeen

Edward Wright Building: Room G51

Tel: +44 1224 273350

Fax: +44 1224 273372

Diary ( Free / Busy )

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Thien Hop
Sent: 31 July 2009 09:11
To: [hidden email]
Subject: Re: Compressing SPSS data file

Thank you Richard & Albert for your reply!

You are right, the funcion "compress" in SPPSS is not really helpful.

Sorry Richard as I'm using version 12 which cannot run 2 processors at the

same time hence I have to set affinity to 1 CPU only. I think the RAM

(500MB) & free disk space (20GBs) shouldn't be a problem.

For now I still can handle the processing but As the file is getting larger

and larger after every month, I wonder if there is any work-around solution

or I have to switch to other software ie. survey craft?

Thank you so much,

TH

The University of Aberdeen is a charity registered in Scotland, No SC013683.

Peck, Jon

Re: Compressing SPSS data file

In reply to this post by Thien Hop

Basic autoscripts do still work, although a minor change described in the help is required. The monolithic autoscript.sbs file is no longer used. Instead, individual files are attached to table subtypes or a base autoscript can be set for all scripting events.

There is no converter from Basic to Python.

HTH,
Jon Peck
(From Heathrow)

From: SPSSX(r) Discussion <[hidden email]>
To: [hidden email] <[hidden email]>
Sent: Sun Aug 02 22:16:00 2009
Subject: Re: [SPSSX-L] Compressing SPSS data file

Thank you John and Jon Peck!

Thank you very much and have a nice week ahead,

----- Original Message -----

From: [hidden email]

To: [hidden email] ; [hidden email]

Sent: Friday, July 31, 2009 4:36 PM

Subject: RE: Compressing SPSS data file

At the risk of starting a ‘bragging war’ of “my data set is bigger than yours” – I think you should consider upgrading your version of SPSS as I’m currently working on a data set with 17.0.2 which has 15 variables, 35,000,000+ cases which takes up about 1.5 Gbytes. This is on two year old 1 Gbyte memory dual core machine which your reply indicates you have and although it works , it can be slow !!

I’m not sure whether SurveyCraft will help as the description of it indicates it exports the data to SPSS !!

Best Wishes

John S. Lemon

Student Liaison Officer

Directorate of Information Technology (DIT) - University of Aberdeen

Edward Wright Building: Room G51

Tel: +44 1224 273350

Fax: +44 1224 273372

Diary ( Free / Busy )

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Thien Hop
Sent: 31 July 2009 09:11
To: [hidden email]
Subject: Re: Compressing SPSS data file

Thank you Richard & Albert for your reply!

You are right, the funcion "compress" in SPPSS is not really helpful.

Sorry Richard as I'm using version 12 which cannot run 2 processors at the

same time hence I have to set affinity to 1 CPU only. I think the RAM

(500MB) & free disk space (20GBs) shouldn't be a problem.

For now I still can handle the processing but As the file is getting larger

and larger after every month, I wonder if there is any work-around solution

or I have to switch to other software ie. survey craft?

Thank you so much,

TH

The University of Aberdeen is a charity registered in Scotland, No SC013683.

Paul Thoresen

Re: PASW Data Collection Interviewer Web and PASW Data Collection Author

In reply to this post by John McConnell-2

Thank you John, I will check it out!
I had not thought of that.

Cheers,

Paul

Paul Thoresen
Manager, Surveys and Research
American Academy of Neurology
1080 Montreal Ave., St. Paul, MN 55116
Ph: 651.695.2803 Fax: 651.361.4903
[hidden email] www.aan.com

-----Original Message-----
From: John McConnell [mailto:[hidden email]]
Sent: Sunday, August 02, 2009 5:21 AM
To: Paul Thoresen; [hidden email]
Subject: RE: PASW Data Collection Interviewer Web and PASW Data Collection Author

Hi Paul

If you haven't done so already you might want to consider posting the same question to the "SPSS Dimensions Users" Group on Linkedin

"Dimensions" as you are probably aware was the former name for PASW Data Collection.

Best regards

John

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Paul Thoresen
Sent: 31 July 2009 16:32
To: [hidden email]
Subject: PASW Data Collection Interviewer Web and PASW Data Collection Author

Hi Everyone,

I am wondering if anyone uses the SPPS products for web data collection (AKA the former mrInterview products)?

We are considering the use of the SPSS Data Collection product for the web using the Data Collection Author to build the surveys. I am hoping there may be a few users in this forum who have experiences with it and could provide me some feedback. From what I see in comparison to a few other products, it has some nice advantages such as:

• Ability to incorporate into full SPSS suite
• Integration with other SPSS products providing for unified way to distribute surveys from one application
• Very user friendly interface i.e. Drag/drop questions, Create survey by uploading Word document, pull in questions from old sav files with variable and value labels.
• Export directly into SPSS data files

It also has some big disadvantages:

• Highest start up cost
• Highest ongoing cost
• Additional Training may be beneficial to fully take advantage of the functionality
• Most of our surveys are mixed method/Hybrid and we use a different software for paper distribution.

Please feel free to email me offline, or to discuss in the forum. Any feedback is appreciated.
Thank you!

Best regards,

Paul

Paul Thoresen
Manager, Surveys and Research
American Academy of Neurology
1080 Montreal Ave., St. Paul, MN 55116
Ph: 651.695.2803 Fax: 651.361.4903
[hidden email] www.aan.com

2009 AAN Fall Conference, November 6 – 8, 2009, in Las Vegas. Early registration deadline is October 5. Register today at www.aan.com/view/Fall2009.

AAN Vision: To be indispensable to our members.
AAN Mission: To promote the highest quality patient-centered neurologic care and enhance member career satisfaction.

Legal Notice: This message may contain confidential or legally privileged information intended only for the use of the addressee(s) named above. Unauthorized use, disclosure, distribution, or copying is prohibited. If you have received this message in error, please reply to the sender and delete the original message. Thank you.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

2009 AAN Fall Conference, November 6 – 8, 2009, in Las Vegas. Early registration deadline is October 5. Register today at www.aan.com/view/Fall2009.

AAN Vision: To be indispensable to our members.
AAN Mission: To promote the highest quality patient-centered neurologic care and enhance member career satisfaction.

Legal Notice: This message may contain confidential or legally privileged information intended only for the use of the addressee(s) named above. Unauthorized use, disclosure, distribution, or copying is prohibited. If you have received this message in error, please reply to the sender and delete the original message. Thank you.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Richard Ristow

Re: Compressing SPSS data file

In reply to this post by Thien Hop

At 04:11 AM 7/31/2009, Thien Hop wrote:

>Sorry Richard as I'm using version 12 which cannot run 2 processors
>at the same time hence I have to set affinity to 1 CPU only. I think
>the RAM (500MB) & free disk space (20GBs) shouldn't be a problem.

Version 12 should be able to handle it; as far as I've heard, later
versions aren't faster.

Being limited to a single processor shouldn't matter. The speed of
most SPSS jobs is limited by the disk transfer rate, not the processor.

As to resources, your 20GB of free disk space is much more than
adequate. The 500MB of RAM may be a little tight (under Windows XP)
or inadequate (under MAC OS or Vista). Increasing to at least 1G may help.

Finally, coding problems can make SPSS programs very inefficient. You
haven't written anything about your code, so I can't say more than
that. But, to start with: Do you have any EXECUTE statements? If you
do, what happens if you take all of them out?

-Best of luck to you,
Richard

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Thien Hop

Re: Compressing SPSS data file

Thank you Richard, I think you are right the RAM is a bit too small and I
have difficulty even opening large file (I normally cannot open directly
from network drive but copy to C drive for any processing. I may need to
upgrade RAM to increase the speed.

Regarding the codes, they work quite fast and smooth with small file. I keep
them almost the same overtime except some small updates on the banner for
the tabs. I think the trouble maker is not coding but file size and RAM as
you mentioned.

Thanks & cheers,

TH
----- Original Message -----
From: "Richard Ristow" <[hidden email]>
To: "Thien Hop" <[hidden email]>; <[hidden email]>
Sent: Tuesday, August 04, 2009 9:11 PM
Subject: Re: Compressing SPSS data file

> At 04:11 AM 7/31/2009, Thien Hop wrote:
>
>>Sorry Richard as I'm using version 12 which cannot run 2 processors at the
>>same time hence I have to set affinity to 1 CPU only. I think the RAM
>>(500MB) & free disk space (20GBs) shouldn't be a problem.
>
> Version 12 should be able to handle it; as far as I've heard, later
> versions aren't faster.
>
> Being limited to a single processor shouldn't matter. The speed of most
> SPSS jobs is limited by the disk transfer rate, not the processor.
>
> As to resources, your 20GB of free disk space is much more than adequate.
> The 500MB of RAM may be a little tight (under Windows XP) or inadequate
> (under MAC OS or Vista). Increasing to at least 1G may help.
>
> Finally, coding problems can make SPSS programs very inefficient. You
> haven't written anything about your code, so I can't say more than that.
> But, to start with: Do you have any EXECUTE statements? If you do, what
> happens if you take all of them out?
>
> -Best of luck to you,
> Richard
>
>
>
>

Richard Ristow

Re: Compressing SPSS data file

At 12:02 AM 8/5/2009, Thien Hop wrote:

>Thank you Richard, I think you are right the RAM is a bit too small
>and I have difficulty even opening large file. I may need to upgrade
>RAM to increase the speed.

Good. I think it will help; it may help dramatically. For example, it
may leave enough room that your entire file can be cached in RAM, and
then processing it would become enormously faster.

>Regarding the codes, they work quite fast and smooth with small
>file. I keep them almost the same overtime except some small updates
>on the banner for the tabs. I think the trouble maker is not coding
>but file size and RAM as you mentioned.

Well, good luck with this, and I hope that increasing RAM does solve
it. But do watch for code inefficiencies. Increasing file size slows
down any SPSS job, and can bring out inefficiencies that didn't show
up with smaller files.

-Best to you,
Richard

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD