deleting rows from CTABLES based on data values

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

deleting rows from CTABLES based on data values

Kylie
Hi all,

I am after some help with what I believe will be a script (though I could be
wrong!). For a current project we will be generating ~300 custom tables with
the following structure:

CTABLES
  /VLABELS VARIABLES=majorgrp agegroup sex nut1 DISPLAY=DEFAULT
  /TABLE majorgrp BY agegroup > sex > nut1 [COLPCT.SUM PCT40.1]
  /CATEGORIES VARIABLES=majorgrp ORDER=A KEY=VALUE EMPTY=INCLUDE TOTAL=YES
POSITION=AFTER
  /CATEGORIES VARIABLES=agegroup sex ORDER=A KEY=VALUE EMPTY=INCLUDE.

Ie, one row per 'majorgrp', with columns by age and sex, and the data cells
containing column percentages.

We need to be able to delete any rows from the table where all values are
less than 1.5%. The actual row should be deleted (including row labels), not
just the data values cleared from the cells.

My best guess is that this will require a script, but I couldn't find
anything online that was close enough to give me a starting point. Does
anyone have any suggestions or alternative ways to approach this? Something
that could be run over all pivot tables in an output file would be great.

Thanks,
Kylie.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: deleting rows from CTABLES based on data values

Bruce Weaver
Administrator
Kylie Lange-3 wrote
Hi all,

I am after some help with what I believe will be a script (though I could be
wrong!). For a current project we will be generating ~300 custom tables with
the following structure:

CTABLES
  /VLABELS VARIABLES=majorgrp agegroup sex nut1 DISPLAY=DEFAULT
  /TABLE majorgrp BY agegroup > sex > nut1 [COLPCT.SUM PCT40.1]
  /CATEGORIES VARIABLES=majorgrp ORDER=A KEY=VALUE EMPTY=INCLUDE TOTAL=YES
POSITION=AFTER
  /CATEGORIES VARIABLES=agegroup sex ORDER=A KEY=VALUE EMPTY=INCLUDE.

Ie, one row per 'majorgrp', with columns by age and sex, and the data cells
containing column percentages.

We need to be able to delete any rows from the table where all values are
less than 1.5%. The actual row should be deleted (including row labels), not
just the data values cleared from the cells.

My best guess is that this will require a script, but I couldn't find
anything online that was close enough to give me a starting point. Does
anyone have any suggestions or alternative ways to approach this? Something
that could be run over all pivot tables in an output file would be great.

Thanks,
Kylie.
See my response here:

http://groups.google.com/group/comp.soft-sys.stat.spss/browse_frm/thread/b5b5e4774e0572ec#

--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

PLEASE NOTE THE FOLLOWING: 
1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above.
2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/).
Reply | Threaded
Open this post in threaded view
|

Re: deleting rows from CTABLES based on data values

Jon K Peck
In reply to this post by Kylie

See below.

Jon Peck
SPSS, an IBM Company
[hidden email]
312-651-3435



From: Kylie Lange <[hidden email]>
To: [hidden email]
Date: 08/17/2010 03:03 AM
Subject: [SPSSX-L] deleting rows from CTABLES based on data values
Sent by: "SPSSX(r) Discussion" <[hidden email]>





Hi all,

I am after some help with what I believe will be a script (though I could be
wrong!). For a current project we will be generating ~300 custom tables with
the following structure:

CTABLES
 /VLABELS VARIABLES=majorgrp agegroup sex nut1 DISPLAY=DEFAULT
 /TABLE majorgrp BY agegroup > sex > nut1 [COLPCT.SUM PCT40.1]
 /CATEGORIES VARIABLES=majorgrp ORDER=A KEY=VALUE EMPTY=INCLUDE TOTAL=YES
POSITION=AFTER
 /CATEGORIES VARIABLES=agegroup sex ORDER=A KEY=VALUE EMPTY=INCLUDE.

Ie, one row per 'majorgrp', with columns by age and sex, and the data cells
containing column percentages.

We need to be able to delete any rows from the table where all values are
less than 1.5%. The actual row should be deleted (including row labels), not
just the data values cleared from the cells.

My best guess is that this will require a script, but I couldn't find
anything online that was close enough to give me a starting point. Does
anyone have any suggestions or alternative ways to approach this? Something
that could be run over all pivot tables in an output file would be great.


>>>This can be done with the SPSSINC MODIFY TABLES extension command available from SPSS Developer Central (www.spss.com/devcentral).  It requires at least V17.0.2 and the Python plugin, although it will work better and faster with V19.

There is a new custom function for it named HideRowBasedOnValues available with the latest version.  Here is an example that hides rows where all the values are less than 5.  If you have row totals and want to exclude them from the test, there are additional parameters for that.

This command is set to process all the tables of the specified type in the Viewer, so you could just specify it at the end of a bunch of CTABLES commands.
More details are in the customstylefunctions module included in the zip.

CTABLES
  /TABLE educ BY gender > jobcat [COLPCT.COUNT].

SPSSINC MODIFY TABLES subtype="Custom Table"
SELECT="<<ALL>>" DIMENSION= ROWS LEVEL = -1 PROCESS = ALL
/STYLES  APPLYTO=LABELS
CUSTOMFUNCTION="customstylefunctions.HideRowBasedOnValues(threshold=5)".

This function doesn't actually delete the rows - it hides them.

HTH,
Jon Peck



Thanks,
Kylie.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


Reply | Threaded
Open this post in threaded view
|

Scripts textbook

Nirit Avnimelech

Hi,

 

Can anyone recommend a good book or online source for learning how to write scripts?  I have 30 years of experience with SPSS (yes, all you young people, my first encounter with SPSS was through punching paper cards, believe it or not!!!),

but never wrote scripts.

 

Thanks,

 

Nirit Avnimelech

 

Reply | Threaded
Open this post in threaded view
|

Re: Scripts textbook

Ruben Geert van den Berg
Dear Nirit,

SPSS® Programming and Data Management, 2nd Edition by Raynald Levesque has a long chapter on scripts and I think it includes some 28 .SBS example files as well. I'm not sure whether you can still download it, if not, let me know, I'll see if I can put it on some FTP server for you.

Best,

Ruben van den Berg
Consultant Models & Methods
TNS NIPO
Email: [hidden email]
Mobiel: +31 6 24641435
Telefoon: +31 20 522 5738
Internet: www.tns-nipo.com





Date: Tue, 17 Aug 2010 18:31:41 +0300
From: [hidden email]
Subject: Scripts textbook
To: [hidden email]

Hi,

 

Can anyone recommend a good book or online source for learning how to write scripts?  I have 30 years of experience with SPSS (yes, all you young people, my first encounter with SPSS was through punching paper cards, believe it or not!!!),

but never wrote scripts.

 

Thanks,

 

Nirit Avnimelech

 

Reply | Threaded
Open this post in threaded view
|

Re: Scripts textbook

Ben Haysom-Newport
Seems to be available at
http://www.spss.com/media/collateral/statistics/ProgDataMgmt_SPSS17.pdf
or http://www.spss.com/sites/dm-book/resources/PASW-Statistics-18-DM-Book.pdf
depending on your version. All recent versions are there.

Google found them - very useful, thanks!


--
Many thanks
Ben Haysom-Newport MSc
Regard Co-Chair (Male)
www.regard.org.uk
www.haysom-newport.com


On 17 August 2010 18:31, Ruben van den Berg
<[hidden email]> wrote:

> Dear Nirit,
> SPSS® Programming and Data Management, 2nd Edition by Raynald Levesque has a
> long chapter on scripts and I think it includes some 28 .SBS example files
> as well. I'm not sure whether you can still download it, if not, let me
> know, I'll see if I can put it on some FTP server for you.
> Best,
> Ruben van den Berg
> Consultant Models & Methods
> TNS NIPO
> Email: [hidden email]
> Mobiel: +31 6 24641435
> Telefoon: +31 20 522 5738
> Internet: www.tns-nipo.com
>
>
>
>
> ________________________________
> Date: Tue, 17 Aug 2010 18:31:41 +0300
> From: [hidden email]
> Subject: Scripts textbook
> To: [hidden email]
>
> Hi,
>
>
>
> Can anyone recommend a good book or online source for learning how to write
> scripts?  I have 30 years of experience with SPSS (yes, all you young
> people, my first encounter with SPSS was through punching paper cards,
> believe it or not!!!),
>
> but never wrote scripts.
>
>
>
> Thanks,
>
>
>
> Nirit Avnimelech
>
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Scripts textbook

Raynald Levesque
In reply to this post by Ruben Geert van den Berg
Hi Nirit,

Assuming you are using a fairly recent version of SPSS I would
recommend you learn Python instead of SaxBasic. Python is more
powerful and more fun to work with.

Regards,

Raynald Levesque
www.spsstools.net



On Tue, Aug 17, 2010 at 1:31 PM, Ruben van den Berg
<[hidden email]> wrote:

> Dear Nirit,
> SPSS® Programming and Data Management, 2nd Edition by Raynald Levesque has a
> long chapter on scripts and I think it includes some 28 .SBS example files
> as well. I'm not sure whether you can still download it, if not, let me
> know, I'll see if I can put it on some FTP server for you.
> Best,
> Ruben van den Berg
> Consultant Models & Methods
> TNS NIPO
> Email: [hidden email]
> Mobiel: +31 6 24641435
> Telefoon: +31 20 522 5738
> Internet: www.tns-nipo.com
>
>
>
>
> ________________________________
> Date: Tue, 17 Aug 2010 18:31:41 +0300
> From: [hidden email]
> Subject: Scripts textbook
> To: [hidden email]
>
> Hi,
>
>
>
> Can anyone recommend a good book or online source for learning how to write
> scripts?  I have 30 years of experience with SPSS (yes, all you young
> people, my first encounter with SPSS was through punching paper cards,
> believe it or not!!!),
>
> but never wrote scripts.
>
>
>
> Thanks,
>
>
>
> Nirit Avnimelech
>
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Scripts textbook thank you's and another question

Nirit Avnimelech
In reply to this post by Ben Haysom-Newport

Thanks, y’all!

I’m actually in the process of “migrating” to the newer version, which can be very confusing re:scripts…

Does anyone know if there’s a script that merges custom pivot tables with their corresponding comparison/significance tables?

 

Nirit

 

From: Jon K Peck [mailto:[hidden email]]
Sent: Wednesday, August 18, 2010 1:44 AM
To: [hidden email]
Cc: [hidden email]; Ben Haysom-Newport
Subject: Re: [SPSSX-L] Scripts textbook

 


If you really want the Sax Basic material, you will need a much older version of the Data Management book than the recent versions listed below.  I would second Ray Levesque's suggestion, though, to learn Python programmability and scripting instead for many reasons.

Jon Peck
SPSS, an IBM Company
[hidden email]
312-651-3435


From:

Ben Haysom-Newport <[hidden email]>

To:

[hidden email]

Date:

08/17/2010 11:46 AM

Subject:

Re: [SPSSX-L] Scripts textbook

Sent by:

"SPSSX(r) Discussion" <[hidden email]>

 





Seems to be available at
http://www.spss.com/media/collateral/statistics/ProgDataMgmt_SPSS17.pdf
or
http://www.spss.com/sites/dm-book/resources/PASW-Statistics-18-DM-Book.pdf
depending on your version. All recent versions are there.

Google found them - very useful, thanks!


--
Many thanks
Ben Haysom-Newport MSc
Regard Co-Chair (Male)
www.regard.org.uk
www.haysom-newport.com


On 17 August 2010 18:31, Ruben van den Berg
<[hidden email]> wrote:
> Dear Nirit,
> SPSS® Programming and Data Management, 2nd Edition by Raynald Levesque has a
> long chapter on scripts and I think it includes some 28 .SBS example files
> as well. I'm not sure whether you can still download it, if not, let me
> know, I'll see if I can put it on some FTP server for you.
> Best,
> Ruben van den Berg
> Consultant Models & Methods
> TNS NIPO
> Email: [hidden email]
> Mobiel: +31 6 24641435
> Telefoon: +31 20 522 5738
> Internet:
www.tns-nipo.com
>
>
>
>
> ________________________________
> Date: Tue, 17 Aug 2010 18:31:41 +0300
> From: [hidden email]
> Subject: Scripts textbook
> To: [hidden email]
>
> Hi,
>
>
>
> Can anyone recommend a good book or online source for learning how to write
> scripts?  I have 30 years of experience with SPSS (yes, all you young
> people, my first encounter with SPSS was through punching paper cards,
> believe it or not!!!),
>
> but never wrote scripts.
>
>
>
> Thanks,
>
>
>
> Nirit Avnimelech
>
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Reply | Threaded
Open this post in threaded view
|

Re: Scripts textbook thank you's and another question

Jon K Peck

Take a look at the SPSSINC MERGE TABLES extension command available from SPSS Developer Central (www.spss.com/devcentral).
Jon Peck
SPSS, an IBM Company
[hidden email]
312-651-3435



From: "Nirit" <[hidden email]>
To: Jon K Peck/Chicago/IBM@IBMUS, <[hidden email]>
Cc: <[hidden email]>, "'Ben Haysom-Newport'" <[hidden email]>
Date: 08/17/2010 06:01 PM
Subject: Scripts textbook thank you's and another question





Thanks, y’all!
I’m actually in the process of “migrating” to the newer version, which can be very confusing re:scripts…
Does anyone know if there’s a script that merges custom pivot tables with their corresponding comparison/significance tables?
 
Nirit
 
From: Jon K Peck [mailto:peck@...]
Sent:
Wednesday, August 18, 2010 1:44 AM
To:
[hidden email]
Cc:
[hidden email]; Ben Haysom-Newport
Subject:
Re: [SPSSX-L] Scripts textbook

 

If you really want the Sax Basic material, you will need a much older version of the Data Management book than the recent versions listed below.  I would second Ray Levesque's suggestion, though, to learn Python programmability and scripting instead for many reasons.


Jon Peck
SPSS, an IBM Company
[hidden email]
312-651-3435

From: Ben Haysom-Newport <[hidden email]>
To: [hidden email]
Date: 08/17/2010 11:46 AM
Subject: Re: [SPSSX-L] Scripts textbook
Sent by: "SPSSX(r) Discussion" <[hidden email]>

 






Seems to be available at

http://www.spss.com/media/collateral/statistics/ProgDataMgmt_SPSS17.pdf
or
http://www.spss.com/sites/dm-book/resources/PASW-Statistics-18-DM-Book.pdf
depending on your version. All recent versions are there.

Google found them - very useful, thanks!


--
Many thanks
Ben Haysom-Newport MSc
Regard Co-Chair (Male)

www.regard.org.uk
www.haysom-newport.com


On 17 August 2010 18:31, Ruben van den Berg
<[hidden email]> wrote:
> Dear Nirit,
> SPSS® Programming and Data Management, 2nd Edition by Raynald Levesque has a
> long chapter on scripts and I think it includes some 28 .SBS example files
> as well. I'm not sure whether you can still download it, if not, let me
> know, I'll see if I can put it on some FTP server for you.
> Best,
> Ruben van den Berg
> Consultant Models & Methods
> TNS NIPO
> Email: [hidden email]
> Mobiel: +31 6 24641435
> Telefoon: +31 20 522 5738
> Internet:
www.tns-nipo.com
>
>
>
>
> ________________________________
> Date: Tue, 17 Aug 2010 18:31:41 +0300
> From: [hidden email]
> Subject: Scripts textbook
> To: [hidden email]
>
> Hi,
>
>
>
> Can anyone recommend a good book or online source for learning how to write
> scripts?  I have 30 years of experience with SPSS (yes, all you young
> people, my first encounter with SPSS was through punching paper cards,
> believe it or not!!!),
>
> but never wrote scripts.
>
>
>
> Thanks,
>
>
>
> Nirit Avnimelech
>
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


Reply | Threaded
Open this post in threaded view
|

Re: deleting rows from CTABLES based on data values

Kylie
In reply to this post by Jon K Peck

Thanks Jon – looks like my time of avoiding Python is over! :)

 

However, I am having problems getting everything installed. I have SPSS 17.0.3 (on Vista) so I installed Python 2.5 (from the SPSS installation CD), and PASWStatisticsPythonPlugInWin32_1702 from DevCentral (there was no option for 17.0.3 so I selected 17.0.2 instead – was this correct?). I downloaded spssinc_modify_tables.zip from the Download Library and extracted it into the /extensions folder within the SPSS installation folder. Within SPSS I was able to add it to the Utilities menu via the ‘Install Custom Dialog’ option (though I had to run SPSS as administrator for this to work). ‘Modify Table Appearance’ is now listed in the Utilities menu.

 

However, when I try to run either Jon’s command or just the basic <SPSSINC MODIFY TABLES /HELP.> provided in the readme file, I get the following error:

 

>Error # 6887.  Command name: BEGIN PROGRAM

>External program failed during initialization.

>This command not executed.

 

I had a look in the Knowledgebase but none of the entries for this error number seemed relevant. I’m guessing I still need to install some more Python modules or something, but I couldn’t see reference to any other dependencies in the documentation. Pretty sure I’ve got the versions all matched up? Is admin rights needed? I tried again while running SPSS as admin, and got the same error.

 

Any suggestions of what I’m missing would be appreciated.

 

Thanks,

Kylie.

 

 

From: Jon K Peck [mailto:[hidden email]]
Sent: Wednesday, 18 August 2010 12:55 AM
To: [hidden email]
Cc: [hidden email]
Subject: Re: [SPSSX-L] deleting rows from CTABLES based on data values

 


See below.

Jon Peck
SPSS, an IBM Company
[hidden email]
312-651-3435


From:

Kylie Lange <[hidden email]>

To:

[hidden email]

Date:

08/17/2010 03:03 AM

Subject:

[SPSSX-L] deleting rows from CTABLES based on data values

Sent by:

"SPSSX(r) Discussion" <[hidden email]>

 





Hi all,

I am after some help with what I believe will be a script (though I could be
wrong!). For a current project we will be generating ~300 custom tables with
the following structure:

CTABLES
 /VLABELS VARIABLES=majorgrp agegroup sex nut1 DISPLAY=DEFAULT
 /TABLE majorgrp BY agegroup > sex > nut1 [COLPCT.SUM PCT40.1]
 /CATEGORIES VARIABLES=majorgrp ORDER=A KEY=VALUE EMPTY=INCLUDE TOTAL=YES
POSITION=AFTER
 /CATEGORIES VARIABLES=agegroup sex ORDER=A KEY=VALUE EMPTY=INCLUDE.

Ie, one row per 'majorgrp', with columns by age and sex, and the data cells
containing column percentages.

We need to be able to delete any rows from the table where all values are
less than 1.5%. The actual row should be deleted (including row labels), not
just the data values cleared from the cells.

My best guess is that this will require a script, but I couldn't find
anything online that was close enough to give me a starting point. Does
anyone have any suggestions or alternative ways to approach this? Something
that could be run over all pivot tables in an output file would be great.


>>>This can be done with the SPSSINC MODIFY TABLES extension command available from SPSS Developer Central (www.spss.com/devcentral).  It requires at least V17.0.2 and the Python plugin, although it will work better and faster with V19.

There is a new custom function for it named HideRowBasedOnValues available with the latest version.  Here is an example that hides rows where all the values are less than 5.  If you have row totals and want to exclude them from the test, there are additional parameters for that.

This command is set to process all the tables of the specified type in the Viewer, so you could just specify it at the end of a bunch of CTABLES commands.
More details are in the customstylefunctions module included in the zip.

CTABLES
  /TABLE educ BY gender > jobcat [COLPCT.COUNT].

SPSSINC MODIFY TABLES subtype="Custom Table"
SELECT="<<ALL>>" DIMENSION= ROWS LEVEL = -1 PROCESS = ALL
/STYLES  APPLYTO=LABELS
CUSTOMFUNCTION="customstylefunctions.HideRowBasedOnValues(threshold=5)".

This function doesn't actually delete the rows - it hides them.

HTH,
Jon Peck



Thanks,
Kylie.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Reply | Threaded
Open this post in threaded view
|

Re: deleting rows from CTABLES based on data values

Kylie
In reply to this post by Jon K Peck

Sorry to repeat post, but I found an old email in the list archives that listed a few things to check. Here are my results:

 

- Be sure you have installed exactly the right plug-in corresponding to your SPSS version. [KL: I am guessing this is where I have gone wrong. I have 17.0.3 but there is no plugin listed for this version, so I installed the 17.0.2 plugin.]

- Check the file spssdxcfg.ini in the SPSS installation directory. It should have the HOME variable set to the Python location [KL: Yes] and the spssdx_version variable matching your SPSS version [KL: I’m not sure. I find this: spssdx_version_major=17.0 spssdx_version_patch=2 spssdx_version_bugfix=[Python]]

- Check that you can actually run Python directly [KL: Yes, I can run the Command Line]

- Try running SPSS from Python by opening a Python command line and running something like <import spss /  spss.Submit(“show dir”)>. [KL: No, entering <import spss> returns the error: NameError: name ‘spss’ is not defined]

 

Hope this gives someone else some clues.

 

Thanks,

Kylie.

 

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Kylie Lange
Sent: Wednesday, 18 August 2010 4:06 PM
To: [hidden email]
Subject: Re: deleting rows from CTABLES based on data values

 

Thanks Jon – looks like my time of avoiding Python is over! :)

 

However, I am having problems getting everything installed. I have SPSS 17.0.3 (on Vista) so I installed Python 2.5 (from the SPSS installation CD), and PASWStatisticsPythonPlugInWin32_1702 from DevCentral (there was no option for 17.0.3 so I selected 17.0.2 instead – was this correct?). I downloaded spssinc_modify_tables.zip from the Download Library and extracted it into the /extensions folder within the SPSS installation folder. Within SPSS I was able to add it to the Utilities menu via the ‘Install Custom Dialog’ option (though I had to run SPSS as administrator for this to work). ‘Modify Table Appearance’ is now listed in the Utilities menu.

 

However, when I try to run either Jon’s command or just the basic <SPSSINC MODIFY TABLES /HELP.> provided in the readme file, I get the following error:

 

>Error # 6887.  Command name: BEGIN PROGRAM

>External program failed during initialization.

>This command not executed.

 

I had a look in the Knowledgebase but none of the entries for this error number seemed relevant. I’m guessing I still need to install some more Python modules or something, but I couldn’t see reference to any other dependencies in the documentation. Pretty sure I’ve got the versions all matched up? Is admin rights needed? I tried again while running SPSS as admin, and got the same error.

 

Any suggestions of what I’m missing would be appreciated.

 

Thanks,

Kylie.

 

 

From: Jon K Peck [mailto:[hidden email]]
Sent: Wednesday, 18 August 2010 12:55 AM
To: [hidden email]
Cc: [hidden email]
Subject: Re: [SPSSX-L] deleting rows from CTABLES based on data values

 


See below.

Jon Peck
SPSS, an IBM Company
[hidden email]
312-651-3435

From:

Kylie Lange <[hidden email]>

To:

[hidden email]

Date:

08/17/2010 03:03 AM

Subject:

[SPSSX-L] deleting rows from CTABLES based on data values

Sent by:

"SPSSX(r) Discussion" <[hidden email]>

 





Hi all,

I am after some help with what I believe will be a script (though I could be
wrong!). For a current project we will be generating ~300 custom tables with
the following structure:

CTABLES
 /VLABELS VARIABLES=majorgrp agegroup sex nut1 DISPLAY=DEFAULT
 /TABLE majorgrp BY agegroup > sex > nut1 [COLPCT.SUM PCT40.1]
 /CATEGORIES VARIABLES=majorgrp ORDER=A KEY=VALUE EMPTY=INCLUDE TOTAL=YES
POSITION=AFTER
 /CATEGORIES VARIABLES=agegroup sex ORDER=A KEY=VALUE EMPTY=INCLUDE.

Ie, one row per 'majorgrp', with columns by age and sex, and the data cells
containing column percentages.

We need to be able to delete any rows from the table where all values are
less than 1.5%. The actual row should be deleted (including row labels), not
just the data values cleared from the cells.

My best guess is that this will require a script, but I couldn't find
anything online that was close enough to give me a starting point. Does
anyone have any suggestions or alternative ways to approach this? Something
that could be run over all pivot tables in an output file would be great.


>>>This can be done with the SPSSINC MODIFY TABLES extension command available from SPSS Developer Central (www.spss.com/devcentral).  It requires at least V17.0.2 and the Python plugin, although it will work better and faster with V19.

There is a new custom function for it named HideRowBasedOnValues available with the latest version.  Here is an example that hides rows where all the values are less than 5.  If you have row totals and want to exclude them from the test, there are additional parameters for that.

This command is set to process all the tables of the specified type in the Viewer, so you could just specify it at the end of a bunch of CTABLES commands.
More details are in the customstylefunctions module included in the zip.

CTABLES
  /TABLE educ BY gender > jobcat [COLPCT.COUNT].

SPSSINC MODIFY TABLES subtype="Custom Table"
SELECT="<<ALL>>" DIMENSION= ROWS LEVEL = -1 PROCESS = ALL
/STYLES  APPLYTO=LABELS
CUSTOMFUNCTION="customstylefunctions.HideRowBasedOnValues(threshold=5)".

This function doesn't actually delete the rows - it hides them.

HTH,
Jon Peck



Thanks,
Kylie.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Reply | Threaded
Open this post in threaded view
|

Re: deleting rows from CTABLES based on data values

Jon K Peck

Jon Peck
SPSS, an IBM Company
[hidden email]
312-651-3435



From: "Kylie Lange" <[hidden email]>
To: Jon K Peck/Chicago/IBM@IBMUS, <[hidden email]>
Date: 08/18/2010 01:03 AM
Subject: RE: deleting rows from CTABLES based on data values





Sorry to repeat post, but I found an old email in the list archives that listed a few things to check. Here are my results:
 
- Be sure you have installed exactly the right plug-in corresponding to your SPSS version. [KL: I am guessing this is where I have gone wrong. I have 17.0.3 but there is no plugin listed for this version, so I installed the 17.0.2 plugin.]
>>>That is correct.  No patch was necessary for 17.0.3

- Check the file spssdxcfg.ini in the SPSS installation directory. It should have the HOME variable set to the Python location [KL: Yes] and the spssdx_version variable matching your SPSS version [KL: I’m not sure. I find this: spssdx_version_major=17.0 spssdx_version_patch=2 spssdx_version_bugfix=[Python]]
>>>Also ok
- Check that you can actually run Python directly [KL: Yes, I can run the Command Line]
- Try running SPSS from Python by opening a Python command line and running something like <import spss /  spss.Submit(“show dir”)>. [KL: No, entering <import spss> returns the error: NameError: name ‘spss’ is not defined]
>>>This indicates that the plugin is not installed correctly, which is consistent with the original error message you reported.  Since you are on Vista, which is quirky - to say the least - about permissions, my guess is that Vista diverted the plugin files to somewhere inappropriate.  Try running the plugin install via Run As Administrator.  Otherwise it may not have write access to the necessary directories (Vista can silently divert the writes to a different location).

HTH,
Jon
 
Hope this gives someone else some clues.
 
Thanks,
Kylie.
 
 
From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Kylie Lange
Sent:
Wednesday, 18 August 2010 4:06 PM
To:
[hidden email]
Subject:
Re: deleting rows from CTABLES based on data values

 
Thanks Jon – looks like my time of avoiding Python is over! :)
 
However, I am having problems getting everything installed. I have SPSS 17.0.3 (on Vista) so I installed Python 2.5 (from the SPSS installation CD), and PASWStatisticsPythonPlugInWin32_1702 from DevCentral (there was no option for 17.0.3 so I selected 17.0.2 instead – was this correct?). I downloaded spssinc_modify_tables.zip from the Download Library and extracted it into the /extensions folder within the SPSS installation folder. Within SPSS I was able to add it to the Utilities menu via the ‘Install Custom Dialog’ option (though I had to run SPSS as administrator for this to work). ‘Modify Table Appearance’ is now listed in the Utilities menu.
 
However, when I try to run either Jon’s command or just the basic <SPSSINC MODIFY TABLES /HELP.> provided in the readme file, I get the following error:
 
>Error # 6887.  Command name: BEGIN PROGRAM
>External program failed during initialization.
>This command not executed.
 
I had a look in the Knowledgebase but none of the entries for this error number seemed relevant. I’m guessing I still need to install some more Python modules or something, but I couldn’t see reference to any other dependencies in the documentation. Pretty sure I’ve got the versions all matched up? Is admin rights needed? I tried again while running SPSS as admin, and got the same error.
 
Any suggestions of what I’m missing would be appreciated.
 
Thanks,
Kylie.
 
 
From: Jon K Peck [mailto:peck@...]
Sent:
Wednesday, 18 August 2010 12:55 AM
To:
[hidden email]
Cc:
[hidden email]
Subject:
Re: [SPSSX-L] deleting rows from CTABLES based on data values

 

See below.


Jon Peck
SPSS, an IBM Company
[hidden email]
312-651-3435

From: Kylie Lange <[hidden email]>
To: [hidden email]
Date: 08/17/2010 03:03 AM
Subject: [SPSSX-L] deleting rows from CTABLES based on data values
Sent by: "SPSSX(r) Discussion" <[hidden email]>

 






Hi all,

I am after some help with what I believe will be a script (though I could be
wrong!). For a current project we will be generating ~300 custom tables with
the following structure:

CTABLES
/VLABELS VARIABLES=majorgrp agegroup sex nut1 DISPLAY=DEFAULT
/TABLE majorgrp BY agegroup > sex > nut1 [COLPCT.SUM PCT40.1]
/CATEGORIES VARIABLES=majorgrp ORDER=A KEY=VALUE EMPTY=INCLUDE TOTAL=YES
POSITION=AFTER
/CATEGORIES VARIABLES=agegroup sex ORDER=A KEY=VALUE EMPTY=INCLUDE.

Ie, one row per 'majorgrp', with columns by age and sex, and the data cells
containing column percentages.

We need to be able to delete any rows from the table where all values are
less than 1.5%. The actual row should be deleted (including row labels), not
just the data values cleared from the cells.

My best guess is that this will require a script, but I couldn't find
anything online that was close enough to give me a starting point. Does
anyone have any suggestions or alternative ways to approach this? Something
that could be run over all pivot tables in an output file would be great.


>>>This can be done with the SPSSINC MODIFY TABLES extension command available from SPSS Developer Central (
www.spss.com/devcentral).  It requires at least V17.0.2 and the Python plugin, although it will work better and faster with V19.

There is a new custom function for it named HideRowBasedOnValues available with the latest version.  Here is an example that hides rows where all the values are less than 5.  If you have row totals and want to exclude them from the test, there are additional parameters for that.


This command is set to process all the tables of the specified type in the Viewer, so you could just specify it at the end of a bunch of CTABLES commands.

More details are in the customstylefunctions module included in the zip.


CTABLES

 /TABLE educ BY gender > jobcat [COLPCT.COUNT].


SPSSINC MODIFY TABLES subtype="Custom Table"

SELECT="<<ALL>>" DIMENSION= ROWS LEVEL = -1 PROCESS = ALL
/STYLES  APPLYTO=LABELS
CUSTOMFUNCTION="customstylefunctions.HideRowBasedOnValues(threshold=5)".


This function doesn't actually delete the rows - it hides them.


HTH,

Jon Peck




Thanks,
Kylie.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


Reply | Threaded
Open this post in threaded view
|

Re: deleting rows from CTABLES based on data values

Kylie
Yes, that was it. I reinstalled the Python plugin as administrator and can now
run the extension commands successfully.

Thanks alot for your help Jon.

Cheers,
Kylie.


Quoting Jon K Peck <[hidden email]>:

> Jon Peck
> SPSS, an IBM Company
> [hidden email]
> 312-651-3435
>
>
>
> From:
> "Kylie Lange" <[hidden email]>
> To:
> Jon K Peck/Chicago/IBM@IBMUS, <[hidden email]>
> Date:
> 08/18/2010 01:03 AM
> Subject:
> RE: deleting rows from CTABLES based on data values
>
>
>
> Sorry to repeat post, but I found an old email in the list archives that
> listed a few things to check. Here are my results:
>
> - Be sure you have installed exactly the right plug-in corresponding to
> your SPSS version. [KL: I am guessing this is where I have gone wrong. I
> have 17.0.3 but there is no plugin listed for this version, so I installed
> the 17.0.2 plugin.]
> >>>That is correct.  No patch was necessary for 17.0.3
>
> - Check the file spssdxcfg.ini in the SPSS installation directory. It
> should have the HOME variable set to the Python location [KL: Yes] and the
> spssdx_version variable matching your SPSS version [KL: I?m not sure. I
> find this: spssdx_version_major=17.0 spssdx_version_patch=2
> spssdx_version_bugfix=[Python]]
> >>>Also ok
> - Check that you can actually run Python directly [KL: Yes, I can run the
> Command Line]
> - Try running SPSS from Python by opening a Python command line and
> running something like <import spss /  spss.Submit(?show dir?)>. [KL: No,
> entering <import spss> returns the error: NameError: name ?spss? is not
> defined]
> >>>This indicates that the plugin is not installed correctly, which is
> consistent with the original error message you reported.  Since you are on
> Vista, which is quirky - to say the least - about permissions, my guess is
> that Vista diverted the plugin files to somewhere inappropriate.  Try
> running the plugin install via Run As Administrator.  Otherwise it may not
> have write access to the necessary directories (Vista can silently divert
> the writes to a different location).
>
> HTH,
> Jon
>
> Hope this gives someone else some clues.
>
> Thanks,
> Kylie.
>
>
> From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
> Kylie Lange
> Sent: Wednesday, 18 August 2010 4:06 PM
> To: [hidden email]
> Subject: Re: deleting rows from CTABLES based on data values
>
> Thanks Jon ? looks like my time of avoiding Python is over! :)
>
> However, I am having problems getting everything installed. I have SPSS
> 17.0.3 (on Vista) so I installed Python 2.5 (from the SPSS installation
> CD), and PASWStatisticsPythonPlugInWin32_1702 from DevCentral (there was
> no option for 17.0.3 so I selected 17.0.2 instead ? was this correct?). I
> downloaded spssinc_modify_tables.zip from the Download Library and
> extracted it into the /extensions folder within the SPSS installation
> folder. Within SPSS I was able to add it to the Utilities menu via the
> ?Install Custom Dialog? option (though I had to run SPSS as administrator
> for this to work). ?Modify Table Appearance? is now listed in the
> Utilities menu.
>
> However, when I try to run either Jon?s command or just the basic <SPSSINC
> MODIFY TABLES /HELP.> provided in the readme file, I get the following
> error:
>
> >Error # 6887.  Command name: BEGIN PROGRAM
> >External program failed during initialization.
> >This command not executed.
>
> I had a look in the Knowledgebase but none of the entries for this error
> number seemed relevant. I?m guessing I still need to install some more
> Python modules or something, but I couldn?t see reference to any other
> dependencies in the documentation. Pretty sure I?ve got the versions all
> matched up? Is admin rights needed? I tried again while running SPSS as
> admin, and got the same error.
>
> Any suggestions of what I?m missing would be appreciated.
>
> Thanks,
> Kylie.
>
>
> From: Jon K Peck [mailto:[hidden email]]
> Sent: Wednesday, 18 August 2010 12:55 AM
> To: [hidden email]
> Cc: [hidden email]
> Subject: Re: [SPSSX-L] deleting rows from CTABLES based on data values
>
>
> See below.
>
> Jon Peck
> SPSS, an IBM Company
> [hidden email]
> 312-651-3435
>
> From:
> Kylie Lange <[hidden email]>
> To:
> [hidden email]
> Date:
> 08/17/2010 03:03 AM
> Subject:
> [SPSSX-L] deleting rows from CTABLES based on data values
> Sent by:
> "SPSSX(r) Discussion" <[hidden email]>
>
>
>
>
>
> Hi all,
>
> I am after some help with what I believe will be a script (though I could
> be
> wrong!). For a current project we will be generating ~300 custom tables
> with
> the following structure:
>
> CTABLES
>  /VLABELS VARIABLES=majorgrp agegroup sex nut1 DISPLAY=DEFAULT
>  /TABLE majorgrp BY agegroup > sex > nut1 [COLPCT.SUM PCT40.1]
>  /CATEGORIES VARIABLES=majorgrp ORDER=A KEY=VALUE EMPTY=INCLUDE TOTAL=YES
> POSITION=AFTER
>  /CATEGORIES VARIABLES=agegroup sex ORDER=A KEY=VALUE EMPTY=INCLUDE.
>
> Ie, one row per 'majorgrp', with columns by age and sex, and the data
> cells
> containing column percentages.
>
> We need to be able to delete any rows from the table where all values are
> less than 1.5%. The actual row should be deleted (including row labels),
> not
> just the data values cleared from the cells.
>
> My best guess is that this will require a script, but I couldn't find
> anything online that was close enough to give me a starting point. Does
> anyone have any suggestions or alternative ways to approach this?
> Something
> that could be run over all pivot tables in an output file would be great.
>
> >>>This can be done with the SPSSINC MODIFY TABLES extension command
> available from SPSS Developer Central (www.spss.com/devcentral).  It
> requires at least V17.0.2 and the Python plugin, although it will work
> better and faster with V19.
>
> There is a new custom function for it named HideRowBasedOnValues available
> with the latest version.  Here is an example that hides rows where all the
> values are less than 5.  If you have row totals and want to exclude them
> from the test, there are additional parameters for that.
>
> This command is set to process all the tables of the specified type in the
> Viewer, so you could just specify it at the end of a bunch of CTABLES
> commands.
> More details are in the customstylefunctions module included in the zip.
>
> CTABLES
>   /TABLE educ BY gender > jobcat [COLPCT.COUNT].
>
> SPSSINC MODIFY TABLES subtype="Custom Table"
> SELECT="<<ALL>>" DIMENSION= ROWS LEVEL = -1 PROCESS = ALL
> /STYLES  APPLYTO=LABELS
> CUSTOMFUNCTION="customstylefunctions.HideRowBasedOnValues(threshold=5)".
>
> This function doesn't actually delete the rows - it hides them.
>
> HTH,
> Jon Peck
>
>
>
> Thanks,
> Kylie.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD