Re: AW: 'DELETE VARIABLES' impossibly slow - is my datafile 'corrupted'?

[Albert-Jan Roskam]

In Python you can use a set to deduplicate your list of vars, then feed that list as a string into DELETE VARIABLES or ADD FILES.

For instance:
L1 = "id gender bdate educ jobcat salary salbegin jobtime prevexp".split()
L2 = "bdate educ jobcat salary salbegin jobtime prevexp minority".split()
L3 = set(L1 + L2)
spss.Submit("add files / file = * / drop = %s." % " ".join(L3))

Or if you have two data files and you want to chuck away all vars that are not present in both files, you could use sets to get the intersection between two variable lists
spss.Submit("dataset activate this_one.")
L1 = spssaux.GetVariableNamesList()
spss.Submit("dataset activate that_one.")
L2 = spssaux.GetVariableNamesList()
spss.Submit("add files / file = * / drop = %s." % " ".join(set(L1).intersection(set(L2))) )

Cheers!!
Albert-Jan


--- On Thu, 8/27/09, Ruben van den Berg <[hidden email]> wrote:

> From: Ruben van den Berg <[hidden email]>
> Subject: Re: [SPSSX-L] AW:      'DELETE VARIABLES' impossibly slow - is my              datafile              'corrupted'?
> To: [hidden email]
> Date: Thursday, August 27, 2009, 4:35 PM
>
>
>
> #yiv1731997074 .hmmessage P
> {
> margin:0px;padding:0px;}
> #yiv1731997074 {
> font-size:10pt;font-family:Verdana;}
>
>
>
> Dear Gene (and other LISTers of course,
>
>
>
> Even if I could, I wouldn't want to
> delete any variables that no longer exist. I just want
> to delete a number of variables that are not
> relevant. Unfortunately, the irrelevant variable
> names reside in two data sources which overlap.
>
>
>
> I'm not sure which version you use but neither in my
> V14 nor in my V15 are deleted variables greyed out, they
> really disappear -like magic!
>
>
>
> If I convert my two lists of irrelevant variables names
> into 2 syntax files containing a separate DEL VAR
> command for each variable, duplicate DEL VAR commands
> generate error messages but every variable I'd like
> to remove will nevertheless be removed from the data file
> -that's why I tried that first. In theory this was the
> fastest way to achieve my goal and it should work
> fine but in practice, SPSS just got stuck or
> something.
>
>
>
> Fortunately, I've already worked around the problem.
>
>
>
> Kind regards,
>
>
>
> Ruben van den Berg
>
> P.s. difficulties like these may seem strange to some
> people but I often have to work with large, messy
> datafiles and cleaning these files up is actually
> quite a bit of work sometimes.
>
>
>
>
>
>
> > Date: Thu, 27 Aug 2009 09:43:54 -0400
> > From: [hidden email]
> > Subject: Re: AW: 'DELETE VARIABLES' impossibly
> slow - is my datafile 'corrupted'?
> > To: [hidden email]
> >
> > I want to yell out questions from the back bench. To
> Ruben: Why are you
> > trying to delete variables that no longer exist?
> (Although, hasn't everybody
> > who has used delete variables noticed that the names
> of the variables
> > deleted are simply 'grayed out' in the data
> file rather than removed as the
> > data is removed. This seems pretty odd to me.)
> >
> > To spss. When somebody tries to delete an already
> deleted variable why
> > doesn't the program generate an error. In theory
> the variables no longer
> > exist and, i'd assume that the command processor
> checks specified variables
> > names against the variable name vector before
> commencing the command. Or, do
> > the names still exist somewhere?
> >
> >
> > Gene Maguin
> >
> > =====================
> > To manage your subscription to SPSSX-L, send a message
> to
> > [hidden email] (not to SPSSX-L), with no
> body text except the
> > command. To leave the list, send the command
> > SIGNOFF SPSSX-L
> > For a list of commands to manage subscriptions, send
> the command
> > INFO REFCARD
>
> What can you do with the new Windows Live? Find
> out
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD