Deleting Variables with Missings Values

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Deleting Variables with Missings Values

Alice
Hi Jon,

Thank you so much. I have tried this way and it works perfectly.

However, I have another concern with separating their values by a blank
space or a ";".

I have tried with below syntax but it does not work. I think this should
have to be solved by a little Python code, right?

SPSSINC SELECT VARIABLES MACRONAME="!test"
/PROPERTIES PATTERN = "Qad_\d{5}_Resp\d{2}"
/OPTIONS ORDER=FILE REVERSE=NO  PRINT=YES SEPARATOR=",".

COMPUTE finalResult = CONCAT(!test, ";").



Jon Peck wrote

> In the macro generated by SELECT VARIABLES the variables are already
> concatenated with a blank separator (or other separator of your choice) as
> you can see from your FREQUENCIES command.
> So, if you specify a comma as the separator, you can just write a compute
> with
> CONCAT(!test)
> assuming that the variables are all strings and you declare the result
> with
> the STRING command.
>
> On Fri, Jul 27, 2018 at 9:55 PM Alice <

> thuyminhs@

> > wrote:
>
>> Hi Jon,
>>
>> Thank you for your clarification. I'm using SPSS 24 and the spssaux2
>> module
>> is already in the Python site-packages directory now. It also has a lot
>> of
>> useful functions as you said.
>>
>> However, when I run the FindEmptyVariables function with a data set with
>> more than 21000 variables, the SPSS takes a lot of time processing the
>> command AGGREGATE, more than 15 minutes. So I have tried another solution
>> by
>> dividing the data set into each variable group (around 2000 variables for
>> each) before running this function and it works really fast.
>>
>> I have also tried with the command SPSSINC SELECT VARIABLES below and
>> have
>> troubled with how I can loop through each item in the resulted macro and
>> CONCAT all their values? I can do this by using a little bit Python code
>> as
>> well but I prefer using SPSS syntax since my other non-IT background team
>> members can understand it easily. Do you have any idea about this?
>>
>> *SPSS:*
>>
>> SPSSINC SELECT VARIABLES MACRONAME="!test"
>> /PROPERTIES PATTERN = "Qad_\d{5}_Resp\d{2}"
>> /OPTIONS ORDER=FILE REVERSE=NO  PRINT=YES SEPARATOR=" ".
>> FREQUENCIES !test.
>>
>> *Python:*
>>
>> BEGIN PROGRAM.
>> import spss, re
>>
>> varlist = []
>> for ind in range(spss.GetVariableCount()):
>>     varName = spss.GetVariableName(ind)
>>     if re.match("Qad_\d{5}_Resp\d{2}",varName):
>>         varlist.append(varName)
>>
>> # Concat each value in the varlist here...
>>
>> END PROGRAM.
>>
>>
>>





--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Deleting Variables with Missings Values

Jon Peck
I don't know what actually happened from "it does not work".

You need to declare your finalResult variable as a string before running the COMPUTE with sufficient width to hold the concatenated values, e.g.
STRING finalResult(A100).

Also you should not include the ";" in the formula.  It would be just
CONCAT(!test)
unless you have some other purpose for a semicolon at the end.

If you run SET MPRINT ON. before running the COMPUTE, you can see exactly how the macro was expanded.

On Sun, Jul 29, 2018 at 1:53 AM Alice <[hidden email]> wrote:
Hi Jon,

Thank you so much. I have tried this way and it works perfectly.

However, I have another concern with separating their values by a blank
space or a ";".

I have tried with below syntax but it does not work. I think this should
have to be solved by a little Python code, right?

SPSSINC SELECT VARIABLES MACRONAME="!test"
/PROPERTIES PATTERN = "Qad_\d{5}_Resp\d{2}"
/OPTIONS ORDER=FILE REVERSE=NO  PRINT=YES SEPARATOR=",".

COMPUTE finalResult = CONCAT(!test, ";").



Jon Peck wrote
> In the macro generated by SELECT VARIABLES the variables are already
> concatenated with a blank separator (or other separator of your choice) as
> you can see from your FREQUENCIES command.
> So, if you specify a comma as the separator, you can just write a compute
> with
> CONCAT(!test)
> assuming that the variables are all strings and you declare the result
> with
> the STRING command.
>
> On Fri, Jul 27, 2018 at 9:55 PM Alice &lt;

> thuyminhs@

> &gt; wrote:
>
>> Hi Jon,
>>
>> Thank you for your clarification. I'm using SPSS 24 and the spssaux2
>> module
>> is already in the Python site-packages directory now. It also has a lot
>> of
>> useful functions as you said.
>>
>> However, when I run the FindEmptyVariables function with a data set with
>> more than 21000 variables, the SPSS takes a lot of time processing the
>> command AGGREGATE, more than 15 minutes. So I have tried another solution
>> by
>> dividing the data set into each variable group (around 2000 variables for
>> each) before running this function and it works really fast.
>>
>> I have also tried with the command SPSSINC SELECT VARIABLES below and
>> have
>> troubled with how I can loop through each item in the resulted macro and
>> CONCAT all their values? I can do this by using a little bit Python code
>> as
>> well but I prefer using SPSS syntax since my other non-IT background team
>> members can understand it easily. Do you have any idea about this?
>>
>> *SPSS:*
>>
>> SPSSINC SELECT VARIABLES MACRONAME="!test"
>> /PROPERTIES PATTERN = "Qad_\d{5}_Resp\d{2}"
>> /OPTIONS ORDER=FILE REVERSE=NO  PRINT=YES SEPARATOR=" ".
>> FREQUENCIES !test.
>>
>> *Python:*
>>
>> BEGIN PROGRAM.
>> import spss, re
>>
>> varlist = []
>> for ind in range(spss.GetVariableCount()):
>>     varName = spss.GetVariableName(ind)
>>     if re.match("Qad_\d{5}_Resp\d{2}",varName):
>>         varlist.append(varName)
>>
>> # Concat each value in the varlist here...
>>
>> END PROGRAM.
>>
>>
>>





--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Deleting Variables with Missings Values

Alice
Hi Jon,

Below is my whole syntax for this and it works perfectly, except for the
CONCAT command. I want to compute the finalResult by concatenating all the
values of variables returned from the macro !test, including a semicolon
among each value. I meant the finalResult should be
"value1;value2;value3;value4;" instead of "value1value2value3value4".

With this syntax, the macro was expanded to:

COMPUTE finalResult = CONCAT(var1,var2, var3, var4,";").

NOT as my expectation below. How we can do this by using SPSS syntax for the
returned macro? Hope it is clearer now.

COMPUTE finalResult = CONCAT(var1,";",var2,";", var3,";", var4,";").

*The whole syntax:*

STRING finalResult (A500).

SPSSINC SELECT VARIABLES MACRONAME="!test"
/PROPERTIES PATTERN = "Qad_\d{5}_Resp\d{2}"
/OPTIONS ORDER=FILE REVERSE=NO  PRINT=YES IFNONE=ERROR SEPARATOR=",".

COMPUTE finalResult = CONCAT(!test,";").
EXECUTE.



--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Deleting Variables with Missings Values

Jon Peck
Ok.  I didn't realize that you wanted to inject another character between the values.

You can include the semicolon in the separator with quotes.  It is  not limited to a single character.
SPSSINC SELECT VARIABLES MACRONAME="!mac" VARIABLES
/OPTIONS ORDER=ALPHA REVERSE=NO  PRINT=YES IFNONE=ERROR SEPARATOR=",';',".
compute allvalues = concat(!mac).

On Sun, Jul 29, 2018 at 9:13 PM Alice <[hidden email]> wrote:
Hi Jon,

Below is my whole syntax for this and it works perfectly, except for the
CONCAT command. I want to compute the finalResult by concatenating all the
values of variables returned from the macro !test, including a semicolon
among each value. I meant the finalResult should be
"value1;value2;value3;value4;" instead of "value1value2value3value4".

With this syntax, the macro was expanded to:

COMPUTE finalResult = CONCAT(var1,var2, var3, var4,";").

NOT as my expectation below. How we can do this by using SPSS syntax for the
returned macro? Hope it is clearer now.

COMPUTE finalResult = CONCAT(var1,";",var2,";", var3,";", var4,";").

*The whole syntax:*

STRING finalResult (A500).

SPSSINC SELECT VARIABLES MACRONAME="!test"
/PROPERTIES PATTERN = "Qad_\d{5}_Resp\d{2}"
/OPTIONS ORDER=FILE REVERSE=NO  PRINT=YES IFNONE=ERROR SEPARATOR=",".

COMPUTE finalResult = CONCAT(!test,";").
EXECUTE.



--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Deleting Variables with Missings Values

Alice
This post was updated on .
Thank you so much, Jon. It's so easy and flexible, and also help reduce a lot
of my time processing data.

Jon Peck wrote
Ok.  I didn't realize that you wanted to inject another character between
the values.

You can include the semicolon in the separator with quotes.  It is  not
limited to a single character.
SPSSINC SELECT VARIABLES MACRONAME="!mac" VARIABLES
/OPTIONS ORDER=ALPHA REVERSE=NO  PRINT=YES IFNONE=ERROR SEPARATOR=",';',".
compute allvalues = concat(!mac).
Reply | Threaded
Open this post in threaded view
|

Re: Deleting Variables with Missings Values

Alice
In reply to this post by Jon Peck
Hi Jon,

I think there might be a bug with the command SPSSINC SELECT VARIABLES when
I want to inject a blank space between the values instead of a ";".

Since my data set has a lot of variables match the rule
"Q18ad_\d{5}_Resp\d{2}", so the returned macro will contain some new line
characters (\n) between variables, leading the error in the line 67
(attached file  SPSSINC_SELECT_VARIABLES_.txt
<http://spssx-discussion.1045642.n5.nabble.com/file/t341476/SPSSINC_SELECT_VARIABLES_.txt>
). If I use the SEPARATOR=",*';'*," instead of SEPARATOR=",*' '*,", it works
fine.

I would appreciate if you can take a look at this.

*Below is my syntax.*

STRING Q18d_Group (A200).

* Select all Q18ad_xxxxx_Respxx variables.
SPSSINC SELECT VARIABLES MACRONAME="!Q18d_Group"
/PROPERTIES PATTERN = "Q18ad_\d{5}_Resp\d{2}"
/OPTIONS ORDER=FILE IFNONE=ERROR.

* Format and convert Q18ad_Group to string.
FORMATS !Q18d_Group (N2).
ALTER TYPE !Q18d_Group (A2).

* Concatenate their values with a blank space between ech value.
SPSSINC SELECT VARIABLES MACRONAME="!Q18d_Group"
/PROPERTIES PATTERN = "Q18ad_\d{5}_Resp\d{2}"
/OPTIONS ORDER=FILE IFNONE=ERROR SEPARATOR=",' ',".

SET MPRINT ON.
COMPUTE Q18d_Group = CONCAT(!Q18d_Group).



Jon Peck wrote
> Ok.  I didn't realize that you wanted to inject another character between
> the values.
>
> You can include the semicolon in the separator with quotes.  It is  not
> limited to a single character.
> SPSSINC SELECT VARIABLES MACRONAME="!mac" VARIABLES
> /OPTIONS ORDER=ALPHA REVERSE=NO  PRINT=YES IFNONE=ERROR SEPARATOR=",';',".
> compute allvalues = concat(!mac).





--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
12