SYSMIS in ANY?

[Robert L]

I just tried to set up a function to find those cases where there is one or more missing values in order to exclude them. My first thought was that these could be filtered out with the following variable:

COMPUTE exclude=ANY($SYSMIS,v1,v2,v3)

It did not work. However, it worked fine if SYSMIS was replaced by a number, see the syntax below. Since dealing with missing values is so common, I would like to know why the code above won't work. Any suggestions? Or suggestions for better ways to spot cases with one or more missing values?

*==============================*
DATA LIST FIXED/ nr 1 v1 3 v2 5 v3 7.
BEGIN DATA
1 2   2
2 4 1 3
3   4 5
4 2 4 3  
END DATA.

RECODE v1 TO v3 (SYSMIS=-9).

COMPUTE exclude=ANY(-9,v1,v2,v3).

EXECUTE.

*Deletion of cases...
*===============================*

Robert

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

Just to be clear, ANY($SYSMIS,v1,v2,v3) is not causing an error--it is just returning SYSMIS in all cases.  E.g., using this syntax with Robert's sample dataset...

RECODE v1 TO v3 (SYSMIS=-9).
COMPUTE exclude1=ANY($SYSMIS,v1,v2,v3).
COMPUTE exclude2=ANY(-9,v1,v2,v3).
FORMATS ALL (F2.0).
LIST.

...gives the following output (with no error messages):

nr v1 v2 v3 exclude1 exclude2
 
 1  2 -9  2     .        1
 2  4  1  3     .        0
 3 -9  4  5     .        1
 4  2  4  3     .        0

The FM section on "treatment of missing values in arguments" says this about ANY:

-----------------------
ANY (x,x1,x2,...xk)

For numeric values, if x is missing or all the remaining arguments are missing, the result is system-missing. For string values, user-missing value are treated as valid values, and the result is never missing.
-----------------------

Replacing the x in that example with $SYSMIS seems to invoke the first condition that is listed:  "if x is missing".  

Cheers,
Bruce

Robert Lundqvist-3 wrote

I just tried to set up a function to find those cases where there is one or more missing values in order to exclude them. My first thought was that these could be filtered out with the following variable:

COMPUTE exclude=ANY($SYSMIS,v1,v2,v3)

It did not work. However, it worked fine if SYSMIS was replaced by a number, see the syntax below. Since dealing with missing values is so common, I would like to know why the code above won't work. Any suggestions? Or suggestions for better ways to spot cases with one or more missing values?

*==============================*
DATA LIST FIXED/ nr 1 v1 3 v2 5 v3 7.
BEGIN DATA
1 2   2
2 4 1 3
3   4 5
4 2 4 3  
END DATA.

RECODE v1 TO v3 (SYSMIS=-9).

COMPUTE exclude=ANY(-9,v1,v2,v3).

EXECUTE.

*Deletion of cases...
*===============================*

Robert

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Andy W]

Also see the NMISS function,

COMPUTE exclude3 = NMISS(v1,v2,v3).

or

COMPUTE exclude3 = NMISS(v1 TO v3).

Which gets to where you want to go Robert.

[Jon Peck]

The ANY function is just shorthand for writing out all the conditions... if v1 eq $sysmis or v2 eq $sysmis etc.

Since sysmis is not equal to any value, including itself, you get the observed result.  The NMISS or NVALID function might be the most appropriate for your purpose.

On Mon, Mar 21, 2016 at 6:15 AM, Bruce Weaver <[hidden email]> wrote:

Just to be clear, ANY($SYSMIS,v1,v2,v3) is not causing an error--it is just
returning SYSMIS in all cases.  E.g., using this syntax with Robert's sample
dataset...

RECODE v1 TO v3 (SYSMIS=-9).
COMPUTE exclude1=ANY($SYSMIS,v1,v2,v3).
COMPUTE exclude2=ANY(-9,v1,v2,v3).
FORMATS ALL (F2.0).
LIST.

...gives the following output (with no error messages):

nr v1 v2 v3 exclude1 exclude2

 1  2 -9  2     .        1
 2  4  1  3     .        0
 3 -9  4  5     .        1
 4  2  4  3     .        0

The FM section on "treatment of missing values in arguments" says this about
ANY:

-----------------------
ANY (x,x1,x2,...xk)

For numeric values, if x is missing or all the remaining arguments are
missing, the result is system-missing. For string values, user-missing value
are treated as valid values, and the result is never missing.
-----------------------

Replacing the x in that example with $SYSMIS seems to invoke the first
condition that is listed:  "if x is missing".

Cheers,
Bruce



Robert Lundqvist-3 wrote

> I just tried to set up a function to find those cases where there is one
> or more missing values in order to exclude them. My first thought was that
> these could be filtered out with the following variable:
>
> COMPUTE exclude=ANY($SYSMIS,v1,v2,v3)
>
> It did not work. However, it worked fine if SYSMIS was replaced by a
> number, see the syntax below. Since dealing with missing values is so
> common, I would like to know why the code above won't work. Any
> suggestions? Or suggestions for better ways to spot cases with one or more
> missing values?
>
> *==============================*
> DATA LIST FIXED/ nr 1 v1 3 v2 5 v3 7.
> BEGIN DATA
> 1 2   2
> 2 4 1 3
> 3   4 5
> 4 2 4 3
> END DATA.
>
> RECODE v1 TO v3 (SYSMIS=-9).
>
> COMPUTE exclude=ANY(-9,v1,v2,v3).
>
> EXECUTE.
>
> *Deletion of cases...
> *===============================*
>
> Robert
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SYSMIS-in-ANY-tp5731776p5731777.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

--

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Bruce Weaver]

Agreed, provided one does not need to distinguish between system missing and user-defined missing.  Neither NMISS nor NVALID make that distinction, IIRC.

If one wants to trap only system missing values with NMISS, they must temporarily treat user-defined missing values as valid.  E.g.,


DATA LIST FIXED/ nr 1 v1 3 v2 5 v3 7.
BEGIN DATA
1 2   2
2 4 1 3
3   4 5
4 2 4 3  
END DATA.

DO IF nr EQ 1.
RECODE v1 TO v3 (SYSMIS=-9).
END IF.
MISSING VALUES ALL (-9).
COMPUTE NumMissing = NMISS(v1 to v3).
EXECUTE.
MISSING VALUES ALL ().
COMPUTE NumSysmis = NMISS(v1 to v3).
EXECUTE.
MISSING VALUES ALL (-9).

FORMATS ALL (F2.0).
LIST.

Before anyone gets on me about those two EXECUTE commands, they appear to be needed in this case.  Compare the output with and without theme below.

WITH the two EXECUTE commands:

nr v1 v2 v3 NumMissing NumSysmis
 
 1  2 -9  2      1         0
 2  4  1  3      0         0
 3  .  4  5      1         1
 4  2  4  3      0         0

WITHOUT the two EXECUTE commands:

nr v1 v2 v3 NumMissing NumSysmis
 
 1  2 -9  2      1         1
 2  4  1  3      0         0
 3  .  4  5      1         1
 4  2  4  3      0         0

Jon Peck wrote

The ANY function is just shorthand for writing out all the conditions... if
v1 eq $sysmis or v2 eq $sysmis etc.

Since sysmis is not equal to any value, including itself, you get the
observed result.  The NMISS or NVALID function might be the most
appropriate for your purpose.

On Mon, Mar 21, 2016 at 6:15 AM, Bruce Weaver <[hidden email]>
wrote:

> Just to be clear, ANY($SYSMIS,v1,v2,v3) is not causing an error--it is just
> returning SYSMIS in all cases.  E.g., using this syntax with Robert's
> sample
> dataset...
>
> RECODE v1 TO v3 (SYSMIS=-9).
> COMPUTE exclude1=ANY($SYSMIS,v1,v2,v3).
> COMPUTE exclude2=ANY(-9,v1,v2,v3).
> FORMATS ALL (F2.0).
> LIST.
>
> ...gives the following output (with no error messages):
>
> nr v1 v2 v3 exclude1 exclude2
>
>  1  2 -9  2     .        1
>  2  4  1  3     .        0
>  3 -9  4  5     .        1
>  4  2  4  3     .        0
>
> The FM section on "treatment of missing values in arguments" says this
> about
> ANY:
>
> -----------------------
> ANY (x,x1,x2,...xk)
>
> For numeric values, if x is missing or all the remaining arguments are
> missing, the result is system-missing. For string values, user-missing
> value
> are treated as valid values, and the result is never missing.
> -----------------------
>
> Replacing the x in that example with $SYSMIS seems to invoke the first
> condition that is listed:  "if x is missing".
>
> Cheers,
> Bruce
>
>
>
> Robert Lundqvist-3 wrote
> > I just tried to set up a function to find those cases where there is one
> > or more missing values in order to exclude them. My first thought was
> that
> > these could be filtered out with the following variable:
> >
> > COMPUTE exclude=ANY($SYSMIS,v1,v2,v3)
> >
> > It did not work. However, it worked fine if SYSMIS was replaced by a
> > number, see the syntax below. Since dealing with missing values is so
> > common, I would like to know why the code above won't work. Any
> > suggestions? Or suggestions for better ways to spot cases with one or
> more
> > missing values?
> >
> > *==============================*
> > DATA LIST FIXED/ nr 1 v1 3 v2 5 v3 7.
> > BEGIN DATA
> > 1 2   2
> > 2 4 1 3
> > 3   4 5
> > 4 2 4 3
> > END DATA.
> >
> > RECODE v1 TO v3 (SYSMIS=-9).
> >
> > COMPUTE exclude=ANY(-9,v1,v2,v3).
> >
> > EXECUTE.
> >
> > *Deletion of cases...
> > *===============================*
> >
> > Robert
> >
> > =====================
> > To manage your subscription to SPSSX-L, send a message to
>
> > LISTSERV@.UGA
>
> >  (not to SPSSX-L), with no body text except the
> > command. To leave the list, send the command
> > SIGNOFF SPSSX-L
> > For a list of commands to manage subscriptions, send the command
> > INFO REFCARD
>
>
>
>
>
> -----
> --
> Bruce Weaver
> [hidden email]
> http://sites.google.com/a/lakeheadu.ca/bweaver/
>
> "When all else fails, RTFM."
>
> NOTE: My Hotmail account is not monitored regularly.
> To send me an e-mail, please use the address shown above.
>
> --
> View this message in context:
> http://spssx-discussion.1045642.n5.nabble.com/SYSMIS-in-ANY-tp5731776p5731777.html
> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>



--
Jon K Peck
[hidden email]

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[David Marso]

See Jon's and Bruce's description below.
However (probably not what you want due to verbosity ;-(
--
DATA LIST FREE / x y z.
BEGIN DATA
1 2 3
. 1 2
1 . 3
END DATA.
COMPUTE SomeSysmis= ANY(1,SYSMIS(x),SYSMIS(y),SYSMIS(z)).
LIST.

Robert Lundqvist-3 wrote

I just tried to set up a function to find those cases where there is one or more missing values in order to exclude them. My first thought was that these could be filtered out with the following variable:

COMPUTE exclude=ANY($SYSMIS,v1,v2,v3)

It did not work. However, it worked fine if SYSMIS was replaced by a number, see the syntax below. Since dealing with missing values is so common, I would like to know why the code above won't work. Any suggestions? Or suggestions for better ways to spot cases with one or more missing values?

*==============================*
DATA LIST FIXED/ nr 1 v1 3 v2 5 v3 7.
BEGIN DATA
1 2   2
2 4 1 3
3   4 5
4 2 4 3  
END DATA.

RECODE v1 TO v3 (SYSMIS=-9).

COMPUTE exclude=ANY(-9,v1,v2,v3).

EXECUTE.

*Deletion of cases...
*===============================*

Robert

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

That's more like it.  

David Marso wrote

See Jon's and Bruce's description below.
However (probably not what you want due to verbosity ;-(
--
DATA LIST FREE / x y z.
BEGIN DATA
1 2 3
. 1 2
1 . 3
END DATA.
COMPUTE SomeSysmis= ANY(1,SYSMIS(x),SYSMIS(y),SYSMIS(z)).
LIST.

Robert Lundqvist-3 wrote

I just tried to set up a function to find those cases where there is one or more missing values in order to exclude them. My first thought was that these could be filtered out with the following variable:

COMPUTE exclude=ANY($SYSMIS,v1,v2,v3)

It did not work. However, it worked fine if SYSMIS was replaced by a number, see the syntax below. Since dealing with missing values is so common, I would like to know why the code above won't work. Any suggestions? Or suggestions for better ways to spot cases with one or more missing values?

*==============================*
DATA LIST FIXED/ nr 1 v1 3 v2 5 v3 7.
BEGIN DATA
1 2   2
2 4 1 3
3   4 5
4 2 4 3  
END DATA.

RECODE v1 TO v3 (SYSMIS=-9).

COMPUTE exclude=ANY(-9,v1,v2,v3).

EXECUTE.

*Deletion of cases...
*===============================*

Robert

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

For a large number of variables, you'd probably want to go with a DO-REPEAT or VECTOR & LOOP approach.  

DATA LIST FIXED/ nr 1 v1 3 v2 5 v3 7.
BEGIN DATA
1 2 3 2
2   1 3
3 4   5
4 2 4
5 3
6   4
7     5
8  
9 9 9 9
END DATA.

MISSING VALUES v1 to v3 (9).
COMPUTE NumSysmis =0.
VECTOR V = v1 to v3.
LOOP #i = 1 to 3.
- COMPUTE NumSysmis = SUM(NumSysmis,SYSMIS(v(#i))).
END LOOP.
COMPUTE NumMissing = NMIS(v1 to v3).
FORMATS ALL(F2.0).
LIST.

OUTPUT:

nr v1 v2 v3 NumSysmis NumMissing
 
 1  2  3  2      0         0
 2  .  1  3      1         1
 3  4  .  5      1         1
 4  2  4  .      1         1
 5  3  .  .      2         2
 6  .  4  .      2         2
 7  .  .  5      2         2
 8  .  .  .      3         3
 9  9  9  9      0         3

NumSysmis = number of system missing values, NumMissing = number of missing values, including system missing and user-defined missing.

David Marso wrote

See Jon's and Bruce's description below.
However (probably not what you want due to verbosity ;-(
--
DATA LIST FREE / x y z.
BEGIN DATA
1 2 3
. 1 2
1 . 3
END DATA.
COMPUTE SomeSysmis= ANY(1,SYSMIS(x),SYSMIS(y),SYSMIS(z)).
LIST.

Robert Lundqvist-3 wrote

I just tried to set up a function to find those cases where there is one or more missing values in order to exclude them. My first thought was that these could be filtered out with the following variable:

COMPUTE exclude=ANY($SYSMIS,v1,v2,v3)

It did not work. However, it worked fine if SYSMIS was replaced by a number, see the syntax below. Since dealing with missing values is so common, I would like to know why the code above won't work. Any suggestions? Or suggestions for better ways to spot cases with one or more missing values?

*==============================*
DATA LIST FIXED/ nr 1 v1 3 v2 5 v3 7.
BEGIN DATA
1 2   2
2 4 1 3
3   4 5
4 2 4 3  
END DATA.

RECODE v1 TO v3 (SYSMIS=-9).

COMPUTE exclude=ANY(-9,v1,v2,v3).

EXECUTE.

*Deletion of cases...
*===============================*

Robert

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Art Kendall]

I don't have SPSS or my documentation available but shouldn't this work
count N_sysmis var1 to var10(@sysmis).
compute exclude = N_sysmis ne 0.

[Bruce Weaver]

Good idea, Art.  You were off a bit in the details, but COUNT will indeed do the job much more efficiently than that VECTOR-LOOP stuff I posted earlier.

DATA LIST FIXED/ nr 1 v1 3 v2 5 v3 7.
BEGIN DATA
1 2 3 2
2   1 3
3 4   5
4 2 4
5 3
6   4
7     5
8  
9 9 9 9
END DATA.

MISSING VALUES v1 to v3 (9).
*******************************.
* Using COUNT, as suggested by Art K.
COUNT N_sysmis = v1 to v3(SYSMIS).
*******************************.
COMPUTE exclude = N_sysmis ne 0.
FORMATS ALL (F2.0).
LIST.

OUTPUT:

nr v1 v2 v3 N_sysmis exclude
 
 1  2  3  2     0        0
 2  .  1  3     1        1
 3  4  .  5     1        1
 4  2  4  .     1        1
 5  3  .  .     2        1
 6  .  4  .     2        1
 7  .  .  5     2        1
 8  .  .  .     3        1
 9  9  9  9     0        0


Note to self:  Always ask if it can be done more efficiently with COUNT!

Art Kendall wrote

I don't have SPSS or my documentation available but shouldn't this work
count N_sysmis var1 to var10(@sysmis).
compute exclude = N_sysmis ne 0.