Re: do repeat

Posted by Art Kendall on
URL: http://spssx-discussion.165.s1.nabble.com/do-repeat-tp5719707p5719753.html

Much of my time has been spent on providing just-in-time help of stat methods, and SPSS.
I developed a lot of my soapboxes by seeing how people were shooting themselves in the foot.

My use of SYSMIS is this.
There is a value for a variable such as the system was unable to follow my instructions.
I use the occurrence of a SYSMIS value as a clue in going back to find out what was wrong with my instructions (syntax).
(Over the years I have come to believe that it is almost always the instructions that are to blame.).
I then go back and change the instruction so that they provide for the special cases that caused the original SYSMIS.
Hence, in this instance

value labels varlist -999     'missing to avoid   dividing by zero' .

in public policy issues and program evaluation it is very common to do what in auditing is called "referencing" .  This is a process where another person  goes over the whole data gathering, transformation, and analysis, to check that the statements made are consistent with what was done.  I would not want legislation or a court case to rely on  my statements unless I exercised due diligence.

In addition to redrafting the syntax to provide for the special cases, I also redraft syntax to include comments that explain why there are warnings.

These go along with my soapbox about readability.  Except for my first few years of using computers, by 1974 or so, I came to value human factors criteria, such as readability of syntax and listings, over machine efficiency. YMMV but I find it much more saving of my time in the long run to use syntax that explains what is going on.

BTW even when I am the only one who is going to see the syntax, emphasizing readability also helps me when I am inevitably interrupted during an analysis.  Even a short effort such as replying to a post on this list may be subject to phone calls, calls of nature, meals, etc.  Foe example, I had two phone calls from my  physicians while responding to this post.

I also have a soapbox about about carefully differentiating the different kinds of user missing values including things like 'missing to avoid   dividing by zero' .  I find this very useful in the reasoning that the statistics are in support of.   But that is another story.

On 4/26/2013 10:03 AM, David Marso [via SPSSX Discussion] wrote:

Art,
  What is your concept of SYSMIS and do you think it should ever remain in a data file?
Isn't the very essence of system missing an entity that cannot be assigned a value by the 'system'?
I would consider division by zero as a candidate for treatment as SYSTEM MISSING rather than USER MISSING.

Interesting, you have been using SPSS 11 years longer than I have (1983).  

However, I did spend 11 years working at SPSS in the trenches (6 doing teksport,5 as a Full time Consultant)- Teksport was existence in the belly of the beast-answering all sorts of 'exotic questions' (Macro, INPUT PROGRAM, MATRIX etc -yeah, all the weirdness that nobody else wanted to touch with a 10 foot pole-  ).

In the early days of my taking on the 'exotic' I probably learned more from the users calling in than they learned from me.  My very first exposure to macro was someone trying to do:
!LET !arg2=!arg1 + 1.
Well, that had me baffled.  Well it is beyond obvious now, but believe it or not, once upon a time I was a rank newbie too.  
I didn't pop out of my mother with a full blown SPSS manual in my brain ;-)

There was even a special folder in the support database called "Marso Madness".  I wanted to call it "Doing Unnatural Acts with SPSS" but that got vetoed ;-).  

I LEARNED what I know today by being exposed to all the weird things 'weird' people try to do with SPSS for 6 years, 8 hours a day.  
 
I seriously doubt that support will even consider answering those sorts of questions any more.  Maybe I'll call them for the heck of it and screw with their minds with some easy but actual 'how do I do this' question. and see how long it takes them to refer me to the consulting department ;-)))

Art Kendall wrote

I admit it is a soapbox
        topic.  
     
      You surely are not a beginner and despite having used SPSS since
                            1972 I still learn from your posts. 
                            However, people who follow  the
                              list or search the archives are at least
                              to some degree learners.
                             
                            To aid debugging and for
                    quality assurance review I try to have people catch
                    all instances of sysmis, redraft the syntax,  and
                    provide for that condition in the transformations. 
                    Then assign a missing value that can
                    be labelled.
                      Redrafting would not be finished
                        until all sysmis
                            had been changed to to user missing with
                            value labels
                      for example in this instance I would put
                    value labels varlist -999
                              'missing to avoid
                                dividing by zero' .
                 
               
               
      Art Kendall
Social Research Consultants
      On 4/25/2013 1:47 PM, David Marso [via SPSSX Discussion] wrote:
   
     In principle I agree with the sysmis on right side
      for the most part.
     
      However in this case it is a direct correlate to the notion that
      the SYSTEM could NOT carry out some operation namely an attempted
      division by 0 (which would indeed end up as SYSMIS in normal
      calculations after spewing a host of warning messages.  Protecting
      the division with DO IF will render the same result.  So, my using
      SYSMIS in this case is deliberate and significant in reflecting
      the inability of the system to caary out the computation.
     
      --
      In MATRIX such an misadventure results in a fatal untrappable
      error and SPSS takes its ball and goes home (end of game), leading
      to unhappy emails and or phone calls.  
     
     
      My shameless hack attempts to capitalize on the cranked up
      energizer bunny properties of MATRIX vector efficiency while
      gracefully avoiding division by 0.  It is crucial to mop up
      afterwards internally to avoid ludicrous results but it is easier
      than protecting EVERY DIVISION in MATRIX with the following
      awkward monstrosity.
     
      DO IF (zero_check_matrix).
     
      COMPUTE newarray = oldarray / value.
     
      ELSE.
     
      COMPUTE newarray=MAKE(NROW(oldarray),NCOL(oldArray),0).
     
      LOOP #=1 TO NROW(oldarray).
     
      LOOP ##=1 TO NCOL(oldArray).
     
      COMPUTE newarray(#,##)=oldarray(#,##)/value.
     
      END LOOP.
     
      END LOOP.
     
      END IF.
     
      This also ends up being a PITA because one must take great care to
      treat the resulting 0 appropriately in subsequent calculations.
       In the HACK mode one can test for HUGE values which are
      distinguishable from 0's which can legitimately arise from 0
      numerators .
     
      Yeah, there are trade offs, and I am nowhere near solving every
      weird thing that can crop up in complex situations.
     
      But, I do stand by my using SYSMIS for this situation ;-)
     
      --
     
     
       
          Art
            Kendall wrote
          OOPS! sysmis on
            right side of
           
                  assignment operator.
                  RECODE ALL  (LO THRU 1.0 = COPY)
           
                 
                               (ELSE    -999    = ).
           
                 
                  missing values all (-999).
                  Art Kendall
           
            Social Research Consultants
                  On 4/25/2013 12:15 PM, Richard Ristow [via SPSSX
            Discussion]
           
                  wrote:
               
                 At 09:43 AM 4/25/2013, David Marso wrote:
           
                 
                 
                  >If the numbers are all positive then you can nuke
            the very
           
                  small fudge factor
           
                 
                  >and remove the recode.
           
                 
                  >If you have negative values then you will need to
            modify the
           
                  recode (I'm
           
                 
                  >sure you can sort that).
           
                 
                 
                  >RECODE ALL (1.000001 THRU HI=SYSMIS).
           
                 
                 
                  I think a cleaner way to do this RECODE, sidestepping
            having to
           
                 
                  choose a fudge factor, is
           
                 
                 
                  RECODE ALL  (LO THRU 1.0 = COPY)
           
                 
                               (ELSE        = SYSMIS).
           
                 
                 
                  But actually, I think the dividing-by-zero problem is
            a strong
           
                  reason
           
                 
                  to do the computation in a transformation program,
            rather than
           
                  MATRIX.
           
                 
                 
                  =====================
           
                 
                  To manage your subscription to SPSSX-L, send a message
            to
           
                 
                  [hidden email]
                  (not to SPSSX-L), with no body text except the
           
                 
                  command. To leave the list, send the command
           
                 
                  SIGNOFF SPSSX-L
           
                 
                  For a list of commands to manage subscriptions, send
            the command
           
                 
                  INFO REFCARD
           
                 
                 
                 
                 
                 
                    If you reply to this email, your
           
                      message will be added to the discussion below:
                    http://spssx-discussion.1045642.n5.nabble.com/do-repeat-tp5719707p5719720.html  
           
                 
                 
                    To start a new topic under SPSSX Discussion, email
           
                    [hidden
              email]  
                    To unsubscribe from SPSSX Discussion, click
           
                      here .
                    NAML
         
       
     
       Please
        reply to the list and not to my personal email.
       
        Those desiring my consulting or training services please feel
        free to email me.
       
        ---
       
        "Nolite dare sanctum canibus neque mittatis margaritas vestras
        ante porcos ne forte conculcent eas pedibus suis."
       
        Cum es damnatorum possederunt porcos iens ut salire off
        sanguinum cliff in abyssum?"
     
     
     
     
        If you reply to this email, your
          message will be added to the discussion below:
        http://spssx-discussion.1045642.n5.nabble.com/do-repeat-tp5719707p5719723.html 
     
     
        To start a new topic under SPSSX Discussion, email
        [hidden email] 
        To unsubscribe from SPSSX Discussion, click
          here .
        NAML

---

If you reply to this email, your message will be added to the discussion below:

http://spssx-discussion.1045642.n5.nabble.com/do-repeat-tp5719707p5719750.html

To start a new topic under SPSSX Discussion, email [hidden email]
To unsubscribe from SPSSX Discussion, click here.
NAML