Some interesting features of MSAVE

[Bruce Weaver]

This post follows up on an earlier thread about filling in the top half of a correlation matrix, and specifically this post in that thread:

http://spssx-discussion.1045642.n5.nabble.com/Filling-in-the-top-half-of-a-correlation-matrix-tp5722229p5722246.html

As noted in that post, when I used MSAVE to save a correlation matrix, MSAVE 'fixed' the 2's that I had on the main diagonal, replacing them with 1's in the resulting matrix dataset.  I've now learned (with the help of a colleague) that MSAVE can perform some other wonders too.

It turns out that when you save a correlation matrix via MSAVE, you don't need the full correlation matrix:  THE TOP HALF (excluding the main diagonal) will suffice.  When the lower half is missing, MSAVE sets the values below the diagonal to the corresponding values above the diagonal (see the warning message below).  Furthermore, it sets the values of the cells on the main diagonal to 1, even if they are some other incorrect value (i.e., not 1's).  

* Three examples.

NEW FILE.
DATASET CLOSE all.
MATRIX.
* Create lower half of a correlation matrix, but with 0's in the top half
* and 1's on the main diagonal.
COMPUTE CM =
 {1,0,0,0,0 ;
 .508,1,0,0,0 ;
 .347,.583,1,0,0;
 .204,.243,.294,1,0;
 .108,.166,.213,.250,1 }.
PRINT CM / format = "f5.3".
MSAVE T(CM) /TYPE=CORR /OUTFILE=* /VARIABLES=V1 to V5.
END MATRIX.
FORMATS V1 to V5 (f5.3).
LIST.

NEW FILE.
DATASET CLOSE all.
MATRIX.
* Create lower half of a correlation matrix, but with 0's in the top half
* and on the main diagonal.
COMPUTE CM =
 {0,0,0,0,0 ;
 .508,0,0,0,0 ;
 .347,.583,0,0,0;
 .204,.243,.294,0,0;
 .108,.166,.213,.250,0 }.
PRINT CM / format = "f5.3".
MSAVE T(CM) /TYPE=CORR /OUTFILE=* /VARIABLES=V1 to V5.
END MATRIX.
FORMATS V1 to V5 (f5.3).
LIST.

NEW FILE.
DATASET CLOSE all.
MATRIX.
* Create lower half of a correlation matrix, but with 0's in the top half
* and a mix of 1's and 0's on the main diagonal.
COMPUTE CM =
 {1,0,0,0,0 ;
 .508,0,0,0,0 ;
 .347,.583,0,0,1;
 .204,.243,.294,0,0;
 .108,.166,.213,.250,1 }.
PRINT CM / format = "f5.3".
MSAVE T(CM) /TYPE=CORR /OUTFILE=* /VARIABLES=V1 to V5.
END MATRIX.
FORMATS V1 to V5 (f5.3).
LIST.

For each of these examples, there is an error message and a warning, both associated with the matrix program.  For example:

>Error encountered in source line #  4115
 
>Warning # 14346
>Correlation matrix in MSAVE found to be non-symmetric.  A symmetric matrix
>will be saved with elements below diagonal set to the values of the
>corresponding elements above diagonal.

But despite that, all 3 examples generate the same correct output:

ROWTYPE_ VARNAME_    V1    V2    V3    V4    V5
 
CORR     V1       1.000  .508  .347  .204  .108
CORR     V2        .508 1.000  .583  .243  .166
CORR     V3        .347  .583 1.000  .294  .213
CORR     V4        .204  .243  .294 1.000  .250
CORR     V5        .108  .166  .213  .250 1.000


If these features of MSAVE are documented in the FM, I've not found the relevant pages yet.  

[Art Kendall]

Very interesting post.

For the sake of the archives perhaps you should provide more detail on why one might want matrices with zeros on the upper/lower triangle, and why mixed ones/zeros on the main diagonal

Ones on the  main diagonal are usually the correct values for a correlation matrix. 
However, the difference between the principal components kind of factor analysis and principal axes factor analysis is that in what used to be called ordinary fatcor analysis (principal axes) the main diagonal is replaced with reliability estimates, e.g., the squared multiple  correlation with other variables in the factor analysis.

If the matrix is a subjective dominance matrix or similarity matrix there can be intransitivities, i.e., the upper and lower triangles are not symmetric.


On 10/7/2013 6:48 PM, Bruce Weaver [via SPSSX Discussion] wrote:

This post follows up on an earlier thread about filling in the top half of a correlation matrix, and specifically this post in that thread:

http://spssx-discussion.1045642.n5.nabble.com/Filling-in-the-top-half-of-a-correlation-matrix-tp5722229p5722246.html

As noted in that post, when I used MSAVE to save a correlation matrix, MSAVE 'fixed' the 2's that I had on the main diagonal, replacing them with 1's in the resulting matrix dataset.  I've now learned (with the help of a colleague) that MSAVE can perform some other wonders too.

It turns out that when you save a correlation matrix via MSAVE, you don't need the full correlation matrix:  THE TOP HALF (excluding the main diagonal) will suffice.  When the lower half is missing, MSAVE sets the values below the diagonal to the corresponding values above the diagonal (see the warning message below).  Furthermore, it sets the values of the cells on the main diagonal to 1, even if they are some other incorrect value (i.e., not 1's).  

* Three examples.

NEW FILE.
DATASET CLOSE all.
MATRIX.
* Create lower half of a correlation matrix, but with 0's in the top half
* and 1's on the main diagonal.
COMPUTE CM =
 {1,0,0,0,0 ;
 .508,1,0,0,0 ;
 .347,.583,1,0,0;
 .204,.243,.294,1,0;
 .108,.166,.213,.250,1 }.
PRINT CM / format = "f5.3".
MSAVE T(CM) /TYPE=CORR /OUTFILE=* /VARIABLES=V1 to V5.
END MATRIX.
FORMATS V1 to V5 (f5.3).
LIST.

NEW FILE.
DATASET CLOSE all.
MATRIX.
* Create lower half of a correlation matrix, but with 0's in the top half
* and on the main diagonal.
COMPUTE CM =
 {0,0,0,0,0 ;
 .508,0,0,0,0 ;
 .347,.583,0,0,0;
 .204,.243,.294,0,0;
 .108,.166,.213,.250,0 }.
PRINT CM / format = "f5.3".
MSAVE T(CM) /TYPE=CORR /OUTFILE=* /VARIABLES=V1 to V5.
END MATRIX.
FORMATS V1 to V5 (f5.3).
LIST.

NEW FILE.
DATASET CLOSE all.
MATRIX.
* Create lower half of a correlation matrix, but with 0's in the top half
* and a mix of 1's and 0's on the main diagonal.
COMPUTE CM =
 {1,0,0,0,0 ;
 .508,0,0,0,0 ;
 .347,.583,0,0,1;
 .204,.243,.294,0,0;
 .108,.166,.213,.250,1 }.
PRINT CM / format = "f5.3".
MSAVE T(CM) /TYPE=CORR /OUTFILE=* /VARIABLES=V1 to V5.
END MATRIX.
FORMATS V1 to V5 (f5.3).
LIST.

For each of these examples, there is an error message and a warning, both associated with the matrix program.  For example:

>Error encountered in source line #  4115
 
>Warning # 14346
>Correlation matrix in MSAVE found to be non-symmetric.  A symmetric matrix
>will be saved with elements below diagonal set to the values of the
>corresponding elements above diagonal.

But despite that, all 3 examples generate the same correct output:

ROWTYPE_ VARNAME_    V1    V2    V3    V4    V5
 
CORR     V1       1.000  .508  .347  .204  .108
CORR     V2        .508 1.000  .583  .243  .166
CORR     V3        .347  .583 1.000  .294  .213
CORR     V4        .204  .243  .294 1.000  .250
CORR     V5        .108  .166  .213  .250 1.000


If these features of MSAVE are documented in the FM, I've not found the relevant pages yet.  

---

If you reply to this email, your message will be added to the discussion below:

http://spssx-discussion.1045642.n5.nabble.com/Some-interesting-features-of-MSAVE-tp5722431.html

To start a new topic under SPSSX Discussion, email [hidden email]
To unsubscribe from SPSSX Discussion, click here.
NAML

[Bruce Weaver]

Hi Art.  For me, it was not a case of wanting a matrix with zeros (or ones) in the upper or lower part of the matrix.  Rather, when I used MVA to generate a matrix of "expectation maximization" (EM) correlations, it gave me only the main diagonal and the lower half.  While playing around with a matrix program to fill in the top half, I first discovered that MSAVE sets the main diagonal to ones when it writes out a matrix file of type COR, regardless of what values are currently there in the matrix program.  A colleague subsequently discovered that MSAVE will also fill in the bottom half of the correlation matrix if the matrix is not symmetrical.  See the new example below with extreme impossible values in the top half and on the main diagonal.

Cheers,
Bruce

NEW FILE.
DATASET CLOSE all.
MATRIX.
* Create lower half of a correlation matrix, impossible values
* in the top half and on the main diagonal.
COMPUTE CM =
 {100,10,20,30,40 ;
 .508,200,10,20,30 ;
 .347,.583,300,10,20;
 .204,.243,.294,400,10;
 .108,.166,.213,.250,500 }.
PRINT CM / format = "f5.3".
MSAVE T(CM) /TYPE=CORR /OUTFILE=* /VARIABLES=V1 to V5.
END MATRIX.
FORMATS V1 to V5 (f5.3).
LIST.

OUTPUT:
ROWTYPE_ VARNAME_    V1    V2    V3    V4    V5
 
CORR     V1       1.000  .508  .347  .204  .108
CORR     V2        .508 1.000  .583  .243  .166
CORR     V3        .347  .583 1.000  .294  .213
CORR     V4        .204  .243  .294 1.000  .250
CORR     V5        .108  .166  .213  .250 1.000

Art Kendall wrote

Very interesting post.
       
        For the sake of the archives perhaps you should provide more
        detail on why one might want matrices with zeros on the
        upper/lower triangle, and why mixed ones/zeros on the main
        diagonal
       
        Ones on the  main diagonal are usually the correct values
      for a correlation matrix. 
      However, the difference between the principal components kind of
      factor analysis and principal axes factor analysis is that in what
      used to be called ordinary fatcor analysis (principal axes) the
      main diagonal is replaced with reliability estimates, e.g., the
      squared multiple  correlation with other variables in the factor
      analysis.
     
      If the matrix is a subjective dominance matrix or similarity
      matrix there can be intransitivities, i.e., the upper and lower
      triangles are not symmetric.
     
     
      Art Kendall
Social Research Consultants
      On 10/7/2013 6:48 PM, Bruce Weaver [via SPSSX Discussion] wrote:
   
     This post follows up on an earlier thread about
      filling in the top half of a correlation matrix, and specifically
      this post in that thread:
     
     
      http://spssx-discussion.1045642.n5.nabble.com/Filling-in-the-top-half-of-a-correlation-matrix-tp5722229p5722246.html 
     
      As noted in that post, when I used MSAVE to save a correlation
      matrix, MSAVE 'fixed' the 2's that I had on the main diagonal,
      replacing them with 1's in the resulting matrix dataset.  I've now
      learned (with the help of a colleague) that MSAVE can perform some
      other wonders too.
     
      It turns out that when you save a correlation matrix via MSAVE,
      you don't need the full correlation matrix:  THE TOP HALF
      (excluding the main diagonal) will suffice.  When the lower half
      is missing, MSAVE sets the values below the diagonal to the
      corresponding values above the diagonal (see the warning message
      below).  Furthermore, it sets the values of the cells on the main
      diagonal to 1, even if they are some other incorrect value (i.e.,
      not 1's).  
     
     
      * Three examples.
     
     
      NEW FILE.
     
      DATASET CLOSE all.
     
      MATRIX.
     
      * Create lower half of a correlation matrix, but with 0's in the
      top half
      * and 1's on the main diagonal.
     
      COMPUTE CM =
     
       {1,0,0,0,0 ;
     
       .508,1,0,0,0 ;
     
       .347,.583,1,0,0;
     
       .204,.243,.294,1,0;
     
       .108,.166,.213,.250,1 }.
     
      PRINT CM / format = "f5.3".
     
      MSAVE T(CM) /TYPE=CORR /OUTFILE=* /VARIABLES=V1 to V5.
     
      END MATRIX.
     
      FORMATS V1 to V5 (f5.3).
     
      LIST.
     
      NEW FILE.
     
      DATASET CLOSE all.
     
      MATRIX.
     
      * Create lower half of a correlation matrix, but with 0's in the
      top half
      * and on the main diagonal.
     
      COMPUTE CM =
     
       {0,0,0,0,0 ;
     
       .508,0,0,0,0 ;
     
       .347,.583,0,0,0;
     
       .204,.243,.294,0,0;
     
       .108,.166,.213,.250,0 }.
     
      PRINT CM / format = "f5.3".
     
      MSAVE T(CM) /TYPE=CORR /OUTFILE=* /VARIABLES=V1 to V5.
     
      END MATRIX.
     
      FORMATS V1 to V5 (f5.3).
     
      LIST.
     
      NEW FILE.
     
      DATASET CLOSE all.
     
      MATRIX.
     
      * Create lower half of a correlation matrix, but with 0's in the
      top half
      * and a mix of 1's and 0's on the main diagonal.
     
      COMPUTE CM =
     
       {1,0,0,0,0 ;
     
       .508,0,0,0,0 ;
     
       .347,.583,0,0,1;
     
       .204,.243,.294,0,0;
     
       .108,.166,.213,.250,1 }.
     
      PRINT CM / format = "f5.3".
     
      MSAVE T(CM) /TYPE=CORR /OUTFILE=* /VARIABLES=V1 to V5.
     
      END MATRIX.
     
      FORMATS V1 to V5 (f5.3).
     
      LIST.
     
      For each of these examples, there is an error message and a
      warning, both associated with the matrix program.  For example:
     
     
      >Error encountered in source line #  4115
       
     
      >Warning # 14346
      >Correlation matrix in MSAVE found to be non-symmetric.  A
      symmetric matrix
      >will be saved with elements below diagonal set to the values
      of the
      >corresponding elements above diagonal.
     
      But despite that, all 3 examples generate the same correct output:
     
     
      ROWTYPE_ VARNAME_    V1    V2    V3    V4    V5
       
     
      CORR     V1       1.000  .508  .347  .204  .108
      CORR     V2        .508 1.000  .583  .243  .166
      CORR     V3        .347  .583 1.000  .294  .213
      CORR     V4        .204  .243  .294 1.000  .250
      CORR     V5        .108  .166  .213  .250 1.000
     
     
     
      If these features of MSAVE are documented in the FM, I've not
      found the relevant pages yet.  
       --
       
        Bruce Weaver
       
        [hidden email] 
       
        http://sites.google.com/a/lakeheadu.ca/bweaver/ 
       
        "When all else fails, RTFM."
       
       
        NOTE: My Hotmail account is not monitored regularly.
         
          To send me an e-mail, please use the address shown above.
         
         
     
     
     
     
        If you reply to this email, your
          message will be added to the discussion below:
        http://spssx-discussion.1045642.n5.nabble.com/Some-interesting-features-of-MSAVE-tp5722431.html 
     
     
        To start a new topic under SPSSX Discussion, email
        [hidden email] 
        To unsubscribe from SPSSX Discussion, click
          here .
        NAML