MATRIX procedure enhancement

[jkpeck]

The Viewer output from the MATRIX procedure in SPSS Statistics is just plain text rather than the pivot tables produced by other procedures.  But there is a little-known feature introduced back in version 24 to change this.  As far as I can tell, almost no one knows about this even though it is documented in the CSR (which everyone surely reads cover to cover).

You can run the command SET MDISPLAY TABLES outside the MATRIX syntax to have the output from the PRINT command in MATRIX automatically displayed as pivot tables.  None of the code in the MATRIX procedure needs to change.

I ran this on a few big MATRIX jobs, including Andrew Hayes' PROCESS job, and it worked perfectly.  This mode does not affect titles or messages from the procedure.



SET MDISPLAY TEXT restore the plain text mode.

[Kirill Orlov]

Thank you very much, Jon, for reminding it.

[Kirill Orlov]

But they would have better enhance the MATRIX library of functions! It is a shame that MATRIX did not develop from the past century. Especially in the domain of linear algebra and optimization functions.

[Kirill Orlov]

Jon, this feature probably from version 26, not 24. I can't find it in version 25.

[jkpeck]

According to the CSR, this was added in "the November update" to V24, which I think means in a fixpack.  Do you get an error with SET MDISPLAY=TABLES?

btw, MATRIX did get a big update.  From the CSR history.
Statistics 27.0
• The NCDF.BETA cumulative distribution function is now supported.
• Probability density functions are now supported (they were previously only supported by the COMPUTE
command).
• Tail probability functions are now supported (they were previously only supported by the COMPUTE
command).
• Random variable functions are now supported (they were previously only supported by the COMPUTE
command).
Subscription November 2017 update / Statistics 26.0
• Long variable names (up to 64 bytes) can be used to name a matrix or vector name (such as COMPUTE,
CALL, PRINT, READ, WRITE, GET, SAVE, MGET, MSAVE, DISPLAY, RELEASE, and so on).
• Variable names that are included in a vector or matrix object are truncated to 8 bytes. This is because
the matrix/vector structure is an array of numbers, and each number can match a string only up to 8
bytes. Long names (up to 64 bytes) are supported only when explicitly specified.
• Long variable names are supported in GET and SAVE commands when explicitly specified on the /
VARIABLES subcommand (and when specified on the /STRINGS subcommand for the SAVE
command). Variable names for GET and SAVE commands are truncated to 8 bytes when they are
referenced through a vector in the /NAMES subcommand.
• The GET, SAVE, MGET, or MSAVE statements support both dataset references and physical file
specifications.
• MATRIX-END MATRIX supports statistical functions that were previously only supported by the
COMPUTE command (for example IDF.CHISQ, CDF.NORMAL, NCDF.F, and so on). For more information,
see Matrix Functions (MATRIX-END MATRIX command).

[Kirill Orlov]

I have the error, and CSR of vers. 25 does not mention MDISPLAY anywhere. I think the feature actually appeared in 26 or 27.

I'm aware of the enhancements of MATRIX in vers. 27 you mention. These are all good or necessary, sure, but not principal innovations. Matrix should update its collection of linear algebra and matrix manipulation functions. Or, at worst, create an option to read, directly within MATRIX, external FORTRAN functions (taken from LAPACK, for instance), to be able to run them.

[jkpeck]

What functions in particular would you like to see added?

[Kirill Orlov]

There are *many* to list. I will just mention now the ones I needed recently. Eigendecomposition of a general (asymmetric) matrix, Tridiagonalization, Houholder reflections & Hessenberg reduction, Forward/backward substitution. Then, of course, QR decomposition by other methods than just Gram-Schmidt gsch() only available currently. These are all interconnected functions, but they are often needed *separetely as well*.

(Because MATRIX misses them, I had to program some of them myself on SPSS MATRIX syntax, which, of course, made them slow and not modern. You can find them in my "Matrix-End matrix" collection on my page.)

MATRIX need more functions or Call statements because they will be on Fortran (or C) and so will be *fast*. Speed is very important, *crucial* for MATRIX as a potential tool for programming of algorithms and statistical procedures.

I would also call for some obvious, "easy" functions for speed and convenience reasons, such as, for example, (1) removing of arrays from a matrix (e.g. remove(X,ind1,:) *deletes* rows with positions ind1 vector from X matrix. (2) Faster sort than the method available currently. (3) Find function (such as, find all instances of a value(s) in a vector and return its positions). I mean, functions which, if you want to write them yourself in syntax, you either can't do it without loop-end loop or, if you can, it - the function - comes out not enough fast.

[Bruce Weaver]

Good discussion, guys.  

Katharina Schippert, have you seen this?  ;-)  

Kirill Orlov wrote

There are *many* to list. I will just mention now the ones I needed recently. Eigendecomposition of a general (asymmetric) matrix, Tridiagonalization, Houholder reflections & Hessenberg reduction, Forward/backward substitution. Then, of course, QR decomposition by other methods than just Gram-Schmidt gsch() only available currently. These are all interconnected functions, but they are often needed *separetely as well*.

(Because MATRIX misses them, I had to program some of them myself on SPSS MATRIX syntax, which, of course, made them slow and not modern. You can find them in my "Matrix-End matrix" collection on my page.)

MATRIX need more functions or Call statements because they will be on Fortran (or C) and so will be *fast*. Speed is very important, *crucial* for MATRIX as a potential tool for programming of algorithms and statistical procedures.

I would also call for some obvious, "easy" functions for speed and convenience reasons, such as, for example, (1) removing of arrays from a matrix (e.g. remove(X,ind1,:) *deletes* rows with positions ind1 vector from X matrix. (2) Faster sort than the method available currently. (3) Find function (such as, find all instances of a value(s) in a vector and return its positions). I mean, functions which, if you want to write them yourself in syntax, you either can't do it without loop-end loop or, if you can, it - the function - comes out not enough fast.

[jkpeck]

Forwarded to product management.

[Kirill Orlov]

Bruce, I've been wondering for 15 years why SPSS, a  renown statististical analytic tool with generally rich and user-friendly syntax (so that fingers itch to program on it), almost lacks linear algebra library. Sometimes I feel awkward in front of, say, Matlab users (statisticians), because I'm unwilling to reveal, "I can't do it, can't program it, my software don't have the necessary matrix functions".