CALL FOR CODE: Share etudes for more MATRIX functions

[Kirill Orlov]

To those SPSSers who use or program in MATRIX - END MATRIX or just appreciate it:

You know that MATRIX is just like an imbedded Matlab in SPSS, - very handy to program your own statistical routines. Unfortunately, MATRIX has not been developed by the company for many years, so there exist only very basic functions. Let us users not wait until the company capitally updates MATRIX but add there more functions ourselves, to use them now!

I've already written some important functions for MATRIX (see http://rivita.ru/spssmacros_en.shtml, "MATRIX - END MATRIX functions") which are small macros. I'd like to ask you to contribute. Please share your ideas/code for more functions. Send to me on kior[dog]comtv.ru. If I find it interesting I'll re-write it in the style of those macros and add to my list and publish on my site, with note of acknowledgement. Let us make MATRIX as rich as Matlab.

Some suggestions:
- The functions you are sending shouldn't be full-scale analysis routines or be too localized. They must be handy, scientific community recognized blocks, useful in composing analyses and other tasks in MATRIX.
- Should be written in SPSS MATRIX language or as pseudocode (so that I could implement it myself).
- Please comment the code, if possible.
- Links for (pseudo)code of important functions (especially math functions useful in statistics) are welcome too. I also encourage you to translate functions from programs/languages you know into SPSS syntax, and send it to me.

Thanks!

[David Marso]

Hi Kirill,

"those SPSSers who use or program in MATRIX"..
Let's see:  That would be you, Brian D and me? ;-)  I would add Marta but she's gone over to Stata.  I am sure there are others but can't recall anyone else ever posting MATRIX stuff:  Calling all lurkers?

  I think it has been made very clear that we can hold our breath until the hot place freezes over before IBM will update MATRIX.  They have too many programmers dedicated to salvaging the MVM to be able to spare any for important things like updating MATRIX or teaching MACRO how to do math.  Compared to SAS IML it is a sad crippled toy.  Let them eat python is the mantra of the day!  
I have tons of gold MATRIX nuggets scattered about.  Might be a week or so before I can put on my miner's hat and dig through my HD.  I am working on a couple MATRIX project currently.  I would need to talk with my client about sharing them but I'm sure it will be fine since it will be widely distributed/publicly available when it is finished.  Maybe appeal also to our HCS### group on LI.
D.

Kirill Orlov wrote

To those SPSSers who use or program in MATRIX - END MATRIX or just appreciate it:

You know that MATRIX is just like an imbedded Matlab in SPSS, - very handy to program your own statistical routines. Unfortunately, MATRIX has not been developed by the company for many years, so there exist only very basic functions. Let us users not wait until the company capitally updates MATRIX but add there more functions ourselves, to use them now!

I've already written some important functions for MATRIX (see http://rivita.ru/spssmacros_en.shtml, "MATRIX - END MATRIX functions") which are small macros. I'd like to ask you to contribute. Please share your ideas/code for more functions. Send to me on kior[dog]comtv.ru. If I find it interesting I'll re-write it in the style of those macros and add to my list and publish on my site, with note of acknowledgement. Let us make MATRIX as rich as Matlab.

Some suggestions:
- The functions you are sending shouldn't be full-scale analysis routines or be too localized. They must be handy, scientific community recognized blocks, useful in composing analyses and other tasks in MATRIX.
- Should be written in SPSS MATRIX language or as pseudocode (so that I could implement it myself).
- Please comment the code, if possible.
- Links for (pseudo)code of important functions (especially math functions useful in statistics) are welcome too. I also encourage you to translate functions from programs/languages you know into SPSS syntax, and send it to me.

Thanks!

[Bruce Weaver]

I am reminded of this old thread about the lack of IDFs in MATRIX.

http://spssx-discussion.1045642.n5.nabble.com/Inclusion-of-IDF-functions-in-matrix-language-td4680040.html

David Marso wrote

Hi Kirill,

"those SPSSers who use or program in MATRIX"..
Let's see:  That would be you, Brian D and me? ;-)  I would add Marta but she's gone over to Stata.  I am sure there are others but can't recall anyone else ever posting MATRIX stuff:  Calling all lurkers?

  I think it has been made very clear that we can hold our breath until the hot place freezes over before IBM will update MATRIX.  They have too many programmers dedicated to salvaging the MVM to be able to spare any for important things like updating MATRIX or teaching MACRO how to do math.  Compared to SAS IML it is a sad crippled toy.  Let them eat python is the mantra of the day!  
I have tons of gold MATRIX nuggets scattered about.  Might be a week or so before I can put on my miner's hat and dig through my HD.  I am working on a couple MATRIX project currently.  I would need to talk with my client about sharing them but I'm sure it will be fine since it will be widely distributed/publicly available when it is finished.  Maybe appeal also to our HCS### group on LI.
D.

Kirill Orlov wrote

To those SPSSers who use or program in MATRIX - END MATRIX or just appreciate it:

You know that MATRIX is just like an imbedded Matlab in SPSS, - very handy to program your own statistical routines. Unfortunately, MATRIX has not been developed by the company for many years, so there exist only very basic functions. Let us users not wait until the company capitally updates MATRIX but add there more functions ourselves, to use them now!

I've already written some important functions for MATRIX (see http://rivita.ru/spssmacros_en.shtml, "MATRIX - END MATRIX functions") which are small macros. I'd like to ask you to contribute. Please share your ideas/code for more functions. Send to me on kior[dog]comtv.ru. If I find it interesting I'll re-write it in the style of those macros and add to my list and publish on my site, with note of acknowledgement. Let us make MATRIX as rich as Matlab.

Some suggestions:
- The functions you are sending shouldn't be full-scale analysis routines or be too localized. They must be handy, scientific community recognized blocks, useful in composing analyses and other tasks in MATRIX.
- Should be written in SPSS MATRIX language or as pseudocode (so that I could implement it myself).
- Please comment the code, if possible.
- Links for (pseudo)code of important functions (especially math functions useful in statistics) are welcome too. I also encourage you to translate functions from programs/languages you know into SPSS syntax, and send it to me.

Thanks!

[David Marso]

Yeah, that comes to mind.  
You had to bring it up again on this lovely Saturday.
Utterly FUBARed my great mood!

I presently need the NCDF.T for a project ;-(
Fortunately it is only required at the VERY end so I just calculate everything
bounce to SPSS and then do it outside of MATRIX.
So, one could do something like:

MATRIX.
Get yer crap.
Calculate more crap .
END MATRIX.

COMPUTE blah blah = NCDF.T ().
COMPUTE blahblahblah=IDF.T().
...
...
MATRIX
Get old crap and other crap (blah blah).
Compute some more crap.
END MATRIX.

Phhhhhht!!!!!! The sound of cranial flatulence ?
Smells almost as good as Napalm in the morning!

Bruce Weaver wrote

I am reminded of this old thread about the lack of IDFs in MATRIX.

http://spssx-discussion.1045642.n5.nabble.com/Inclusion-of-IDF-functions-in-matrix-language-td4680040.html

David Marso wrote

Hi Kirill,

"those SPSSers who use or program in MATRIX"..
Let's see:  That would be you, Brian D and me? ;-)  I would add Marta but she's gone over to Stata.  I am sure there are others but can't recall anyone else ever posting MATRIX stuff:  Calling all lurkers?

  I think it has been made very clear that we can hold our breath until the hot place freezes over before IBM will update MATRIX.  They have too many programmers dedicated to salvaging the MVM to be able to spare any for important things like updating MATRIX or teaching MACRO how to do math.  Compared to SAS IML it is a sad crippled toy.  Let them eat python is the mantra of the day!  
I have tons of gold MATRIX nuggets scattered about.  Might be a week or so before I can put on my miner's hat and dig through my HD.  I am working on a couple MATRIX project currently.  I would need to talk with my client about sharing them but I'm sure it will be fine since it will be widely distributed/publicly available when it is finished.  Maybe appeal also to our HCS### group on LI.
D.

Kirill Orlov wrote

To those SPSSers who use or program in MATRIX - END MATRIX or just appreciate it:

You know that MATRIX is just like an imbedded Matlab in SPSS, - very handy to program your own statistical routines. Unfortunately, MATRIX has not been developed by the company for many years, so there exist only very basic functions. Let us users not wait until the company capitally updates MATRIX but add there more functions ourselves, to use them now!

I've already written some important functions for MATRIX (see http://rivita.ru/spssmacros_en.shtml, "MATRIX - END MATRIX functions") which are small macros. I'd like to ask you to contribute. Please share your ideas/code for more functions. Send to me on kior[dog]comtv.ru. If I find it interesting I'll re-write it in the style of those macros and add to my list and publish on my site, with note of acknowledgement. Let us make MATRIX as rich as Matlab.

Some suggestions:
- The functions you are sending shouldn't be full-scale analysis routines or be too localized. They must be handy, scientific community recognized blocks, useful in composing analyses and other tasks in MATRIX.
- Should be written in SPSS MATRIX language or as pseudocode (so that I could implement it myself).
- Please comment the code, if possible.
- Links for (pseudo)code of important functions (especially math functions useful in statistics) are welcome too. I also encourage you to translate functions from programs/languages you know into SPSS syntax, and send it to me.

Thanks!

[Kirill Orlov]

Bruce, David,
Distributional functions are almost entirely missing in MATRIX. I'm myself a bit of a lamer in that field. Don't you wish to contribute your expertise and to plot any functions (when you find time) to see it published among my functions?

[Bruce Weaver]

The method I ended up using (to get critical t-values to use in computing 95% confidence intervals) was the one Marta showed in her first post in that thread.


loop tvalue95=1960 to 12706.
- compute tcrit=tvalue95/1000.
- compute onetail=1-tcdf(tcrit,dfe).
- do if abs(onetail-0.025) lt 0.00005.
-   break.
- end if.
end loop.

As she said, it gives you the critical t (for alpha = .05, 2-tailed) with a small amount of error that is probably acceptable for most purposes.

It could be made more general by having alpha as variable, and replacing the hard-coded 0.025 with alpha/2.  The starting value on the loop would have be lowered too if you wanted to allow for alpha > .05.  

David came up with a different method later in that thread.  I don't remember all the details now, but suspect David's method ran a bit faster.  I stuck with Marta's method because it entailed less code, and ran plenty fast enough for what I was doing, and had small enough error for what I was doing.

Kirill Orlov wrote

Bruce, David,
Distributional functions are almost entirely missing in MATRIX. I'm
myself a bit of a lamer in that field. Don't you wish to contribute your
expertise and to plot any functions (when you find time) to see it
published among my functions?

[David Marso]

I've got a couple of fires to put out, some documentation to write and a proposal to create for a project.  After that I'll mine my HD and go through the archives for any interesting stuff I can find and send it your direction.  I notice your functions are macros which are matrix fragments ';-)  I guess great minds think alike ?
Within MATRIX there are
CDFNORM :Cumulative normal distribution function
CHICDF :Cumulative chi-squared distribution function
FCDF :Cumulative F distribution function
TCDF :Cumulative normal t distribution function

--

Kirill Orlov wrote

Bruce, David,
Distributional functions are almost entirely missing in MATRIX. I'm
myself a bit of a lamer in that field. Don't you wish to contribute your
expertise and to plot any functions (when you find time) to see it
published among my functions?

[David Marso]

Barring any revolutionary activity within the IBM development crew,
I think I'll just use my bouncy method (MATRIX (calc)-> SPSS (eval)-> MATRIX (calc).

Bruce Weaver wrote

The method I ended up using (to get critical t-values to use in computing 95% confidence intervals) was the one Marta showed in her first post in that thread.


loop tvalue95=1960 to 12706.
- compute tcrit=tvalue95/1000.
- compute onetail=1-tcdf(tcrit,dfe).
- do if abs(onetail-0.025) lt 0.00005.
-   break.
- end if.
end loop.

As she said, it gives you the critical t (for alpha = .05, 2-tailed) with a small amount of error that is probably acceptable for most purposes.

It could be made more general by having alpha as variable, and replacing the hard-coded 0.025 with alpha/2.  The starting value on the loop would have be lowered too if you wanted to allow for alpha > .05.  

David came up with a different method later in that thread.  I don't remember all the details now, but suspect David's method ran a bit faster.  I stuck with Marta's method because it entailed less code, and ran plenty fast enough for what I was doing, and had small enough error for what I was doing.

Kirill Orlov wrote

Bruce, David,
Distributional functions are almost entirely missing in MATRIX. I'm
myself a bit of a lamer in that field. Don't you wish to contribute your
expertise and to plot any functions (when you find time) to see it
published among my functions?

[Rich Ulrich]

Here is an application where I used Matrix.  I don't have the code.

I needed to score up exact Factor scores for data that included cases not
in the factor analysis -- including some with a (very) few Missings. 

IIRC, the Factor scores can be computed as a product of the the Factor
structure coefficients (saved from the Factoring) and variable z-scores. 
I used means and SDs (in SPSS syntax, as I remember) to get z-scores;
recoded the Missing z-scores to zero; and did the matrix multiply using Matrix. 

Even after I was done, it bothered me that it seemed like I had to multiply
matrix "A times Transpose(B)"  where I wanted to do "B times Transpose(A)".

--
Rich Ulrich

> Date: Sat, 11 May 2013 00:26:12 -0700

> From: [hidden email]
> Subject: CALL FOR CODE: Share etudes for more MATRIX functions
> To: [hidden email]
>
> To those SPSSers who use or program in MATRIX - END MATRIX or just appreciate
> it:
>
> You know that MATRIX is just like an imbedded Matlab in SPSS, - very handy
> to program your own statistical routines. Unfortunately, MATRIX has not been
> developed by the company for many years, so there exist only very basic
> functions. Let us users not wait until the company capitally updates MATRIX
> but add there more functions ourselves, to use them now!
>
> I've already written some important functions for MATRIX (see
> http://rivita.ru/spssmacros_en.shtml, "MATRIX - END MATRIX functions") which
> are small macros. I'd like to ask you to contribute. Please share your
> ideas/code for more functions. Send to me on kior[dog]comtv.ru. If I find it
> interesting I'll re-write it in the style of those macros and add to my list
> and publish on my site, with note of acknowledgement. Let us make MATRIX as
> rich as Matlab.
>
> Some suggestions:
> - The functions you are sending shouldn't be full-scale analysis routines or
> be too localized. They must be handy, scientific community recognized
> blocks, useful in composing analyses and other tasks in MATRIX.
> - Should be written in SPSS MATRIX language or as pseudocode (so that I
> could implement it myself).
> - Please comment the code, if possible.
> - Links for (pseudo)code of important functions (especially math functions
> useful in statistics) are welcome too. I also encourage you to translate
> functions from programs/languages you know into SPSS syntax, and send it to
> me.
>
> Thanks!
> ...