What are SPSS perspective of Julia?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

What are SPSS perspective of Julia?

Kirill Orlov
Julia language (https://en.wikipedia.org/wiki/Julia_(programming_language)) is a new fast programming language for math and science.

What do SPSS Statistics designers/developers think about it? Is it worth incorporating its support in Programmability? Python, R, or VB. Net are not particularly fast in computations. They are not as good as, say, Fortran or C if you need to compose a statistical or other numerical algorithm, especially for big arrays. But it looks as if Julia could be a good candidate. Performance innovations (for user-written programs) is what SPSS gasp for.

SPSS acutely needs a new powerful MATRIX facility. Can it be made as based on Julia? SPSS acutely needs a new MACRO (yes, metaprogrammical) facility. Can that be borrowed from Julia which supports "hygienic macro?

What people here say? What Jon Peck can say about it?

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: What are SPSS perspective of Julia?

Jon K Peck


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
phone: 720-342-5621




From:        Kirill Orlov <[hidden email]>
To:        SPSSX-List <[hidden email]>
Cc:        Jon K Peck/Chicago/IBM@IBMUS
Date:        08/23/2015 01:13 PM
Subject:        What are SPSS perspective of Julia?




Julia language (https://en.wikipedia.org/wiki/Julia_(programming_language))is a new fast programming language for math and science.

>>>Julia is still very young, and IMO it would be too soon to add a plugin for it, but ultimately it comes down to demand.  So far, we haven't seen requests for it.  In the meantime, there are other ways to speed up Python or R code.  Cython can compile Python code to near compiled C speed, and Rcpp does something similar for R.  And the Java plugin, which has been available for several releases now provides (compiled) Java speed.

What do SPSS Statistics designers/developers think about it? Is it worth incorporating its support in Programmability? Python, R, or VB. Net are not particularly fast in computations. They are not as good as, say, Fortran or C if you need to compose a statistical or other numerical algorithm, especially for big arrays. But it looks as if Julia could be a good candidate. Performance innovations (for user-written programs) is what SPSS gasp for.

SPSS acutely needs a new powerful MATRIX facility. Can it be made as based on Julia? SPSS acutely needs a new MACRO (yes, metaprogrammical) facility. Can that be borrowed from Julia which supports "hygienic macro?


>>>I would really like to see a new matrix capability or at least serious enhancements to what is already there, but there isn't any need to go to Julia for that.  numpy, for example, provides a really nice matrix facility in Python, and Python 3 provides much improved matrix support right in the language.  Most of the computation in the Partial Least Squares procedure (PLS), for example, is implemented in numpy.  numpy is very fast for matrix operations as well as having an elegant programming interface.  I have been lobbying for MATRIX improvements for some time, but I can't promise anything.

As for MACRO, as regular readers know, my strong belief is that Python programmability provides a really good alternative to the old MACRO facility, so IMO we already have one.  And for non-Python folks, there are several extension commands that provide macro-like functionality in standard syntax.


What people here say? What Jon Peck can say about it?


===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: What are SPSS perspective of Julia?

Kirill Orlov
Thank you Jon for the immediate response, and for concern about MATRIX enhancements.

As for macros, I don't feel Python (acknowledging its own wealth and strength) is woven as smoothly as SPSS macro is, with the regular syntax. The metaprogrammic nature of macros is what's most valuable. Will you be able to directly entwine lines of Python and syntax is this gorgeous fashion, for example:

do if .....
compute....
!IF (....)... !THEN
else if....
!IFEND
compute.....
end if.

where the second compute plays entirely different role dependent on a macro condition?

Or

add files /file= ... !DO !file !IN(...) /!file !DOEND .

where the files list is being evolved, syntactically, within the command line.

Python is great (for those who like it) but it is script, not a macro, for syntax.


23.08.2015 22:47, Jon K Peck пишет:


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
phone: 720-342-5621




From:        Kirill Orlov [hidden email]
To:        SPSSX-List [hidden email]
Cc:        Jon K Peck/Chicago/IBM@IBMUS
Date:        08/23/2015 01:13 PM
Subject:        What are SPSS perspective of Julia?




Julia language (https://en.wikipedia.org/wiki/Julia_(programming_language))is a new fast programming language for math and science.

>>>Julia is still very young, and IMO it would be too soon to add a plugin for it, but ultimately it comes down to demand.  So far, we haven't seen requests for it.  In the meantime, there are other ways to speed up Python or R code.  Cython can compile Python code to near compiled C speed, and Rcpp does something similar for R.  And the Java plugin, which has been available for several releases now provides (compiled) Java speed.

What do SPSS Statistics designers/developers think about it? Is it worth incorporating its support in Programmability? Python, R, or VB. Net are not particularly fast in computations. They are not as good as, say, Fortran or C if you need to compose a statistical or other numerical algorithm, especially for big arrays. But it looks as if Julia could be a good candidate. Performance innovations (for user-written programs) is what SPSS gasp for.

SPSS acutely needs a new powerful MATRIX facility. Can it be made as based on Julia? SPSS acutely needs a new MACRO (yes, metaprogrammical) facility. Can that be borrowed from Julia which supports "hygienic macro?


>>>I would really like to see a new matrix capability or at least serious enhancements to what is already there, but there isn't any need to go to Julia for that.  numpy, for example, provides a really nice matrix facility in Python, and Python 3 provides much improved matrix support right in the language.  Most of the computation in the Partial Least Squares procedure (PLS), for example, is implemented in numpy.  numpy is very fast for matrix operations as well as having an elegant programming interface.  I have been lobbying for MATRIX improvements for some time, but I can't promise anything.

As for MACRO, as regular readers know, my strong belief is that Python programmability provides a really good alternative to the old MACRO facility, so IMO we already have one.  And for non-Python folks, there are several extension commands that provide macro-like functionality in standard syntax.


What people here say? What Jon Peck can say about it?


===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD


===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: What are SPSS perspective of Julia?

Jon K Peck
It's true that Python has to be used in blocks, but technically what you describe could be done by creating a simple extension command.  That is, however, more work and probably wouldn't be worth it for simple usage.

However, what people often do is to reverse the whole process and build and run the entire set of syntax inside a Python program, so you could do something like
begin program.
syntax = [
"COMPUTE ...."
"COMPUTE ...."
]
if <some condition >:
    syntax.append("blah blah.")
else:
    syntax.append("some other blah blah.")
...
spss.Submit(syntax)
end program.

and many more variations.  Since Python has access to the current data and metadata, unlike macro, it can generate syntax much more flexibly.  There are ways to pass parameters in to the program as well.

Note also that extension commands such as SPSSINC SELECT VARIABLES actually creates a macro listing variables that pass various metadata requirements such as selecting numeric or scale level or matching name patterns or custom attributes variables.  And it has regular syntax, since it is an extension command.

Since V22, Python has been a standard part of the main Statistics installation process, so no extra steps are required to make it available to a job.

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
phone: 720-342-5621




From:        Kirill Orlov <[hidden email]>
To:        Jon K Peck/Chicago/IBM@IBMUS, [hidden email]
Date:        08/23/2015 02:34 PM
Subject:        Re: What are SPSS perspective of Julia?




Thank you Jon for the immediate response, and for concern about MATRIX enhancements.

As for macros, I don't feel Python (acknowledging its own wealth and strength) is woven as smoothly as SPSS macro is, with the regular syntax. The metaprogrammic nature of macros is what's most valuable. Will you be able to directly entwine lines of Python and syntax is this gorgeous fashion, for example:

do if .....
compute....
!IF (....)... !THEN
else if....
!IFEND
compute.....
end if.

where the second compute plays entirely different role dependent on a macro condition?

Or

add files /file= ... !DO !file !IN(...) /!file !DOEND .

where the files list is being evolved, syntactically, within the command line.

Python is great (for those who like it) but it is script, not a macro, for syntax.


23.08.2015 22:47, Jon K Peck пишет:


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM

peck@...
phone: 720-342-5621





From:        
Kirill Orlov <kior@...>
To:        
SPSSX-List [hidden email]
Cc:        
Jon K Peck/Chicago/IBM@IBMUS
Date:        
08/23/2015 01:13 PM
Subject:        
What are SPSS perspective of Julia?




Julia language
(
https://en.wikipedia.org/wiki/Julia_(programming_language))is a new fast programming language for math and science.

>>>Julia is still very young, and IMO it would be too soon to add a plugin for it, but ultimately it comes down to demand.  So far, we haven't seen requests for it.  In the meantime, there are other ways to speed up Python or R code.  Cython can compile Python code to near compiled C speed, and Rcpp does something similar for R.  And the Java plugin, which has been available for several releases now provides (compiled) Java speed.

What do SPSS Statistics designers/developers think about it? Is it worth incorporating its support in Programmability? Python, R, or VB. Net are not particularly fast in computations. They are not as good as, say, Fortran or C if you need to compose a statistical or other numerical algorithm, especially for big arrays. But it looks as if Julia could be a good candidate. Performance innovations (for user-written programs) is what SPSS gasp for.

SPSS acutely needs a new powerful MATRIX facility. Can it be made as based on Julia? SPSS acutely needs a new MACRO (yes, metaprogrammical) facility. Can that be borrowed from Julia which supports "hygienic macro?

>>>I would really like to see a new matrix capability or at least serious enhancements to what is already there, but there isn't any need to go to Julia for that.  numpy, for example, provides a really nice matrix facility in Python, and Python 3 provides much improved matrix support right in the language.  Most of the computation in the Partial Least Squares procedure (PLS), for example, is implemented in numpy.  numpy is very fast for matrix operations as well as having an elegant programming interface.  I have been lobbying for MATRIX improvements for some time, but I can't promise anything.

As for MACRO, as regular readers know, my strong belief is that Python programmability provides a really good alternative to the old MACRO facility, so IMO we already have one.  And for non-Python folks, there are several extension commands that provide macro-like functionality in standard syntax.


What people here say? What Jon Peck can say about it?


===================== To manage your subscription to SPSSX-L, send a message to
LISTSERV@...(not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: What are SPSS perspective of Julia?

Kirill Orlov
Yes, but...
Although your last reply, Jon, is (expectedly) worded so as to display Python's abilities and flexibility once again for the audience, the truth you admit in the second paragraph, by the example code, is there: Python is awkward and looks ugly with syntax - all those blocks and quotes and parentheses - as if it has just swallowed a pig. That's why and where macro is better. There my plea is only about two things: 1) teach macro to do arithmetics; 2) make it be able to take values back from the expanded commands results and to update the expansions (re-expand partially) upon having such feedback. The second task is, of course, a great challenge.

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: What are SPSS perspective of Julia?

Jon K Peck
It's fine to discuss enhancement requests on this and other SPSS sites, but to get official consideration of them, they need to be submitted to the enhancement request site, which has been previously posted on this list.


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
phone: 720-342-5621




From:        Kirill Orlov <[hidden email]>
To:        [hidden email]
Date:        08/23/2015 03:41 PM
Subject:        Re: [SPSSX-L] What are SPSS perspective of Julia?
Sent by:        "SPSSX(r) Discussion" <[hidden email]>




Yes, but...
Although your last reply, Jon, is (expectedly) worded so as to display Python's abilities and flexibility once again for the audience, the truth you admit in the second paragraph, by the example code, is there: Python is awkward and looks ugly with syntax - all those blocks and quotes and parentheses - as if it has just swallowed a pig. That's why and where macro is better. There my plea is only about two things: 1) teach macro to do arithmetics; 2) make it be able to take values back from the expanded commands results and to update the expansions (re-expand partially) upon having such feedback. The second task is, of course, a great challenge.

===================== To manage your subscription to SPSSX-L, send a message to LISTSERV@...(not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD