FTP Data Extraction and Automation via Syntax + Finding SPSS Help Manuals Online.

[J. R. Carroll]

All,

Tried searching the web but my searches turned up nothing relevant.  I am trying to find a way to automate some monthly analyses without having to go through some intermediate steps.  

Question 1:  Currently, my team has to log into an FTP site, manually select and download a specific file on the server and download it to their own HDD, then run the SPSS script necessary to analyze the data.  We are looking into ways that we might just link directly to the FTP file via syntax (e.g. some sort of "open" syntax command that allows FTP).  Does SPSS syntax support this?  If not, are there alternatives like if I wrap the syntax into a python layer (using a module like sockets or twister for packet management or some other FTP module - haven't really looked into this yet)?  The file is either a flat/text file or a CSV and prepared by a client, so we do not have an ODBC connection option.

Question 2:  I couldn't "easily" find the SPSS help manuals online (I found some old archived SPSS help manuals on a few personal sites).  I also checked the SPSS/IBM wiki but nothing jumped out at me.  Specifically I was looking for the PDF's that come packaged with SPSS (syntax ref, programming ref/python, etc); the PC I am on doesn't have SPSS installed on it right now. Am I blind and just totally glazed over it?  Or is not available for d/w?  Whatever the case, can someone suggest a solution to getting access to the manuals while I am away from a PC that has SPSS?

I've already tried:  google, IBM/SPSS Forums, old syntax ref manuals, and a few other odds and ends for both questions.

Thanks,

----


J. R. Carroll

Researcher for Hurtz Labs

Instructor at California State University, Sacramento

Research Methods, Test Development, and Statistics

www.jrcresearch.net

Cell:  [phone]

Email: [hidden email]

          [hidden email]

          [hidden email]

[mpirritano]

Justin,

 

1. It looks like all the manuals are posted on the ibm site here:

https://www-304.ibm.com/support/docview.wss?uid=swg27021213#en

 

2. What ftp client are you using? Chances are there is a way to automate the uploads. I use ws_ftp and have a simple batch statement that uploads the file. I then have windows scheduler run this batch statement regularly. I have it set to run at 6am every day. This is what my batch file looks like with generic filenames. The last line renames the file with the current date. ‘h’ is just what the network (local) drive was labeled in this case:

 

wsftppro.exe -s ftpConnectionName:\remote folder path\remote filename  -d  "local:h:/local path/local filename"

cd /d h:\go to directory where file you just loaded lives

ren filename filename_%date:~10,4%%date:~4,2%%date:~7,2%.xls

 

 

3. I’ve also scheduled syntax to run in a python wrapper using the windows scheduler. So you could schedule the ftp inload, allow for sufficient time for the file to be loaded, and then schedule the python file with the spss syntax to run. I would imagine that you can run any ftp client like this.

 

Happy holidays!

Matt

 

 

Matthew Pirritano, Ph.D.

Research Analyst IV

Medical Services Initiative (MSI)

Orange County Health Care Agency

[phone]

---

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of J. R. Carroll
Sent: Wednesday, December 21, 2011 12:30 PM
To: [hidden email]
Subject: FTP Data Extraction and Automation via Syntax + Finding SPSS Help Manuals Online.

 

All,

 

Tried searching the web but my searches turned up nothing relevant.  I am trying to find a way to automate some monthly analyses without having to go through some intermediate steps.  

 

Question 1:  Currently, my team has to log into an FTP site, manually select and download a specific file on the server and download it to their own HDD, then run the SPSS script necessary to analyze the data.  We are looking into ways that we might just link directly to the FTP file via syntax (e.g. some sort of "open" syntax command that allows FTP).  Does SPSS syntax support this?  If not, are there alternatives like if I wrap the syntax into a python layer (using a module like sockets or twister for packet management or some other FTP module - haven't really looked into this yet)?  The file is either a flat/text file or a CSV and prepared by a client, so we do not have an ODBC connection option.

 

Question 2:  I couldn't "easily" find the SPSS help manuals online (I found some old archived SPSS help manuals on a few personal sites).  I also checked the SPSS/IBM wiki but nothing jumped out at me.  Specifically I was looking for the PDF's that come packaged with SPSS (syntax ref, programming ref/python, etc); the PC I am on doesn't have SPSS installed on it right now. Am I blind and just totally glazed over it?  Or is not available for d/w?  Whatever the case, can someone suggest a solution to getting access to the manuals while I am away from a PC that has SPSS?

 

I've already tried:  google, IBM/SPSS Forums, old syntax ref manuals, and a few other odds and ends for both questions.

 

Thanks,

 

----


J. R. Carroll

Researcher for Hurtz Labs

Instructor at California State University, Sacramento

Research Methods, Test Development, and Statistics

www.jrcresearch.net

Cell:  [phone]

Email: [hidden email]

          [hidden email]

          [hidden email]

 

[Jon K Peck]

Listserv rejected this...


1) There is an extension command called
SPSSINC GETURI DATA
available from the SPSS Community site.  Maybe it can get the data in
question.  It isn't specifically aimed at ftp, but it might still work or
be tweakable.

2) There is a link to the online pdfs in the important bookmarks section
of the SPSS Community site.  That link will take you here:
http://www-01.ibm.com/support/docview.wss?uid=swg27021213

[J. R. Carroll]

Thanks Jon/Matt,

The manuals/books = helpful.

And yeah I was thinking of alternatives such as Cron, or Cron-like schedulers to automate the d/l which would work well with this one situation, but the general need for a way to access data via FTP still remains for other projects that are in production now or in the foreseeable future; I'm going to look into the extension suggestion from Jon - but I already see that in previous response on this list-serv a few months ago Jon noted that CSV files can't be used with this ext command, so I need to get more details about what types of files we are working with prior to moving to the next step.

If I find a solution (in regards to FTP) I'll be sure to respond to the serve to share the results.

Thanks,

-J

----


J. R. Carroll

Researcher for Hurtz Labs

Instructor at California State University, Sacramento

Research Methods, Test Development, and Statistics

www.jrcresearch.net

Cell:  [phone]

Email: [hidden email]

          [hidden email]

          [hidden email]

On Wed, Dec 21, 2011 at 1:03 PM, Pirritano, Matthew <[hidden email]> wrote:

Justin,

 

1. It looks like all the manuals are posted on the ibm site here:

https://www-304.ibm.com/support/docview.wss?uid=swg27021213#en

 

2. What ftp client are you using? Chances are there is a way to automate the uploads. I use ws_ftp and have a simple batch statement that uploads the file. I then have windows scheduler run this batch statement regularly. I have it set to run at 6am every day. This is what my batch file looks like with generic filenames. The last line renames the file with the current date. ‘h’ is just what the network (local) drive was labeled in this case:

 

wsftppro.exe -s ftpConnectionName:\remote folder path\remote filename  -d  "local:h:/local path/local filename"

cd /d h:\go to directory where file you just loaded lives

ren filename filename_%date:~10,4%%date:~4,2%%date:~7,2%.xls

 

 

3. I’ve also scheduled syntax to run in a python wrapper using the windows scheduler. So you could schedule the ftp inload, allow for sufficient time for the file to be loaded, and then schedule the python file with the spss syntax to run. I would imagine that you can run any ftp client like this.

 

Happy holidays!

Matt

 

 

Matthew Pirritano, Ph.D.

Research Analyst IV

Medical Services Initiative (MSI)

Orange County Health Care Agency

<a href="tel:%28714%29%20568-5648" value="[phone]" target="_blank">[phone]

---

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of J. R. Carroll
Sent: Wednesday, December 21, 2011 12:30 PM
To: [hidden email]
Subject: FTP Data Extraction and Automation via Syntax + Finding SPSS Help Manuals Online.

 

All,

 

Tried searching the web but my searches turned up nothing relevant.  I am trying to find a way to automate some monthly analyses without having to go through some intermediate steps.  

 

Question 1:  Currently, my team has to log into an FTP site, manually select and download a specific file on the server and download it to their own HDD, then run the SPSS script necessary to analyze the data.  We are looking into ways that we might just link directly to the FTP file via syntax (e.g. some sort of "open" syntax command that allows FTP).  Does SPSS syntax support this?  If not, are there alternatives like if I wrap the syntax into a python layer (using a module like sockets or twister for packet management or some other FTP module - haven't really looked into this yet)?  The file is either a flat/text file or a CSV and prepared by a client, so we do not have an ODBC connection option.

 

Question 2:  I couldn't "easily" find the SPSS help manuals online (I found some old archived SPSS help manuals on a few personal sites).  I also checked the SPSS/IBM wiki but nothing jumped out at me.  Specifically I was looking for the PDF's that come packaged with SPSS (syntax ref, programming ref/python, etc); the PC I am on doesn't have SPSS installed on it right now. Am I blind and just totally glazed over it?  Or is not available for d/w?  Whatever the case, can someone suggest a solution to getting access to the manuals while I am away from a PC that has SPSS?

 

I've already tried:  google, IBM/SPSS Forums, old syntax ref manuals, and a few other odds and ends for both questions.

 

Thanks,

 

----


J. R. Carroll

Researcher for Hurtz Labs

Instructor at California State University, Sacramento

Research Methods, Test Development, and Statistics

www.jrcresearch.net

Cell:  <a href="tel:%28916%29%20628-4204" value="[phone]" target="_blank">[phone]

Email: [hidden email]

          [hidden email]

          [hidden email]

 

[Rick Oliver-3]

The help system and PDF manuals are both available at the IBM site:

Help system:  http://publib.boulder.ibm.com/infocenter/spssstat/v20r0m0/index.jsp

PDF manuals: http://www.ibm.com/support/docview.wss?uid=swg27021213

Rick Oliver
Senior Information Developer
IBM Business Analytics (SPSS)
E-mail: [hidden email]
Phone: [phone] | T/L: 206-4922



From:        "Pirritano, Matthew" <[hidden email]>
To:        [hidden email]
Date:        12/21/2011 03:06 PM
Subject:        Re: FTP Data Extraction and Automation via Syntax + Finding SPSS              Help Manuals Online.
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

Justin,
 
1.        It looks like all the manuals are posted on the ibm site here:
https://www-304.ibm.com/support/docview.wss?uid=swg27021213#en
 
2.        What ftp client are you using? Chances are there is a way to automate the uploads. I use ws_ftp and have a simple batch statement that uploads the file. I then have windows scheduler run this batch statement regularly. I have it set to run at 6am every day. This is what my batch file looks like with generic filenames. The last line renames the file with the current date. ‘h’ is just what the network (local) drive was labeled in this case:
 
wsftppro.exe -s ftpConnectionName:\remote folder path\remote filename  -d  "local:h:/local path/local filename"
cd /d h:\go to directory where file you just loaded lives
ren filename filename_%date:~10,4%%date:~4,2%%date:~7,2%.xls
 
 
3.        I’ve also scheduled syntax to run in a python wrapper using the windows scheduler. So you could schedule the ftp inload, allow for sufficient time for the file to be loaded, and then schedule the python file with the spss syntax to run. I would imagine that you can run any ftp client like this.
 
Happy holidays!
Matt
 
 
Matthew Pirritano, Ph.D.
Research Analyst IV
Medical Services Initiative (MSI)
Orange County Health Care Agency
[phone]

---

From: SPSSX(r) Discussion [[hidden email]] On Behalf Of J. R. Carroll
Sent: Wednesday, December 21, 2011 12:30 PM
To: [hidden email]
Subject: FTP Data Extraction and Automation via Syntax + Finding SPSS Help Manuals Online.
 
All,
 
Tried searching the web but my searches turned up nothing relevant.  I am trying to find a way to automate some monthly analyses without having to go through some intermediate steps.  
 
Question 1:  Currently, my team has to log into an FTP site, manually select and download a specific file on the server and download it to their own HDD, then run the SPSS script necessary to analyze the data.  We are looking into ways that we might just link directly to the FTP file via syntax (e.g. some sort of "open" syntax command that allows FTP).  Does SPSS syntax support this?  If not, are there alternatives like if I wrap the syntax into a python layer (using a module like sockets or twister for packet management or some other FTP module - haven't really looked into this yet)?  The file is either a flat/text file or a CSV and prepared by a client, so we do not have an ODBC connection option.
 
Question 2:  I couldn't "easily" find the SPSS help manuals online (I found some old archived SPSS help manuals on a few personal sites).  I also checked the SPSS/IBM wiki but nothing jumped out at me.  Specifically I was looking for the PDF's that come packaged with SPSS (syntax ref, programming ref/python, etc); the PC I am on doesn't have SPSS installed on it right now. Am I blind and just totally glazed over it?  Or is not available for d/w?  Whatever the case, can someone suggest a solution to getting access to the manuals while I am away from a PC that has SPSS?
 
I've already tried:  google, IBM/SPSS Forums, old syntax ref manuals, and a few other odds and ends for both questions.
 
Thanks,
 

----


J. R. Carroll
Researcher for Hurtz Labs
Instructor at California State University, Sacramento
Research Methods, Test Development, and Statistics
www.jrcresearch.net
Cell:  [phone]
Email: jrcarroll@...
          jrcarroll@...
          jrc.csus@...
 

[Albert-Jan Roskam]

Hi,

Ad 2.:  Wouldn't it be cleaner do the file transfer and the syntax job in one python program? http://docs.python.org/library/ftplib.html

One function for the ftp connection and if it terminates successfully, the syntax job is fired up.

 

Cheers!!
Albert-Jan

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

---

From: "Pirritano, Matthew" <[hidden email]>
To: [hidden email]
Sent: Wednesday, December 21, 2011 10:03 PM
Subject: Re: [SPSSX-L] FTP Data Extraction and Automation via Syntax + Finding SPSS Help Manuals Online.

Justin,

 

1. It looks like all the manuals are posted on the ibm site here:

https://www-304.ibm.com/support/docview.wss?uid=swg27021213#en

 

2. What ftp client are you using? Chances are there is a way to automate the uploads. I use ws_ftp and have a simple batch statement that uploads the file. I then have windows scheduler run this batch statement regularly. I have it set to run at 6am every day. This is what my batch file looks like with generic filenames. The last line renames the file with the current date. ‘h’ is just what the network (local) drive was labeled in this case:

 

wsftppro.exe -s ftpConnectionName:\remote folder path\remote filename  -d  "local:h:/local path/local filename"

cd /d h:\go to directory where file you just loaded lives

ren filename filename_%date:~10,4%%date:~4,2%%date:~7,2%.xls

 

 

3. I’ve also scheduled syntax to run in a python wrapper using the windows scheduler. So you could schedule the ftp inload, allow for sufficient time for the file to be loaded, and then schedule the python file with the spss syntax to run. I would imagine that you can run any ftp client like this.

 

Happy holidays!

Matt

 

 

Matthew Pirritano, Ph.D.

Research Analyst IV

Medical Services Initiative (MSI)

Orange County Health Care Agency

[phone]

---

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of J. R. Carroll
Sent: Wednesday, December 21, 2011 12:30 PM
To: [hidden email]
Subject: FTP Data Extraction and Automation via Syntax + Finding SPSS Help Manuals Online.

 

All,

 

Tried searching the web but my searches turned up nothing relevant.  I am trying to find a way to automate some monthly analyses without having to go through some intermediate steps.  

 

Question 1:  Currently, my team has to log into an FTP site, manually select and download a specific file on the server and download it to their own HDD, then run the SPSS script necessary to analyze the data.  We are looking into ways that we might just link directly to the FTP file via syntax (e.g. some sort of "open" syntax command that allows FTP).  Does SPSS syntax support this?  If not, are there alternatives like if I wrap the syntax into a python layer (using a module like sockets or twister for packet management or some other FTP module - haven't really looked into this yet)?  The file is either a flat/text file or a CSV and prepared by a client, so we do not have an ODBC connection option.

 

Question 2:  I couldn't "easily" find the SPSS help manuals online (I found some old archived SPSS help manuals on a few personal sites).  I also checked the SPSS/IBM wiki but nothing jumped out at me.  Specifically I was looking for the PDF's that come packaged with SPSS (syntax ref, programming ref/python, etc); the PC I am on doesn't have SPSS installed on it right now. Am I blind and just totally glazed over it?  Or is not available for d/w?  Whatever the case, can someone suggest a solution to getting access to the manuals while I am away from a PC that has SPSS?

 

I've already tried:  google, IBM/SPSS Forums, old syntax ref manuals, and a few other odds and ends for both questions.

 

Thanks,

 

----


J. R. Carroll

Researcher for Hurtz Labs

Instructor at California State University , Sacramento

Research Methods, Test Development, and Statistics

www.jrcresearch.net

Cell:  [phone]

Email: [hidden email]

          [hidden email]

          [hidden email]

 

[Looman, Wendy]

I am out of the office December 22, 2011 through January 2, 2012. If you need assistance prior to January 3rd, please contact: David M. Bass, Vice President for Research, Benjamin Rose Institute on Aging, Margaret Blenkner Research Institute, 11900 Fairhill Road, #300, Cleveland, OH 44120-1053. Phone: [phone]. Email: [hidden email]

 

If your message concerns the HHCAHPS surveys, please contact: Sue Ambro, Benjamin Rose Institute on Aging, Margaret Blenkner Research Institute, 11900 Fairhill Road, Suite 300, Cleveland, OH 44120-1053. Phone: [phone]. Email: [hidden email]

-----------------------------------------------------------------------------------------------------------------
CONFIDENTIALITY NOTICE: The documents accompanying this email may contain confidential information, which is legally privileged. This information is intended only for the use of the recipient named above. If you are not the addressee or the employee or agent of the intended recipient, you are hereby notified that you are strictly prohibited from printing, storing, disseminating, distributing, or copying this communication. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer.
-----------------------------------------------------------------------------------------------------------------

[Ling Ting]

I will be out of the office until January 3. I will respond to my e-mails when I return. If you need immediate assistance (on December 22), please contact help desk at 479-575-2905. Happy Holidays! 

[J. R. Carroll]

Albert-Jan, 

That's the aim of my fact-finding mission (try to simplify and cleanly d/l and analyze the file with minimal effort and maintenance).  The FTP module you are suggesting sounds like it might be exactly what I need.  I am also looking into cURL (another third party python module).  Does anyone have experience working with either (hopefully both)?  I'd love to pick your brain about implementation and pros/cons to either one (off-list for right now while I determine the best solution to the topic/question presented)...

Thanks Albert-Jan,

-J

----


J. R. Carroll

Researcher for Hurtz Labs

Instructor at California State University, Sacramento

Research Methods, Test Development, and Statistics

www.jrcresearch.net

Cell:  [phone]

Email: [hidden email]

          [hidden email]

          [hidden email]

On Wed, Dec 21, 2011 at 2:52 PM, Albert-Jan Roskam <[hidden email]> wrote:

Hi,

Ad 2.:  Wouldn't it be cleaner do the file transfer and the syntax job in one python program? http://docs.python.org/library/ftplib.html

One function for the ftp connection and if it terminates successfully, the syntax job is fired up.

 

Cheers!!
Albert-Jan

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

---

From: "Pirritano, Matthew" <[hidden email]>
To: [hidden email]
Sent: Wednesday, December 21, 2011 10:03 PM
Subject: Re: [SPSSX-L] FTP Data Extraction and Automation via Syntax + Finding SPSS Help Manuals Online.

Justin,

 

1. It looks like all the manuals are posted on the ibm site here:

https://www-304.ibm.com/support/docview.wss?uid=swg27021213#en

 

2. What ftp client are you using? Chances are there is a way to automate the uploads. I use ws_ftp and have a simple batch statement that uploads the file. I then have windows scheduler run this batch statement regularly. I have it set to run at 6am every day. This is what my batch file looks like with generic filenames. The last line renames the file with the current date. ‘h’ is just what the network (local) drive was labeled in this case:

 

wsftppro.exe -s ftpConnectionName:\remote folder path\remote filename  -d  "local:h:/local path/local filename"

cd /d h:\go to directory where file you just loaded lives

ren filename filename_%date:~10,4%%date:~4,2%%date:~7,2%.xls

 

 

3. I’ve also scheduled syntax to run in a python wrapper using the windows scheduler. So you could schedule the ftp inload, allow for sufficient time for the file to be loaded, and then schedule the python file with the spss syntax to run. I would imagine that you can run any ftp client like this.

 

Happy holidays!

Matt

 

 

Matthew Pirritano, Ph.D.

Research Analyst IV

Medical Services Initiative (MSI)

Orange County Health Care Agency

<a href="tel:%28714%29%20568-5648" value="[phone]" target="_blank">[phone]

---

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of J. R. Carroll
Sent: Wednesday, December 21, 2011 12:30 PM
To: [hidden email]
Subject: FTP Data Extraction and Automation via Syntax + Finding SPSS Help Manuals Online.

 

All,

 

Tried searching the web but my searches turned up nothing relevant.  I am trying to find a way to automate some monthly analyses without having to go through some intermediate steps.  

 

Question 1:  Currently, my team has to log into an FTP site, manually select and download a specific file on the server and download it to their own HDD, then run the SPSS script necessary to analyze the data.  We are looking into ways that we might just link directly to the FTP file via syntax (e.g. some sort of "open" syntax command that allows FTP).  Does SPSS syntax support this?  If not, are there alternatives like if I wrap the syntax into a python layer (using a module like sockets or twister for packet management or some other FTP module - haven't really looked into this yet)?  The file is either a flat/text file or a CSV and prepared by a client, so we do not have an ODBC connection option.

 

Question 2:  I couldn't "easily" find the SPSS help manuals online (I found some old archived SPSS help manuals on a few personal sites).  I also checked the SPSS/IBM wiki but nothing jumped out at me.  Specifically I was looking for the PDF's that come packaged with SPSS (syntax ref, programming ref/python, etc); the PC I am on doesn't have SPSS installed on it right now. Am I blind and just totally glazed over it?  Or is not available for d/w?  Whatever the case, can someone suggest a solution to getting access to the manuals while I am away from a PC that has SPSS?

 

I've already tried:  google, IBM/SPSS Forums, old syntax ref manuals, and a few other odds and ends for both questions.

 

Thanks,

 

----


J. R. Carroll

Researcher for Hurtz Labs

Instructor at California State University , Sacramento

Research Methods, Test Development, and Statistics

www.jrcresearch.net

Cell:  <a href="tel:%28916%29%20628-4204" value="[phone]" target="_blank">[phone]

Email: [hidden email]

          [hidden email]

          [hidden email]