GET DATA loop through folder

[jfca283]

Hi
I need to run the same syntax over some files. They are in the folder named DBase.
I mean, there is a db01.sav and over it run the syntax, then do the same for db02.sav and so on. How do i make it? How run the process only listing one time the files i'm interested?
Thanks for your time and interest.

[Bruce Weaver]

My first thought was to use a macro to loop through the list of files, and for each one to use INSERT FILE to run the common syntax.  

Depending on what the common syntax does, a second (possibly better and simpler) approach would be to stack the data files via ADD FILES, using /IN to create flags telling you which file each case came from.  Then combine those flags into a single categorical variable, which you can use for a SPLIT FILE before running the common syntax.  

What does the common syntax do?  

HTH.

jfca283 wrote

Hi
I need to run the same syntax over some files. They are in the folder named DBase.
I mean, there is a db01.sav and over it run the syntax, then do the same for db02.sav and so on. How do i make it? How run the process only listing one time the files i'm interested?
Thanks for your time and interest.

[Jon K Peck]

Use the SPSSINC PROCESS FILES extension command (Utilities > Process Data Files) to iterate over the files and apply any desired syntax.


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
phone: [phone]




From:        Bruce Weaver <[hidden email]>
To:        [hidden email],
Date:        10/24/2013 01:55 PM
Subject:        Re: [SPSSX-L] GET DATA loop through folder
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

My first thought was to use a macro to loop through the list of files, and
for each one to use INSERT FILE to run the common syntax.

Depending on what the common syntax does, a second (possibly better and
simpler) approach would be to stack the data files via ADD FILES, using /IN
to create flags telling you which file each case came from.  Then combine
those flags into a single categorical variable, which you can use for a
SPLIT FILE before running the common syntax.

What does the common syntax do?

HTH.



jfca283 wrote
> Hi
> I need to run the same syntax over some files. They are in the folder
> named DBase.
> I mean, there is a db01.sav and over it run the syntax, then do the same
> for db02.sav and so on. How do i make it? How run the process only listing
> one time the files i'm interested?
> Thanks for your time and interest.





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/GET-DATA-loop-through-folder-tp5722702p5722703.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[jfca283]

The syntax is very simple. I apply some filter and then i run  a crosstab. The chosen variables are not fixed. I changed them when i need them. But i have 16 sav files. Imagine opening the data, applying the filter and then doing a crosstab only to view the output. It's pretty boring and ineficient. I want just list the 16 files and then get the output for each sav file. Thanks for your answers.

[David Marso]

" The chosen variables are not fixed. I changed them when i need them."

How do you imagine this is going to happen if the variables of interest change from file to file?
If you are doing this with dialogs then I can only imagine this is 'boring and inefficient".
Perhaps it's time to suit up and learn SPSS Syntax?
See that Paste button on the dialogs?  Learn to love it.

jfca283 wrote

The syntax is very simple. I apply some filter and then i run  a crosstab. The chosen variables are not fixed. I changed them when i need them. But i have 16 sav files. Imagine opening the data, applying the filter and then doing a crosstab only to view the output. It's pretty boring and ineficient. I want just list the 16 files and then get the output for each sav file. Thanks for your answers.

[jfca283]

No, i express myself not clearly. The idea is running a crosstab for var1 and var2 over the 16 sav files. That's all. When i need other crosstab for the 16 databases, i just edit the syntax and write the new variables i want to analyse.

[David Marso]

Then just follow Jon's or Bruce's advice posted previously.
---

jfca283 wrote

No, i express myself not clearly. The idea is running a crosstab for var1 and var2 over the 16 sav files. That's all. When i need other crosstab for the 16 databases, i just edit the syntax and write the new variables i want to analyse.

[Bruce Weaver]

Assuming you have stacked the files (via ADD FILES with /IN) and then computed a single categorical variable (called WhichFile, for example) to code which file each record comes from:

CROSSTABS var1 by var2 by WhichFile.

David Marso wrote

Then just follow Jon's or Bruce's advice posted previously.
---

jfca283 wrote

No, i express myself not clearly. The idea is running a crosstab for var1 and var2 over the 16 sav files. That's all. When i need other crosstab for the 16 databases, i just edit the syntax and write the new variables i want to analyse.

[jfca283]

What about doing this

CD 'folder'.
DEFINE !path1 () '2001.sav'  !ENDDEFINE.
DEFINE !path2 () '2002.sav'  !ENDDEFINE.
...
DEFINE !path2 () '2012.sav'  !ENDDEFINE.

GET FILE=!path1.
 and the rest of the instructions? I mean, do some kind of FOR EACH database declared as !pathx apply the syntax. Some knows how to open a database but declaring it as member of a list? That thing in excel is so easy but here i'm still lost.

[David Marso]

When you post a Q. follow the thread.
Jon Peck posted an answer using a python extension .
It just iterates over each file in a folder.
Bruce another.
Why do you think you need to process each file independently?
Just add them and use split file or a layered XTAB.
For what you are thinking you could also use the macro facility.
See DEFINE !ENDEEFINE, the operator you seek is the !DO !DOEND

jfca283 wrote

What about doing this

CD 'folder'.
DEFINE !path1 () '2001.sav'  !ENDDEFINE.
DEFINE !path2 () '2002.sav'  !ENDDEFINE.
...
DEFINE !path2 () '2012.sav'  !ENDDEFINE.

GET FILE=!path1.
 and the rest of the instructions? I mean, do some kind of FOR EACH database declared as !pathx apply the syntax. Some knows how to open a database but declaring it as member of a list? That thing in excel is so easy but here i'm still lost.

[Bruce Weaver]

FOR EXAMPLE...

* Generate some data files to illustrate.
* The OP does not have to do this, as he already has the data files!.

* Open "Employee data.sav" and use XSAVE to write it out into 3 separate files.

NEW FILE.
DATASET CLOSE all.
* Modify the path below to point to where you have the sample files stored.
GET FILE = "C:\SPSSdata\Employee data.sav".

* Modify the paths below if you do not have a C:\Temp folder -- or make such a folder.
DO IF jobcat EQ 1.
- XSAVE OUTFILE = "C:\TEMP\db01.sav".
ELSE IF jobcat EQ 2.
- XSAVE OUTFILE = "C:\TEMP\db02.sav".
ELSE IF jobcat EQ 3.
- XSAVE OUTFILE = "C:\TEMP\db03.sav".
END IF.
EXECUTE.


***** Now the actual solution *****.

NEW FILE.
DATASET CLOSE all.

* The OP will have to add more FILE lines.
ADD FILES
  FILE = "C:\TEMP\db01.sav" / IN=db1 /
  FILE = "C:\TEMP\db02.sav" / IN=db2 /
  FILE = "C:\TEMP\db03.sav" / IN=db3
.
EXECUTE.

* OP will have to modify the DO REPEAT.
DO REPEAT flag = db1 to db3 / # = 1 to 3.
- IF flag WhichFile = #.
END REPEAT.
FORMATS WhichFile (f2.0).
* OP will have to modify variable names in the CROSSTAB command.
CROSSTABS gender by minority by WhichFile.

Bruce Weaver wrote

Assuming you have stacked the files (via ADD FILES with /IN) and then computed a single categorical variable (called WhichFile, for example) to code which file each record comes from:

CROSSTABS var1 by var2 by WhichFile.

David Marso wrote

Then just follow Jon's or Bruce's advice posted previously.
---

jfca283 wrote

No, i express myself not clearly. The idea is running a crosstab for var1 and var2 over the 16 sav files. That's all. When i need other crosstab for the 16 databases, i just edit the syntax and write the new variables i want to analyse.

[jfca283]

Unfortunately, i don't have the essentials modules for python installed on my computer. And your code is chinese for me. I run it and always i receive an error message. Totally wrong.

[David Marso]

@jfca283.
I suppose this is directed to Jon Peck?
Netiquette and prevention of thread flustercluck would suggest that you to quote the post to which you are referring.  Also 'I always receive an error message' NOT helpful. POST THE ERROR MESSAGE.

This is just a skeleton

DEFINE LoopIt (path !CHAREND ("/") / files !CMDEND ).
!DO !F !IN (!FILES)
GET FILE !QUOTE(!CONCAT(!UNQUOTE(!path),"/",!UNQUOTE(!F))) .
XTABS or whatever...
!DOEND
!ENDEFINE.
LoopIt path "C:\data\....whatever yourdatapath" / files 'file1' 'file2' ....... list them out .

OTOH: Bruce gave you a solution which will be easier to manage as far as output.
Basically, what is the end game?  What are you going to do with the 16 x whatever output objects?
Rather than how do I do 'whatever you are thinking about doing..."
I need to eventually achieve blah blah blah... I have 16 files ...
Are you intending to copy paste 16 different times into somewhere?
Probably easier to deal with a single output table rather than 16.
That's it.  I'm done.

jfca283 wrote

Unfortunately, i don't have the essentials modules for python installed on my computer. And your code is chinese for me. I run it and always i receive an error message. Totally wrong.

[jfca283]

I'll try installing python and essentials. Clearly spss syntax is not for me. How i wish the database were in stata format.

[Bruce Weaver]

D'oh!  Apparently you have not looked at the File Type options available under File > Save As.  Check it out.

jfca283 wrote

I'll try installing python and essentials. Clearly spss syntax is not for me. How i wish the database were in stata format.

[jfca283]

I don't think it will save all the text. I mean the labels.

[Art Kendall]

It used to be, and may still be, that no other package format retains as much of the data definition as SPSS does.
On 10/25/2013 4:06 PM, jfca283 [via SPSSX Discussion] wrote:

I don't think it will save all the text. I mean the labels.

---

If you reply to this email, your message will be added to the discussion below:

http://spssx-discussion.1045642.n5.nabble.com/GET-DATA-loop-through-folder-tp5722702p5722728.html

To start a new topic under SPSSX Discussion, email [hidden email]
To unsubscribe from SPSSX Discussion, click here.
NAML

[David Marso]

Maybe someone should write a book:
Mastering SPSS in 24 hrs ;-)
Nothing worthwhile is free (except perhaps my advise).
Take some time to RTF"ine"M and maybe it'll grow on you.
I just sought out some Stata syntax and it looks like it was written by geeks for geeks.
The output on the other hand is first class ;-(
Whatever, different strokes for different folks.

jfca283 wrote

I'll try installing python and essentials. Clearly spss syntax is not for me. How i wish the database were in stata format.

[jfca283]

I'll try stata. I didn't like spss syntax. Thanks for your posts.

[jfca283]

Please close this thread. Thanks. I've installed spssinc and else. Some problems have arisen, but i'll post then in another thread.