Dear SPSS users,
I have a very straight forward syntax for importing variables and data to use on multiple txt files. After variable definition, and labeling I save the out file with the same name as the input txt file. Does anyone know how I can loop the GET DATA, VARIABLE LABELS, and SAVE commands through a list of input text files? Here is the syntax GET DATA /TYPE = TXT /FILE = 'dir\fileX.txt' /ARRANGEMENT = DELIMITED /FIRSTCASE = 1 /DELCASE = LINE /DELIMITERS = "\t" /IMPORTCASE = ALL /VARIABLES = a F10.2 b F10.2 c F10.2 etc. VARIABLE LABELS a 'labela' b 'labelb' etc. SAVE OUTFILE = 'dir\FileX.sav'. Thanks for your time Kam ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Here's a Python solution: begin program python. import spss filename=['file1', 'file2', 'file3'] for name in filename: spss.Submit(""" GET DATA /TYPE = TXT /FILE = 'dir\%(name)s.txt' /ARRANGEMENT = DELIMITED /FIRSTCASE = 1 /DELCASE = LINE /DELIMITERS = "\t" /IMPORTCASE = ALL /VARIABLES = a F10.2 b F10.2 c F10.2 etc. VARIABLE LABELS a 'labela' b 'labelb' etc. SAVE OUTFILE = '%(name)s.sav'. """ %locals()) end program.
Dear SPSS users, I have a very straight forward syntax for importing variables and data to use on multiple txt files. After variable definition, and labeling I save the out file with the same name as the input txt file. Does anyone know how I can loop the GET DATA, VARIABLE LABELS, and SAVE commands through a list of input text files? Here is the syntax GET DATA /TYPE = TXT /FILE = 'dir\fileX.txt' /ARRANGEMENT = DELIMITED /FIRSTCASE = 1 /DELCASE = LINE /DELIMITERS = "\t" /IMPORTCASE = ALL /VARIABLES = a F10.2 b F10.2 c F10.2 etc. VARIABLE LABELS a 'labela' b 'labelb' etc. SAVE OUTFILE = 'dir\FileX.sav'. Thanks for your time Kam ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Thank you Rick. I know nothing about Python, and even less about SPSS together with Python. I am assuming this is a Python shell script I call at the command line. FYI I am running Python 2.5.1 on IDLE on a XP machine. I get the following error running the shell:
Traceback (most recent call last): File "S:\tavabik\CHOP\NIH6\MEG\adata\SAM\SPSS\GETDATApy", line 40, in <module> """%locals()) ValueError: unsupported format character 't' (0x74) at index 88 Any idea what is going on here.? On Mon, Jun 14, 2010 at 5:29 PM, Rick Oliver <[hidden email]> wrote:
-- -K. |
Administrator
|
This should be easily doable with an old-fashioned SPSS macro--no Python required. The following is untested, and may need some tweaking, but it will give you the general idea.
define !getfiles ( flist !cmdend ) !do !f !in !flist GET DATA /TYPE = TXT /FILE = !quote(!concat("dir\",!f,".txt")) /ARRANGEMENT = DELIMITED /FIRSTCASE = 1 /DELCASE = LINE /DELIMITERS = "\t" /IMPORTCASE = ALL /VARIABLES = a F10.2 b F10.2 c F10.2 etc. VARIABLE LABELS a 'labela' b 'labelb' etc. SAVE OUTFILE = !quote(!concat("dir\",!f,".sav")) . !doend !enddefine . * Call the macro . !getfiles flist = "file1" "file2" "file3" .
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
In reply to this post by Kam T
This is a Python program run from within SPSS, running in a syntax window. Begin Program and End Program are SPSS commands that identify the beginning and end of Python block. You need to install the Python plug-in for SPSS in order for this to work. You can also run the code from Python (without Begin Program/End Program), although there may be some additional code required to start SPSS.
Thank you Rick. I know nothing about Python, and even less about SPSS together with Python. I am assuming this is a Python shell script I call at the command line. FYI I am running Python 2.5.1 on IDLE on a XP machine. I get the following error running the shell: Traceback (most recent call last): File "S:\tavabik\CHOP\NIH6\MEG\adata\SAM\SPSS\GETDATApy", line 40, in <module> """%locals()) ValueError: unsupported format character 't' (0x74) at index 88 Any idea what is going on here.? On Mon, Jun 14, 2010 at 5:29 PM, Rick Oliver <oliverr@...> wrote: Here's a Python solution: begin program python. import spss filename=['file1', 'file2', 'file3'] for name in filename: spss.Submit(""" GET DATA /TYPE = TXT /FILE = 'dir\%(name)s.txt' /ARRANGEMENT = DELIMITED /FIRSTCASE = 1 /DELCASE = LINE /DELIMITERS = "\t" /IMPORTCASE = ALL /VARIABLES = a F10.2 b F10.2 c F10.2 etc. VARIABLE LABELS a 'labela' b 'labelb' etc. SAVE OUTFILE = '%(name)s.sav'. """ %locals()) end program.
Dear SPSS users, I have a very straight forward syntax for importing variables and data to use on multiple txt files. After variable definition, and labeling I save the out file with the same name as the input txt file. Does anyone know how I can loop the GET DATA, VARIABLE LABELS, and SAVE commands through a list of input text files? Here is the syntax GET DATA /TYPE = TXT /FILE = 'dir\fileX.txt' /ARRANGEMENT = DELIMITED /FIRSTCASE = 1 /DELCASE = LINE /DELIMITERS = "\t" /IMPORTCASE = ALL /VARIABLES = a F10.2 b F10.2 c F10.2 etc. VARIABLE LABELS a 'labela' b 'labelb' etc. SAVE OUTFILE = 'dir\FileX.sav'. Thanks for your time Kam ===================== To manage your subscription to SPSSX-L, send a message to LISTSERV@... (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD -- -K. |
In reply to this post by Bruce Weaver
Hi Bruce,
Not sure you're still on the list but maybe someone else can help... I'm a new SPSS user (much more expereience in SAS but that is not an option on this project). I'm trying to use a variation of the SPSS macro you suggested previusly to import multiple txt files (not delimited) but I'm running into trouble with the GET DATA command. Or perhaps there is an easier way to do this now? I get: "ERROR Command name: Get Data Failure to open file." I suspect this is something simple, but I'm having no success troubleshooting. Thank you in advance, Victoria Syntax: define !getfiles ( flist !cmdend ) !do !f !in (!flist) GET DATA /TYPE = TXT /FILE = !quote(!concat("Z:\Data Entry\",!f,".txt")) /FIXCASE=1 /ARRANGEMENT=FIXED /FIRSTCASE=1 /IMPORTCASE=ALL /VARIABLES= /1 ID1_0 0-3 F4.0 ... DATASET NAME !quote(!concat("Z:\Data Entry\",!f,".sav")) . !doend !enddefine . |
At 04:15 PM 7/23/2012, varranda wrote:
>I'm trying to use an SPSS macro to import multiple txt files (not >delimited) but I'm running into trouble with the GET DATA command. > >Or perhaps there is an easier way to do this now? > >I get: >"ERROR Command name: Get Data Failure to open file." First, try running your macro with SET MPRINT ON -- put this just before the invocation, not the definition. I ran yours, not trying to actually read data; when I ran with !getfiles flist=a b c. the GET FILES commands expanded like this: 147 M> GET DATA /TYPE = TXT /FILE = 'Z:\Data Entry\a.txt' /FIXCASE=1 /ARRANGEM ENT=FIXED /FIRSTCASE=1 /IMPORTCASE=ALL /VARIABLES= /1 ID1_0 0-3 F4.0 . On the face of it that's syntactically valid, and the SPSS interpreter noted no syntax errors. I got the same error you did, but I really have no such file. Anyway, use MPRINT to look at exactly the GET FILES your macro expands into, and see if somehow they're pointing to files that really *don't* exist. Second, your DATASET NAME commands specify the full file and path name. That isn't valid; the dataset name must be a valid SPSS name, among other things having no embedded "\". I suggest DATASET NAME !f . This isn't a full solution, but maybe it's a starting point. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Administrator
|
Good suggestions by Richard.
One other comment is that your macro will only work (I think) if the text file names contain no spaces. Is that true of your file names? HTH.
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
Thanks for the help!
There was an issue with the file names and paths (now fixed) and I think I have isolated the problem to the DATASET NAME line. If I write in a single file name (i.e., DATASET NAME baseimport . ) the macro runs but replaces this dataset with each subsequent txt import file (logical). If I try to use just !f as the file name (i.e., DATASET NAME !f . ) I get the following warning: "Expecting dataset name or WINDOW. Found 20120713. [my file name] Execution of command stops." But it still appears to have imported at least the last data file because the data from 20120713 is visible in the untitled data editor - "Untitled 3". And, if I re-run the macro the untitled data visible in the data editor is now "untitled 6" - so I think it's actually importing the three data sets I'm just not directing it to save them how/where I want... I think perhaps I am confused about where SPSS is storing the data after import (I'm used the library set-up of SAS) so I'm going to go find more thorough SPSS documentation. Any other suggestions are very much appreciated! Thanks, Victoria SET MPRINT ON . define !getfiles ( flist !cmdend ) !do !f !in (!flist) GET DATA /TYPE = TXT /FILE = !quote(!concat("C:\SPSStest\",!f,".txt")) /FIXCASE=1 /ARRANGEMENT=FIXED /FIRSTCASE=1 /IMPORTCASE=ALL /VARIABLES= /1 ID1_0 0-3 F4.0 EXECUTE. DATASET NAME ??? . !doend !enddefine . !getfiles flist = 20111122 20120119 20120713 . |
Administrator
|
From the FM:
"DATA SET NAME: Basic Specification The basic specification for DATASET NAME is the command name followed by a name that conforms to variable naming rules. For more information, see the topic Variable Names on p. 50. .....From pg 50... "Variable names can be up to 64 bytes long, and the first character must be a letter or one of the characters @, #, or $. Subsequent characters can be any combination of letters, numbers........." --- OP: "I think perhaps I am confused about where SPSS is storing the data after import"... It is *NOT* storing the data because you are tossing it into the bit bucket since you neither SAVE the data file NOR do you provide a properly specified DATASET NAME. Maybe try altering the offending command with? DATASET NAME !CONCAT(XX,!f). ??? . ---
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me. --- "Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis." Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?" |
In reply to this post by varranda
It will read the files just fine, but the
dataset names are not valid since they don't follow variable naming rules.
In the absence of valid dataset names, the files are opened sequentially
and at the end only the last file opened is available for any further processing,
because that's what happens when datasets are unnamed.
Rick Oliver Senior Information Developer IBM Business Analytics (SPSS) E-mail: [hidden email] Phone: 312.893.4922 | T/L: 206-4922 From: varranda <[hidden email]> To: [hidden email], Date: 07/24/2012 11:25 AM Subject: Re: GET DATA to multiple TXT files Sent by: "SPSSX(r) Discussion" <[hidden email]> Thanks for the help! There was an issue with the file names and paths (now fixed) and I think I have isolated the problem to the DATASET NAME line. If I write in a single file name (i.e., DATASET NAME baseimport . ) the macro runs but replaces this dataset with each subsequent txt import file (logical). If I try to use just !f as the file name (i.e., DATASET NAME !f . ) I get the following warning: "Expecting dataset name or WINDOW. Found 20120713. [my file name] Execution of command stops." But it still appears to have imported at least the last data file because the data from 20120713 is visible in the untitled data editor - "Untitled 3". And, if I re-run the macro the untitled data visible in the data editor is now "untitled 6" - so I think it's actually importing the three data sets I'm just not directing it to save them how/where I want... I think perhaps I am confused about where SPSS is storing the data after import (I'm used the library set-up of SAS) so I'm going to go find more thorough SPSS documentation. Any other suggestions are very much appreciated! Thanks, Victoria SET MPRINT ON . define !getfiles ( flist !cmdend ) !do !f !in (!flist) GET DATA /TYPE = TXT /FILE = !quote(!concat("C:\SPSStest\",!f,".txt")) /FIXCASE=1 /ARRANGEMENT=FIXED /FIRSTCASE=1 /IMPORTCASE=ALL /VARIABLES= /1 ID1_0 0-3 F4.0 EXECUTE. DATASET NAME ??? . !doend !enddefine . !getfiles flist = 20111122 20120119 20120713 . -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/GET-DATA-to-multiple-TXT-files-tp1069059p5714409.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
In reply to this post by varranda
I'll duplicate some of what David Marso wrote, but maybe expand a little.
At 12:06 PM 7/24/2012, varranda wrote: >I think I have isolated the problem to the DATASET NAME line. > >If I write in a single file name (i.e., DATASET NAME baseimport >. ) the macro runs but replaces this dataset with each subsequent >txt import file (logical). Right. In detail(*), . All (loaded) SPSS data is in SPSS datasets . There is one 'active' dataset; syntax commands act on the active dataset. It may, but need not, have a name. . There may be one or more 'inactive' dataset, but *they must have names*. . When a new dataset is opened, e.g. by GET FILE or GET DATA, it becomes the active dataset, replacing the previous active one. If the previous active dataset had a named, it becomes inactive under that name. If it had no name, it is lost. . If the active dataset is given the name of an existing inactive dataset, that inactive dataset is lost. So, when you use DATASET NAME baseimport . each file you load goes into a new active dataset, which you then name 'baseimport', losing a previous dataset with that name. >If I try to use just !f as the file name (i.e., DATASET NAME !f . ) I get >the following warning: > >"Expecting dataset name or WINDOW. Found 20120713. [my file name] Executi >of command stops." Right. As David Marso wrote, "20120713" is not a valid SPSS name; so it can't be a dataset name; so the DATASET NAME command isn't executed, your dataset remains unnamed, and it's lost when you start a new one. David Marso's suggestion is a good one: prefix your file name with something alphabetic, so the result is a valid name. DATASET NAME !CONCAT('File_',!f). >But it still appears to have imported at least the last data file >because the data from 20120713 is visible in the untitled data >editor - "Untitled 3". Exactly. In fact, it's imported all the data files; but because the DATASET NAME commands didn't work, each imported file was in an unnamed dataset, and was lost when a new dataset was opened. You didn't replace your last-created dataset, so it remains active and loaded even though unnamed. .............................................. (*) Stop! You may not want to read from here on. It's correct, but it's easily confusing, and it's not relevant to your problem. The term 'active dataset' is used in *two* senses in SPSS documentation: as I've used it above (to mean the dataset that syntax acts on); or to mean the dataset whose Data Editor window is in the foreground, whose variables are displayed in the menus for commands. They need not be the same. I recommend, to help keep them the same, specifying WINDoW=FRONT on all DATASET NAME and DATASET ACTIVATE commands. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Free forum by Nabble | Edit this page |