SPSS running slow
Locked
 
|
Hi All,
I am working on SPSS v20. sometime I need to put back number of blank variable in data; just to make my data look in order. Currently I have 10K variable in data and need to put 10K more variable. I simply create these variable in excel and manually copy paste in SPSS which is taking almost 30 min to 1 hour in pasting. During these period my system performance is also very slow. Can anyone suggest how can I make my system/ SPSS performance improve. Regards, Gaurav |
Administrator
|
I am not parsing this:
"sometime I need to put back number of blank variable in data; just to make my data look in order. " Please elaborate. "manually copy paste in SPSS" Certainly on my top 10 list of horrible practices. I have previously shared eye witness testimony about how this can be oh so FUBARific. http://spssx-discussion.1045642.n5.nabble.com/Spss-Syntax-select-all-values-in-a-variable-td5713897.html#a5713914 Use Syntax to generate and UPDATE to populate. --
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me. --- "Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis." Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?" |
|
|
Hi David,
I think the same scenario is happening with me as I am trying to manual co[py and paste. BTW I use the syntax to create variable but still I am struggling to re-order the variable as now it's 25k variable.
I don't think SPSS take that much time. Is something wrong at my end. I am simply match file command. Please suggest. Regards, Gaurav On Fri, Dec 14, 2012 at 3:55 PM, David Marso <[hidden email]> wrote: I am not parsing this: |
Re: SPSS running slow
|
In reply to this post by David Marso
"Certainly on my top 10 list of horrible practices."
Good point. Same experience here. I spent (wasted) a lot of time ploughing through colleagues' journal files in a desperate attempt to reconstruct what on earth they had been doing. It was often like a sausage: you don't want to know what's in there. And you really don't want to know what was sent (and blindly accepted) by their clients. Same for graduation students by the way. So on my top 10 of good SPSS practices, the first place has long been taken by "do everything with syntax, pasted from the GUI if necessary". And to throw in another one: "include the raw data and all syntax you used on it with the reports you send to your clients (or the graduation thesis you're submitting)". Best, Ruben > Date: Fri, 14 Dec 2012 02:25:58 -0800 > From: [hidden email] > Subject: Re: SPSS running slow > To: [hidden email] > > I am not parsing this: > "sometime I need to put back number of blank variable in data; just to make > my data look in order. " > Please elaborate. > > "manually copy paste in SPSS" > Certainly on my top 10 list of horrible practices. I have previously shared > eye witness testimony about how this can be oh so FUBARific. > http://spssx-discussion.1045642.n5.nabble.com/Spss-Syntax-select-all-values-in-a-variable-td5713897.html#a5713914 > > Use Syntax to generate and UPDATE to populate. > -- > > > GauravSrivastava wrote > > Hi All, > > > > I am working on SPSS v20. sometime I need to put back number of blank > > variable in data; just to make my data look in order. > > Currently I have 10K variable in data and need to put 10K more variable. I > > simply create these variable in excel and manually copy paste in SPSS > > which is taking almost 30 min to 1 hour in pasting. During these period my > > system performance is also very slow. > > > > Can anyone suggest how can I make my system/ SPSS performance improve. > > > > Regards, > > Gaurav > > > > > > ----- > Please reply to the list and not to my personal email. > Those desiring my consulting or training services please feel free to email me. > -- > View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-running-slow-tp5716941p5716943.html > Sent from the SPSSX Discussion mailing list archive at Nabble.com. > > ===================== > To manage your subscription to SPSSX-L, send a message to > [hidden email] (not to SPSSX-L), with no body text except the > command. To leave the list, send the command > SIGNOFF SPSSX-L > For a list of commands to manage subscriptions, send the command > INFO REFCARD |
|
Administrator
|
In reply to this post by GauravSrivastava
I still don't know the specifics of your circumstances.
Take a few steps back and give us some back story on what is and what must be. --
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me. --- "Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis." Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?" |
|
In reply to this post by David Marso
It would be very interesting to see David's top 10
horrible practices.
I have never counted my soapboxes or gathered them into one place. Putting these together might be a worthwhile presentation for the 2014 ASA Conference on Statistical Practice. (I was asked to put something together for the 2013 but did not have the time.) Art Kendall Social Research ConsultantsOn 12/14/2012 5:25 AM, David Marso wrote: I am not parsing this: "sometime I need to put back number of blank variable in data; just to make my data look in order. " Please elaborate. "manually copy paste in SPSS" Certainly on my top 10 list of horrible practices. I have previously shared eye witness testimony about how this can be oh so FUBARific. http://spssx-discussion.1045642.n5.nabble.com/Spss-Syntax-select-all-values-in-a-variable-td5713897.html#a5713914 Use Syntax to generate and UPDATE to populate. -- GauravSrivastava wroteHi All, I am working on SPSS v20. sometime I need to put back number of blank variable in data; just to make my data look in order. Currently I have 10K variable in data and need to put 10K more variable. I simply create these variable in excel and manually copy paste in SPSS which is taking almost 30 min to 1 hour in pasting. During these period my system performance is also very slow. Can anyone suggest how can I make my system/ SPSS performance improve. Regards, Gaurav----- Please reply to the list and not to my personal email. Those desiring my consulting or training services please feel free to email me. -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-running-slow-tp5716941p5716943.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Art Kendall
Social Research Consultants |
Automatic reply: SPSS running slow
|
Thanks for your email. I will be out of the office on Friday December 14th, returning on Monday, December 16th with limited access to email. I will respond to all emails when I return.
Genevieve Odoom Policy and Program Analyst OANHSS Suite 700 - 7050 Weston Rd. Woodbridge, ON L4L 8G7 Tel: (905) 851-8821 x 241 Fax: (905) 851-0744 [hidden email] www.oanhss.org<https://mail.oanhss.org/ecp/Organize/www.oanhss.org> ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
Administrator
|
In reply to this post by Art Kendall
"It would be very interesting to see David's top 10 horrible practices. "
Be careful what you ask for ;-)) Here are 20. Could probably come up with 20 more Could probably write a biting rancorous post about any of them. They are not ranked in any particular order. This is the order in which they fell out of my brain. Some aren't so much 'horrible practices' from a workflow point of view as much as aesthetically/ethically/economically/functionally -mix and match- offensive to one degree or another. -- 01. Releasing repetitive EXECUTE laden code into the wild. 02. Blissful ignorance of data normalization considerations (Wide?/Long?). 03. Coding as a challenge to channel the spirit of Rube Goldberg (can I possibly make this more complicated). 04. Oops. You mean I need to scale this to work with 10 million cases? 05. IF IF IF IF IF IF... What? You mean there is a RECODE command. 06. Copy.... errrr I think I'll paste that crap here.... Ctl Z? 07. DO REPEAT? VECTOR/LOOP? DEFINE-!ENDDEFINE> Ha. I'm paid on a LOCPD basis. 08. COMMENT? We don't need no stinkin' comments (only god and I knew what this did. Now only god knows). 09. everything should look like ee cummings wrote it. 10. COMPUTE The_Variable_Which_Has_No_Meanful_Name_And_No_Discernable_Purpos = 64. 11. /**12-24-2001 * This code should probably be modularized (header for 10000 line monster syntax). /**01-30-2010 * One of these days we are going to modularize this code (14000 lines later). 12. How do I do this (really simple thing) in Python -If I had a hammer....-? 13. I am the only one who is ever going to see this code, so who gives a rat's ass if it's FUGLY. 14. I 'borrowed' this unattributed code from the internet. I have no idea why/how it works but it SEEMS to do my job and I get to take a long lunch today. 15. It is cheaper to hire 5 flying monkeys than pay a trained professional to muck around with my data. 16. /** Don't poke this code. It is demon possessed jello and will jump off the table and suck out your brain **/. 17. /** This is a completely shameless Friday PM hack. Deadline looms... will fix it later (later never happens) **/. 18. This 50th copy of long pasted code does the same thing as the first copy except SEX has been changed to Drugs. !ROCK AND ROLL! 19. What do you mean the natural representation of a DATE isn't a string? 20. Column 1 is sacred. What do you mean by structure? What's wrong with my 10 level nested loop and DO IF monstrosity. ---
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me. --- "Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis." Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?" |
|
|
In reply to this post by David Marso
Hi David,
Still I couldn't figure out the problem with my SPSS. I am trying to reorder my SPSS file using below syntax: MATCH FILES FILE=*/KEEP respid ..... till all 27K variable. exe. But My spss is running very slow and running from last 2 hour but no outcome. Can you suggest any help? Regards, Gaurav |
|
MATCH FILES needs at least one more file. See page 1141 of the FM. John F Hall (Mr) Email: [hidden email] Website: www.surveyresearch.weebly.com MATCH FILES FILE={'savfile'|'dataset'} [TABLE={'savfile'|'dataset'}] {* } {* } [/RENAME=(old varnames=new varnames)...] [/IN=varname] /FILE=... [TABLE= ...] [/BY varlist] [/MAP] [/KEEP={ALL** }] [/DROP=varlist] {varlist} [/FIRST=varname] [/LAST=varname] **Default if the subcommand is omitted. Example MATCH FILES FILE='/data/part1.sav' /FILE='/data/part2.sav' /FILE=*. Overview MATCH FILES combines variables from 2 up to 50 IBM® SPSS® Statistics data files. MATCH FILES can make parallel or nonparallel matches between different files or perform table lookups. Parallel matches combine files sequentially by case (they are sometimes referred to as sequential matches). Nonparallel matches combine files according to the values of one or more key variables. In a table lookup, MATCH FILES looks up variables in one file and transfers those variables to a case file. -----Original Message----- Hi David, Still I couldn't figure out the problem with my SPSS. I am trying to reorder my SPSS file using below syntax: MATCH FILES FILE=*/KEEP respid ..... till all 27K variable. exe. But My spss is running very slow and running from last 2 hour but no outcome. Can you suggest any help? Regards, Gaurav -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-running-slow-tp5716941p5716974.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
Hi John,
Here I am trying to reorder my file in a sequence.
for example: Q1_1 Q1_2 Q1_4 Q1_6 Q1_3 Q1_5 need to rearrange like Q1_1 Q1_2 Q1_3 Q1_4 Q1_5 Q1_6 . I use this code to rearrange my file and it work perfectly.
MATCH FILES FILE=*/KEEP Q1_1 Q1_2 Q1_3 Q1_4 Q1_5 Q1_6. exe. Regards, Gaurav
On Mon, Dec 17, 2012 at 3:01 PM, John F Hall <[hidden email]> wrote:
|
|
Administrator
|
In reply to this post by GauravSrivastava
See item 02 of my horrible practices list.
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me. --- "Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis." Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?" |
|
Administrator
|
In reply to this post by John F Hall
Using MATCH FILES on a single file is perfectly fine.
It can be used to RENAME, DROP, Flag FIRST/LAST occurrences etc.
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me. --- "Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis." Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?" |
|
|
In reply to this post by David Marso
Thanks for all your response.
I am not sure if there is any specific requirement with MATCH FILES syntax. I did it easily using
SAVE OUTFILE = "C:\Users\GGAURAVS\Downloads\abc.sav" /KEEP respid ... till 27K variable.
exe. Regards, Gaurav
On Mon, Dec 17, 2012 at 4:48 PM, David Marso <[hidden email]> wrote: See item 02 of my horrible practices list. |
|
|
In reply to this post by David Marso
FM not so F then? John F Hall (Mr) Email: [hidden email] Website: www.surveyresearch.weebly.com -----Original Message----- Using MATCH FILES on a single file is perfectly fine. It can be used to RENAME, DROP, Flag FIRST/LAST occurrences etc. John F Hall wrote > MATCH FILES needs at least one more file. See page 1141 of the FM. > > > John F Hall (Mr) > > Email: <mailto: > johnfhall@ > > > johnfhall@ > Website: <http://surveyresearch.weebly.com/> > www.surveyresearch.weebly.com > > <SNIP FM details> ----- Please reply to the list and not to my personal email. Those desiring my consulting or training services please feel free to email me. -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-running-slow-tp5716941p5716978.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
In reply to this post by GauravSrivastava
Gaurav, I’m curious about this problem you’re having with your dataset. Let’s talk about the dataset. Are you saying you have 27,000 (thousand) variables in the file? How many cases in the file? Gene Maguin From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of I Am Gaurav Thanks for all your response. I am not sure if there is any specific requirement with MATCH FILES syntax. I did it easily using SAVE OUTFILE = "C:\Users\GGAURAVS\Downloads\abc.sav" /KEEP respid ... till 27K variable. exe. Regards, Gaurav On Mon, Dec 17, 2012 at 4:48 PM, David Marso <[hidden email]> wrote: See item 02 of my horrible practices list. > Hi David, > > Still I couldn't figure out the problem with my SPSS. I am trying to > reorder my SPSS file using below syntax: > MATCH FILES FILE=*/KEEP > respid ..... till all 27K variable. > exe. > > But My spss is running very slow and running from last 2 hour but no > outcome. Can you suggest any help? > > Regards, > Gaurav ----- Please reply to the list and not to my personal email. Those desiring my consulting or training services please feel free to email me. -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-running-slow-tp5716941p5716977.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
Hi Gene,
Yes, Variables are 27K but caes are not too much. it's only approx. 800. actually my data is in loop with brand (350 brands) vs brands attribute (approx 60). Since it'a a tracker so there are many variable which we kept to keep my data consistent. hope this give you a clear picture.
Regards, Gaurav
On Mon, Dec 17, 2012 at 7:22 PM, Maguin, Eugene <[hidden email]> wrote:
|
Automatic reply: SPSS running slow
|
|
I will be out of the office from December 12, 2012 to December 18, 2012. If you need immediate assistance please call 617-287-5541.
Paige Ransford Senior Research Associate The Center for Women in Politics & Public Policy UMass Boston |
|
Administrator
|
In reply to this post by GauravSrivastava
That is a completely unmanagable data arrangement.
-- Consider: Partially normalized CaseID Brand Attrib01....Attrib60. 1 1 1 ... 1 350 2 1 2 ... 2 350 ---- OR Fully normalized. CASEID Brand Attrib Value 1 1 1 00100101 1 ..... 1 1 60 00100160 .... 1 350 60 00135060 ... 800 350 60 80035060 ----------------
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me. --- "Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis." Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?" |
|
|
it's a bit different .. see below
CaseID Brand1_Attrib01 Brand1_Attrib02....Brand350_Attrib60 1 1 1 0 2 0 1 1 3 1 0 1 . . . . . . . . 800 1 1 . Regards, Gaurav On Mon, Dec 17, 2012 at 10:56 PM, David Marso <[hidden email]> wrote: That is a completely unmanagable data arrangement. |
| Free forum by Nabble | Edit this page |