Re: loop and do repeat problem with thousands of unique values to insert

Posted by Art Kendall on
URL: http://spssx-discussion.165.s1.nabble.com/loop-and-do-repeat-problem-with-thousands-of-unique-values-to-insert-tp4268902p4269595.html

I have never used the templates in autorecode.

but some of these possibilities in some combination may help.

try running on a machine with more memory

Make sure that you have a lot of WORKSPACE so your run does not go virtual.�

Make sure that you are running local not from a server.

Make sure that you have lots of disk space.

when you GET the original file /keep on the 14 variables and a caseid.
� � � � � � autorecode that file� SAVE with / keep for the 14 new variables and MATCH CASES

Use N OF CASES to get a subset for developing your syntax and then a larger subset to generate a template.

�

Write varstocases that saves a new file with a single variable and use that to generate a template. then apply the template to the original.

run autorecode on a few variables to generate a template and try that template on all 14.


HTH

Art Kendall



On 3/29/2011 8:29 AM, Maurice Vergeer wrote:

dear all,

thanks for your suggestions.

Regarding autorecode (David and Art's suggestion): I tried this, but
it took enormously long, so I interrupted it. The point is, there are
thousands of unique values, but appr. 4.5 million records (file size
over 3 gigabyte). So, it's large.

regarding vartstocase option, I'm not sure whether spss allows so many
columns. The values as such are not necessarily meaningful but need to
stay unique.

It appears there is no easy or obvious solution.
One option not explored yet is just inserting the string values and
numerical values in the do repeat.
This would result in a very large syntax file. This is a dirty
solution, not sure whether it's quick either.

Tonight I'll try to run one of options above and see whether it'll be
finished when I return from work tomorrow afternoon.

I'll let you kno whether it worked.

thanks again
Maurice




On Tue, Mar 29, 2011 at 20:37, David Marso [hidden email] wrote:

Hi Maurice,
If the AUTORECODE ../GROUP is not what you wish (ie your numeric codes have
some specific meaning).
SORT your external system file by the string variable and save it.
Transform your master file from wide to long using VARSTOCASES retaining
caseidentifier and string and index.
SORT by string.
MATCH FILES using the external file as a table with the string as a key.
transform the file from long to wide.
Done.
HTH, David
--

Maurice Vergeer wrote:

dear fellow list visitors,

please help me with this problem.
I have the following syntax which works perfectly.

It 'replaces' strings in old variables (name1 to name14) into
numerical ones in a new variable (newname1 to newname14).

example:
vector name=name1 to name14.
vector newname(14).
loop i=1 to 14.
do repeat a="alpha" "beta" "gamma" / b=1 2
3.
- if name(i) = a newname(i)=b.
end repeat print.
end loop.


However, instead of three values (alpha beta and gamma) I have
thousands of unique string values stored in a separate system file,
each identified with a unique numerical code.
How can I insert these values in the do repeat function (after 'a='
and after 'b=')?

The reason why I want to change these from string to numeric ones is
that I know the system file will be smaller and hopefully also faster
to read.

You help is much appreciated.

sincerely
Maurice




--
___________________________________________________________________
Maurice Vergeer
Department of communication, Radboud University� � (www.ru.nl)
PO Box 9104, NL-6500 HE Nijmegen, The Netherlands

Visiting Professor Yeungnam University, Gyeongsan, South Korea

Recent publications:
-Vergeer, M., Hermans, L., & Sams, S. (accepted for publication).
Online social networks and micro-blogging in political campaigning:
The exploration of a new campaign tool and a new campaign style. Party
Politics.
-Eisinga, R., Franses, Ph.H., & Vergeer, M. (2010). Weather conditions
and daily television use in the Netherlands, 1996–2005. International
Journal of Meteorology.

Webspace
www.mauricevergeer.nl
http://blog.mauricevergeer.nl/
www.journalisteninhetdigitaletijdperk.nl
maurice.vergeer (skype)
___________________________________________________________________

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/loop-and-do-repeat-problem-with-thousands-of-unique-values-to-insert-tp4268902p4269231.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD