Clean up a string variable

[Paul Oosterveld]

Hi,

I am trying to clean up a string variable. I only want to keep characters (a-z
and A-Z). There should be an easier way than removing all other symbols with
replace I guess. Any suggestions?

Thanks in advance.

Paul Oosterveld

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Maguin, Eugene]

Paul,

So, for example, does 'A12wer' go to 'A  wer' or to 'Awer' or to 'Awer  '?

Gene Maguin




-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Paul Oosterveld
Sent: Friday, July 15, 2011 9:38 AM
To: [hidden email]
Subject: Clean up a string variable

Hi,

I am trying to clean up a string variable. I only want to keep characters
(a-z
and A-Z). There should be an easier way than removing all other symbols with
replace I guess. Any suggestions?

Thanks in advance.

Paul Oosterveld

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[JKRockStomper]

Try this on for size:

** CONVERT ALL NON-WORDSTR(#I) CHARS TO SPACES.

COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "~", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "!", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "@", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "#", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "$", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "%", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "^", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "&", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "*", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "(", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), ")", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "_", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "+", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "`", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "1", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "2", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "3", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "4", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "5", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "6", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "7", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "8", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "9", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "0", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "-", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "+", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "{", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "}", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "[", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "]", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "|", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "\", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), ":", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "'", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), '"', "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), ";", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "<", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), ">", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "?", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), ",", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), ".", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), "/", "").
COMPUTE WORDSTR(#I) = REPLACE(WORDSTR(#I), " ", "").

[Paul Oosterveld]

Gene,

That is indeed the idea

I want to convert string like:
!.õ{<!!!X.õ{<!!!ÀÎ,ü!!!!ÀÎ,ü!!!!I do not know

to
I do not know.

Paul

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[joan casellas]

Hi Paul,

Try with this:

compute name=REPLACE(varname,".","").
compute name=REPLACE(varname,"/","").
compute name=REPLACE(varname,"+","").
.
.
.


Thanks!



Joan Casellas Vega
Media Research Analyst
Phone: [phone]

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Paul Oosterveld
Sent: 15 July 2011 14:38
To: [hidden email]
Subject: Clean up a string variable

Hi,

I am trying to clean up a string variable. I only want to keep characters
(a-z
and A-Z). There should be an easier way than removing all other symbols with
replace I guess. Any suggestions?

Thanks in advance.

Paul Oosterveld

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

The following gets close, I think, but does not get rid of that X in the early part of the string.  I also had to remove some of the characters, because SPSS would not read them in given my settings.

data list list / stringvar(a60).
begin data
"!.{<!!!X.{<!!!,!!!!,!!!!I do not know"
end data.
list.

string good (a53).
compute good = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ ".

loop # = 1 to length(stringvar).
- if index(good,substr(stringvar,#,1)) EQ 0
  stringvar = replace(stringvar,substr(stringvar,#,1),'*').
end loop.
compute stringvar = replace(stringvar,'*','').
list stringvar.

OUTPUT:

stringvar

XI do not know



<quote author="Paul Oosterveld">
Gene,

That is indeed the idea

I want to convert string like:
!.õ{<!!!X.õ{<!!!ÀÎ,ü!!!!ÀÎ,ü!!!!I do not know

to
I do not know.

Paul

=====================
To manage your subscription to SPSSX-L, send a message to
<email>[e-mail] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[David Marso]

Then there is the go for the jugular old school approach ;-)
-------------------------------------------------------------------
DATA LIST/ strvar (A60).
BEGIN DATA
!.õ{<!!!X.õ{<!!!��,ü!!!!��,ü!!!!I do not know
END DATA.
*TESTED OLD SCHOOL (my version of SPSS 11.5 does not have REPLACE function)*.
*Replicates Bruce's example (retains the 'garbage' X at the beginning.
LOOP #=LENGTH(strvar) TO 1 BY -1.
IF ( INDEX("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ ",SUBSTR(strvar,#,1) ) EQ 0 )
     strvar=CONCAT(SUBSTR(strvar,1,#-1),SUBSTR(strvar,#+1)).
END LOOP.
LIST.

* Program 2 (assumes ALL "garbage" characters are at the beginning of string and
* just retains everything to the right of the first "garbage" character).
DATA LIST/ strvar (A60).
BEGIN DATA
!.õ{<!!!X.õ{<!!!��,ü!!!!��,ü!!!!I do not know
END DATA.
LOOP #=LENGTH(strvar) TO 1 BY -1.
+  DO IF (INDEX("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ ",SUBSTR(strvar,#,1) ) EQ 0 ).
+    COMPUTE #FOUND=#.
+    COMPUTE strvar=SUBSTR(strvar,#FOUND+1).
+  END IF.
END LOOP IF #FOUND >0.
LIST.

Bruce Weaver wrote

The following gets close, I think, but does not get rid of that X in the early part of the string.  I also had to remove some of the characters, because SPSS would not read them in given my settings.

data list list / stringvar(a60).
begin data
"!.{<!!!X.{<!!!,!!!!,!!!!I do not know"
end data.
list.

string good (a53).
compute good = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ ".

loop # = 1 to length(stringvar).
- if index(good,substr(stringvar,#,1)) EQ 0
  stringvar = replace(stringvar,substr(stringvar,#,1),'*').
end loop.
compute stringvar = replace(stringvar,'*','').
list stringvar.

OUTPUT:

stringvar

XI do not know



<quote author="Paul Oosterveld">
Gene,

That is indeed the idea

I want to convert string like:
!.õ{<!!!X.õ{<!!!ÀÎ,ü!!!!ÀÎ,ü!!!!I do not know

to
I do not know.

Paul

=====================
To manage your subscription to SPSSX-L, send a message to
<email>[e-mail] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[David Marso]

BTW:  Any attempts to solve this by explicitly listing all possible characters which need to be replaced are doomed to epic failure.

[Albert-Jan Roskam]

Sorry David, can't resist hammering some nails again. Oh no, this lil problem is BEGGING for regexes :-)

>>> import re
>>> s = "!.õ{&lt;!!!X.õ{&lt;!!!��,ü!!!!��,ü!!!!I do not know"
>>> re.sub("(&(l|g)t)|[^a-zA-Z ]", "", s)
'XI do not know'

Wrap this in a datastep and you're done. It also takes care of &gt, should they appear. No idea how to handle the X.

Cheers!!
Albert-Jan

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

--- On Fri, 7/15/11, David Marso <[hidden email]> wrote:

From: David Marso <[hidden email]>
Subject: Re: [SPSSX-L] Clean up a string variable
To: [hidden email]
Date: Friday, July 15, 2011, 5:24 PM

Then there is the go for the jugular old school approach ;-)
-------------------------------------------------------------------
DATA LIST/ strvar (A60).
BEGIN DATA
!.õ{&lt;!!!X.õ{&lt;!!!��,ü!!!!��,ü!!!!I do not know
END DATA.
*TESTED OLD SCHOOL (my version of SPSS 11.5 does not have REPLACE
function)*.
*Replicates Bruce's example (retains the 'garbage' X at the beginning.
LOOP #=LENGTH(strvar) TO 1 BY -1.
IF ( INDEX(&quot;abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ
&quot;,SUBSTR(strvar,#,1) ) EQ 0 )
     strvar=CONCAT(SUBSTR(strvar,1,#-1),SUBSTR(strvar,#+1)).
END LOOP.
LIST.

* Program 2 (assumes ALL &quot;garbage&quot; characters are at the beginning
of string and
* just retains everything to the right of the first &quot;garbage&quot;
character).
DATA LIST/ strvar (A60).
BEGIN DATA
!.õ{&lt;!!!X.õ{&lt;!!!��,ü!!!!��,ü!!!!I do not know
END DATA.
LOOP #=LENGTH(strvar) TO 1 BY -1.
+  DO IF (INDEX(&quot;abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ
&quot;,SUBSTR(strvar,#,1) ) EQ 0 ).
+    COMPUTE #FOUND=#.
+    COMPUTE strvar=SUBSTR(strvar,#FOUND+1).
+  END IF.
END LOOP IF #FOUND &gt;0.
LIST.


Bruce Weaver wrote:

>
> The following gets close, I think, but does not get rid of that X in the
> early part of the string.  I also had to remove some of the characters,
> because SPSS would not read them in given my settings.
>
> data list list / stringvar(a60).
> begin data
> "!.{&lt;!!!X.{&lt;!!!,!!!!,!!!!I do not know&quot;
> end data.
> list.
>
> string good (a53).
> compute good = &quot;abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ
> &quot;.
>
> loop # = 1 to length(stringvar).
> - if index(good,substr(stringvar,#,1)) EQ 0
>   stringvar = replace(stringvar,substr(stringvar,#,1),'*').
> end loop.
> compute stringvar = replace(stringvar,'*','').
> list stringvar.
>
> OUTPUT:
>
> stringvar
>
> XI do not know
>
>
>
> &lt;quote author=&quot;Paul Oosterveld&quot;&gt;
> Gene,
>
> That is indeed the idea
>
> I want to convert string like:
> !.õ{&lt;!!!X.õ{&lt;!!!ÃÃ,ü!!!!ÃÃ,ü!!!!I do not know
>
> to
> I do not know.
>
> Paul
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> &lt;email&gt;LISTSERV@... (not to SPSSX-L), with no body text
> except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Clean-up-a-string-variable-tp4590738p4591065.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
LISTSERV@... (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Art Kendall]

It is unclear what you want as a result.
Open a new instance of SPSS.  Paste the syntax below into a syntax window.
Run it.
Does this do what you want?  Do you want to replace weird characters with something other than a null character (e.g., a blank character)?
the range function has two lists in case your local collating sequence does not have upper and lower case characters in contiguous positions.

Art Kendall
Social Research Consultants

data list list/ dirtystring (a26).
begin data
12345678901234567890123456
1q2w3e4r!q@W#E$R%^Y&u*(mnh
?l>j<h.B.V/q
asdfghjklmnqwertyuiop
"1111111 qwerty"
1A2a3B4b5Z6z
end data.
string cleanstring (a26) testchar(a1) replaceweird (a1).
*change this compute if you want something other than null.
compute replaceweird= "".
do repeat i = 1 to 20.
compute testchar=substr(dirtystring,i,1).
do if range(testchar,'A','Z','a','z').
compute cleanstring = concat(cleanstring,testchar).
ELSE.
compute cleanstring = concat(cleanstring,replaceweird).
end if.
end repeat.
*oddball need for execute do can delete vars.
execute.
delete vars testchar replaceweird.
list.
 

On 7/15/2011 9:38 AM, Paul Oosterveld wrote:

Hi,

I am trying to clean up a string variable. I only want to keep characters (a-z
and A-Z). There should be an easier way than removing all other symbols with
replace I guess. Any suggestions?

Thanks in advance.

Paul Oosterveld

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[David Marso]

I like your use of RANGE!!!
I modified to use LOOP rather than DO REPEAT (DR doesn't like LENGTH(dirty)) and added a couple of missing RTRIMS.  Also using #scratch variables to eliminate need for DELETE VARS.  Finally, since #weird is now #scratch we can condition the compute on only the first case and possibly save a ms or 2 ;-).
--
data list list/ dirty (a26).
begin data
12345678901234567890123456
1q2w3e4r!q@W#E$R%^Y&u*(mnh
?l>j<h.B.V/q
asdfghjklmnqwertyuiop
"1111111 qwerty"
1A2a3B4b5Z6z
end data.

STRING clean (a26) #testch(a1) #weird (a1).
*change this compute if you want something other than null.
IF $CASENUM=1 #weird= "".
LOOP #i = 1 to LENGTH(dirty).
+  COMPUTE #testch=SUBSTR(dirty,#i,1).
+  DO IF RANGE(#testch,'A','Z','a','z').
+    COMPUTE clean = CONCAT(RTRIM(clean),#testch).
+  ELSE.
+    COMPUTE  clean = CONCAT(RTRIM(clean),#weird).
+  END IF.
END LOOP.
LIST.

<quote author="Art Kendall">
It is unclear what you want as a result.
      Open a new instance of SPSS.  Paste the syntax below into a syntax
      window.
      Run it.
      Does this do what you want?  Do you want to replace weird
      characters with something other than a null character (e.g., a
      blank character)?
      the range function has two lists in case your local collating
      sequence does not have upper and lower case characters in
      contiguous positions.
     
      Art Kendall
      Social Research Consultants
     
    data list list/ dirtystring (a26).
        begin data
        12345678901234567890123456
        1q2w3e4r!q@W#E$R%^Y&u*(mnh
        ?l>j<h.B.V/q
        asdfghjklmnqwertyuiop
        "1111111 qwerty"
        1A2a3B4b5Z6z
        end data.
        string cleanstring (a26) testchar(a1) replaceweird (a1).
        *change this compute if you want something other than null.
        compute replaceweird= "".
        do repeat i = 1 to 20.
        compute testchar=substr(dirtystring,i,1).
        do if range(testchar,'A','Z','a','z').
        compute cleanstring = concat(cleanstring,testchar).
        ELSE.
        compute cleanstring = concat(cleanstring,replaceweird).
        end if.
        end repeat.
        *oddball need for execute do can delete vars.
        execute.
        delete vars testchar replaceweird.
        list.
       
   
    On 7/15/2011 9:38 AM, Paul Oosterveld wrote:
   
      Hi,

I am trying to clean up a string variable. I only want to keep characters (a-z
and A-Z). There should be an easier way than removing all other symbols with
replace I guess. Any suggestions?

Thanks in advance.

Paul Oosterveld

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email]  (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


   
 


=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

Nice use of RANGE, Art--I didn't think of that.  But I think blank spaces are also valid characters, so perhaps it should be:

do if range(testchar,'A','Z','a','z',' ',' ').

Cheers,
Bruce

Art Kendall wrote

It is unclear what you want as a result.
      Open a new instance of SPSS.  Paste the syntax below into a syntax
      window.
      Run it.
      Does this do what you want?  Do you want to replace weird
      characters with something other than a null character (e.g., a
      blank character)?
      the range function has two lists in case your local collating
      sequence does not have upper and lower case characters in
      contiguous positions.
     
      Art Kendall
      Social Research Consultants
     
    data list list/ dirtystring (a26).
        begin data
        12345678901234567890123456
        1q2w3e4r!q@W#E$R%^Y&u*(mnh
        ?l>j<h.B.V/q
        asdfghjklmnqwertyuiop
        "1111111 qwerty"
        1A2a3B4b5Z6z
        end data.
        string cleanstring (a26) testchar(a1) replaceweird (a1).
        *change this compute if you want something other than null.
        compute replaceweird= "".
        do repeat i = 1 to 20.
        compute testchar=substr(dirtystring,i,1).
        do if range(testchar,'A','Z','a','z').
        compute cleanstring = concat(cleanstring,testchar).
        ELSE.
        compute cleanstring = concat(cleanstring,replaceweird).
        end if.
        end repeat.
        *oddball need for execute do can delete vars.
        execute.
        delete vars testchar replaceweird.
        list.
       
   
    On 7/15/2011 9:38 AM, Paul Oosterveld wrote:
   
      Hi,

I am trying to clean up a string variable. I only want to keep characters (a-z
and A-Z). There should be an easier way than removing all other symbols with
replace I guess. Any suggestions?

Thanks in advance.

Paul Oosterveld

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email]  (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


   
 


=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[David Marso]

OLD DOG (Me) LEARNS NEW TRICK!!!
From the FM.
RANGE(arg,arg list)  Return 1 if value of argument is in inclusive range of argument list.
I never would have thought to give it multiple pairs of arguments (love how they don't define arg list and there is no example I could find of using RANGE;-)
OTOH:  The dialog box from hell (COMPUTE) implies the nature of the beast RANGE(test,lo hi,lo,hi...).
ROFL, David
-

Bruce Weaver wrote

Nice use of RANGE, Art--I didn't think of that.  But I think blank spaces are also valid characters, so perhaps it should be:

do if range(testchar,'A','Z','a','z',' ',' ').

Cheers,
Bruce

Art Kendall wrote

It is unclear what you want as a result.
      Open a new instance of SPSS.  Paste the syntax below into a syntax
      window.
      Run it.
      Does this do what you want?  Do you want to replace weird
      characters with something other than a null character (e.g., a
      blank character)?
      the range function has two lists in case your local collating
      sequence does not have upper and lower case characters in
      contiguous positions.
     
      Art Kendall
      Social Research Consultants
     
    data list list/ dirtystring (a26).
        begin data
        12345678901234567890123456
        1q2w3e4r!q@W#E$R%^Y&u*(mnh
        ?l>j<h.B.V/q
        asdfghjklmnqwertyuiop
        "1111111 qwerty"
        1A2a3B4b5Z6z
        end data.
        string cleanstring (a26) testchar(a1) replaceweird (a1).
        *change this compute if you want something other than null.
        compute replaceweird= "".
        do repeat i = 1 to 20.
        compute testchar=substr(dirtystring,i,1).
        do if range(testchar,'A','Z','a','z').
        compute cleanstring = concat(cleanstring,testchar).
        ELSE.
        compute cleanstring = concat(cleanstring,replaceweird).
        end if.
        end repeat.
        *oddball need for execute do can delete vars.
        execute.
        delete vars testchar replaceweird.
        list.
       
   
    On 7/15/2011 9:38 AM, Paul Oosterveld wrote:
   
      Hi,

I am trying to clean up a string variable. I only want to keep characters (a-z
and A-Z). There should be an easier way than removing all other symbols with
replace I guess. Any suggestions?

Thanks in advance.

Paul Oosterveld

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email]  (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


   
 


=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[joan casellas]

Hi,

I'm trying to calculate time spent completing a questionnaire. By default, I get the variables in a weird format (numeric), so before I can start to do any calculations I have to do some transformations. So far, I think I've done quite well, but when calculating time spent (delta=time2-time1), I get negative numbers due the 24 hours issue. Any suggestions?

I also would like to recode time1, time2, date1 and date2 into a specific band ranges. Is it possible?

For example:

Val lab time1 time2
1 'less than 5 min'
2 '6 to 10 min'
3 '11 to 15 min'
4 '16 to 20 min'
5 '21 to 25 min'
6 '26 to 30 min'
7 'more than 30 min'.

For example:

Val lab date1 date2
1 'from 27/01/2011 to 28/01/2011'
2 'from 29/01/2011 to 31/01/2011'
3 'from 01/02/2011 to 01/02/2011'
4 'from 02/02/2011 to 03/02/2011'.


****times=time started questionnaire***.
****timef=time finished questionnaire****.
****20110127202010= year, month, day, hours, minutes, seconds)****.


**********************SYNTAX***************************************************.

DATA LIST LIST / id , times, timef .
BEGIN DATA
1, 20110127202010, 20110127202521,
2, 20110128223019, 20110129001031,
3, 20110129012051, 20110129012541,
4, 20110130202010, 20110130202521,
5, 20110131143059, 20110131144531,
6, 20110201162041, 20110201162531,
7, 20110202022031, 20110202023501,
8, 20110203152021, 20110203154531,
9, 20110204232011, 20110205000521
END DATA.

**********************SYNTAX***************************************************.

COMPUTE time1=TIME.HMS(MOD(TRUNC(times/10000),100),MOD(TRUNC(times/100),100),MOD(times,100)).
COMPUTE time2=TIME.HMS(MOD(TRUNC(timef/10000),100),MOD(TRUNC(timef/100),100),MOD(timef,100)).
COMPUTE delta=time2-time1.
FORMATS time1 time2 delta (TIME10).
EXECUTE.


COMPUTE date1=DATE.DMY(MOD(TRUNC(times/1000000),100), MOD(TRUNC(times/100000000),100), MOD(TRUNC(times/10000000000),10000)).
COMPUTE date2=DATE.DMY(MOD(TRUNC(timef/1000000),100), MOD(TRUNC(timef/100000000),100), MOD(TRUNC(timef/10000000000),10000)).
FORMATS date1(DATE10) date2(DATE10).
EXECUTE.

*******************************************************************************.

Any improvements on the syntax would be much appreciated. I would like to find a link with all date formats available for spss. Any suggestions?


Thanks in advance!!!!

Joan Casellas Vega
Media Research Analyst
Phone: [phone]

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[stace swayne]

Hello,

Can anyone on the list suggest a tutorial for running sensitivity and specificity analysis with SPSS?

Thanks,

S

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[parisec]

You can get the sensitivity and specificity by running CROSSTABS and asking for the row or colum percents.

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of stace swayne
Sent: Thursday, July 21, 2011 8:17 AM
To: [hidden email]
Subject: Sensitivity and Specificity Analysis

Hello,

Can anyone on the list suggest a tutorial for running sensitivity and specificity analysis with SPSS?

Thanks,

S

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

And if you have the 2x2 table cell counts rather than a raw data  file, you can enter those cell counts and use WEIGHT, like this:

new file.
dataset close all.

data list list / testresult condition kount (3f5.0).
begin data
1 1 240
1 2  25
2 1  15
2 2 220
end data.

variable labels
 testresult "Test Result"
 condition  "Has Condition" .

value labels testresult condition
 1 "Yes" 2 "No".

weight by kount.

crosstabs testresult by condition / cells = count row col total.

* Sens = column % for cell A (94.1%).
* Spec = column % for cell D (89.8%).
* PV+ = row % for cell A (90.6%).
* PV- = row % for cell D (93.6%).
* Accuracy = sum of Total % for cells A and D = 48.0% + 44.0% = 92.0% .
* Prevalence = Total % in the Total row for the "Has Condition" column = 51%.

You could carry on and compute likelihood ratios if you were so inclined.  I would probably send the table to a dataset via OMS if I wanted to do that though.

HTH.

Parise, Carol A. wrote

You can get the sensitivity and specificity by running CROSSTABS and asking for the row or colum percents.

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of stace swayne
Sent: Thursday, July 21, 2011 8:17 AM
To: [hidden email]
Subject: Sensitivity and Specificity Analysis

Hello,

Can anyone on the list suggest a tutorial for running sensitivity and specificity analysis with SPSS?

Thanks,

S

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[ariel barak]

Hi Joan,

I'm sure someone else on the list will have a way to do this with fewer lines of code but basically, you were almost there. I think what you should have aimed for is to get all of the date and time into one field. Add the syntax below to the bottom of your syntax.

Also, please check your labels - how would you want 315 seconds or 05:15 to show up? The label reads 6 min to 10 min. Maybe you want to have them read "less than 5", "5 to less than 10", etc.

Hope this helps,
Ariel Barak

COMPUTE StartTime=date1+time1.
COMPUTE FinishTime=date2+time2.
EXECUTE.

FORMATS StartTime FinishTime (F11).

COMPUTE Duration=FinishTime-StartTime.
EXECUTE.

* Pick Format of choice  - First is # of seconds, Second is hh:mm:ss.
FORMATS Duration (F11).
FORMATS Duration (TIME8.0).
* Then Apply labels.

COMPUTE TimeFrame=0.
IF Duration<300 TimeFrame=1.
IF Duration>=300 AND Duration<600 TimeFrame=2.
* etc.
EXECUTE.

VALUE LABELS TimeFrame

1 'less than 5 min'

2 '5 to 10 min'.
* Add additional time frames prior to the ".".

On Thu, Jul 21, 2011 at 9:43 AM, joan casellas <[hidden email]> wrote:

Hi,

I'm trying to calculate time spent completing a questionnaire. By default, I get the variables in a weird format (numeric), so before I can start to do any calculations I have to do some transformations. So far, I think I've done quite well, but when calculating time spent (delta=time2-time1), I get negative numbers due the 24 hours issue. Any suggestions?

I also would like to recode time1, time2, date1 and date2 into a specific band ranges. Is it possible?

For example:

Val lab time1 time2
1 'less than 5 min'
2 '6 to 10 min'
3 '11 to 15 min'
4 '16 to 20 min'
5 '21 to 25 min'
6 '26 to 30 min'
7 'more than 30 min'.

For example:

Val lab date1 date2
1 'from 27/01/2011 to 28/01/2011'
2 'from 29/01/2011 to 31/01/2011'
3 'from 01/02/2011 to 01/02/2011'
4 'from 02/02/2011 to 03/02/2011'.


****times=time started questionnaire***.
****timef=time finished questionnaire****.
****20110127202010= year, month, day, hours, minutes, seconds)****.


**********************SYNTAX***************************************************.

DATA LIST LIST / id , times, timef .
BEGIN DATA
1, 20110127202010, 20110127202521,
2, 20110128223019, 20110129001031,
3, 20110129012051, 20110129012541,
4, 20110130202010, 20110130202521,
5, 20110131143059, 20110131144531,
6, 20110201162041, 20110201162531,
7, 20110202022031, 20110202023501,
8, 20110203152021, 20110203154531,
9, 20110204232011, 20110205000521
END DATA.

**********************SYNTAX***************************************************.

COMPUTE time1=TIME.HMS(MOD(TRUNC(times/10000),100),MOD(TRUNC(times/100),100),MOD(times,100)).
COMPUTE time2=TIME.HMS(MOD(TRUNC(timef/10000),100),MOD(TRUNC(timef/100),100),MOD(timef,100)).
COMPUTE delta=time2-time1.
FORMATS time1 time2 delta (TIME10).
EXECUTE.


COMPUTE date1=DATE.DMY(MOD(TRUNC(times/1000000),100), MOD(TRUNC(times/100000000),100), MOD(TRUNC(times/10000000000),10000)).
COMPUTE date2=DATE.DMY(MOD(TRUNC(timef/1000000),100), MOD(TRUNC(timef/100000000),100), MOD(TRUNC(timef/10000000000),10000)).
FORMATS date1(DATE10) date2(DATE10).
EXECUTE.

*******************************************************************************.

Any improvements on the syntax would be much appreciated. I would like to find a link with all date formats available for spss. Any suggestions?


Thanks in advance!!!!

Joan Casellas Vega
Media Research Analyst
Phone: [phone]

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

I agree with Ariel that combining the dates and times into date-time variables is key here.  I would make a few small changes to formatting, and given that the date and time components are not needed after they are combined to create date-time variables, I'd use scratch variables.  It might be good to trap negative time intervals too.  Here's my re-write:


DATA LIST LIST / id , times, timef .
BEGIN DATA
1, 20110127202010, 20110127202521,
2, 20110128223019, 20110129001031,
3, 20110129012051, 20110129012541,
4, 20110130202010, 20110130202521,
5, 20110131143059, 20110131144531,
6, 20110201162041, 20110201162531,
7, 20110202022031, 20110202023501,
8, 20110203152021, 20110203154531,
9, 20110204232011, 20110205000521
10, 20110204232011, 20110204000521
END DATA.

COMPUTE #t1=TIME.HMS(MOD(TRUNC(times/10000),100),MOD(TRUNC(times/100),100),MOD(times,100)).
COMPUTE #t2=TIME.HMS(MOD(TRUNC(timef/10000),100),MOD(TRUNC(timef/100),100),MOD(timef,100)).

COMPUTE #d1=DATE.DMY(MOD(TRUNC(times/1000000),100), MOD(TRUNC(times/100000000),100), MOD(TRUNC(times/10000000000),10000)).
COMPUTE #d2=DATE.DMY(MOD(TRUNC(timef/1000000),100), MOD(TRUNC(timef/100000000),100), MOD(TRUNC(timef/10000000000),10000)).

COMPUTE t1=#d1+#t1.
COMPUTE t2=#d2+#t2.

COMPUTE ETsec = datediff(t2,t1,"seconds").

* To avoid doing mental arithmetic, use a scratch variable (#)
* that is equal to the duration in seconds / 60.

* Select (by uncommenting) one of the following 3 COMPUTE commands,
* depending on  whether you want to round to the nearest minute, truncate  
* the seconds, or do neither.  Given the value labels that the OP
* listed, it sounded like rounding the nearest minute might be desired.
* If one of the other options is chosen, the value labels should be
* changed to something more appropriate.

*compute # = etsec/60. /* No rounding or truncation.
compute # = rnd(ETsec/60). /* Round to nearest minute.
*compute # = trunc(ETsec/60). /* Truncate the seconds.

IF # LE 0 TimeFrame=0.
IF # GT 0 AND # LE 5 TimeFrame=1.
IF # GT 5 AND # LE 10 TimeFrame=2.
IF # GT 10 AND # LE 15 TimeFrame=3.
IF # GT 15 AND # LE 20 TimeFrame=4.
IF # GT 20 AND # LE 25 TimeFrame=5.
IF # GT 25 AND # LE 30 TimeFrame=6.
IF # GT 30 TimeFrame=7.
EXECUTE.

var lab
 ETsec "Elapsed time (seconds)"
 timeframe "Elapsed time (minutes)".
Val lab timeframe
 0 'ERROR - Negative Time!"
 1 '5 min or less'
 2 '6 to 10 min'
 3 '11 to 15 min'
 4 '16 to 20 min'
 5 '21 to 25 min'
 6 '26 to 30 min'
 7 'more than 30 min'.

FORMATS
 t1 t2 (datetime20) /
 etsec etmin (f8.0).

* Check that values within categories are within range.

means etsec by timeframe / cells = min max.

freq timeframe.

ariel barak wrote

Hi Joan,

I'm sure someone else on the list will have a way to do this with fewer
lines of code but basically, you were almost there. I think what you should
have aimed for is to get all of the date and time into one field. Add the
syntax below to the bottom of your syntax.

Also, please check your labels - how would you want 315 seconds or 05:15 to
show up? The label reads 6 min to 10 min. Maybe you want to have them read
"less than 5", "5 to less than 10", etc.

Hope this helps,
Ariel Barak

COMPUTE StartTime=date1+time1.
COMPUTE FinishTime=date2+time2.
EXECUTE.

FORMATS StartTime FinishTime (F11).

***** COMMENT *****   I would use DateTime20 rather than F11 here.


COMPUTE Duration=FinishTime-StartTime.
EXECUTE.

* Pick Format of choice  - First is # of seconds, Second is hh:mm:ss.
FORMATS Duration (F11).
FORMATS Duration (TIME8.0).
* Then Apply labels.

COMPUTE TimeFrame=0.
IF Duration<300 TimeFrame=1.
IF Duration>=300 AND Duration<600 TimeFrame=2.
* etc.
EXECUTE.

VALUE LABELS TimeFrame
1 'less than 5 min'
2 '5 to 10 min'.
* Add additional time frames prior to the ".".

On Thu, Jul 21, 2011 at 9:43 AM, joan casellas
<[hidden email]>wrote:

> Hi,
>
> I'm trying to calculate time spent completing a questionnaire. By default,
> I get the variables in a weird format (numeric), so before I can start to do
> any calculations I have to do some transformations. So far, I think I've
> done quite well, but when calculating time spent (delta=time2-time1), I get
> negative numbers due the 24 hours issue. Any suggestions?
>
> I also would like to recode time1, time2, date1 and date2 into a specific
> band ranges. Is it possible?
>
> For example:
>
> Val lab time1 time2
> 1 'less than 5 min'
> 2 '6 to 10 min'
> 3 '11 to 15 min'
> 4 '16 to 20 min'
> 5 '21 to 25 min'
> 6 '26 to 30 min'
> 7 'more than 30 min'.
>
> For example:
>
> Val lab date1 date2
> 1 'from 27/01/2011 to 28/01/2011'
> 2 'from 29/01/2011 to 31/01/2011'
> 3 'from 01/02/2011 to 01/02/2011'
> 4 'from 02/02/2011 to 03/02/2011'.
>
>
> ****times=time started questionnaire***.
> ****timef=time finished questionnaire****.
> ****20110127202010= year, month, day, hours, minutes, seconds)****.
>
>
>
> **********************SYNTAX***************************************************.
>
> DATA LIST LIST / id , times, timef .
> BEGIN DATA
> 1, 20110127202010, 20110127202521,
> 2, 20110128223019, 20110129001031,
> 3, 20110129012051, 20110129012541,
> 4, 20110130202010, 20110130202521,
> 5, 20110131143059, 20110131144531,
> 6, 20110201162041, 20110201162531,
> 7, 20110202022031, 20110202023501,
> 8, 20110203152021, 20110203154531,
> 9, 20110204232011, 20110205000521
> END DATA.
>
>
> **********************SYNTAX***************************************************.
>
> COMPUTE
> time1=TIME.HMS(MOD(TRUNC(times/10000),100),MOD(TRUNC(times/100),100),MOD(times,100)).
> COMPUTE
> time2=TIME.HMS(MOD(TRUNC(timef/10000),100),MOD(TRUNC(timef/100),100),MOD(timef,100)).
> COMPUTE delta=time2-time1.
> FORMATS time1 time2 delta (TIME10).
> EXECUTE.
>
>
> COMPUTE date1=DATE.DMY(MOD(TRUNC(times/1000000),100),
> MOD(TRUNC(times/100000000),100), MOD(TRUNC(times/10000000000),10000)).
> COMPUTE date2=DATE.DMY(MOD(TRUNC(timef/1000000),100),
> MOD(TRUNC(timef/100000000),100), MOD(TRUNC(timef/10000000000),10000)).
> FORMATS date1(DATE10) date2(DATE10).
> EXECUTE.
>
>
> *******************************************************************************.
>
> Any improvements on the syntax would be much appreciated. I would like to
> find a link with all date formats available for spss. Any suggestions?
>
>
> Thanks in advance!!!!
>
> Joan Casellas Vega
> Media Research Analyst
> Phone: [phone]
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>

[Ling Ting]

I am out of office and will return on July 25th.  I will have very limited access to email. If you need immediate assistance please contact 479-575-2905.  Thank you.