Encoding issue for OE/String fields

[Kevin longfield]

Hello,

We collect data from across the globe and ask survey takers to enter short
phrases so they are entering in their response in their native language,
sometimes I get these types of “responses” in the data records
Field1 = “????? ????”
Field1 = “ch�quier”
I know this is an Encoding Issue. I am using UTF-8 for everything and that
is what this project was built on.  
The question marks are coming from double byte languages (Japan, Chinese,
Arabic) and the Black diamonds are from Latin markets or languages that use
Tildes, Umlauts, etc.

This wave I cannot save out my syntax file as it says “This file contains
one or more characters not recognized by the selected or default encoding.
The syntax could produce errors or produce unintended results.”

In the syntax file I am using the Recode statement as a way to convert the
strings into a code so we can analyze the data.
Do repeat a= Field1 Field2 Field3 /
b=Field1_coded Field2_coded Field3_coded .
if( a  eq “????? ????” ) b = 998.
if( a  eq “ch�quier”) b = 998.
End repeat.

So how can I work this out in SPSS?  I wanted to weigh in and see what or if
I can do anything in SPSS to work through this.

Thanks in advance for the assistance,
~Kevin.




--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Jon Peck]

I am guessing that the inputs are not sav files, since those would have their encoding marked, and they should be correctly converted to Unicode (UTF8) assuming that you are really in Uncode mode.  If the inputs are text files, you would need to declare the encoding when you read them in the DATA LIST or GET DATA commands.

As for the syntax file, you should be able to save that in the Unicode encoding.

On Tue, Feb 5, 2019 at 9:37 AM Kevin longfield <[hidden email]> wrote:

Hello,

We collect data from across the globe and ask survey takers to enter short
phrases so they are entering in their response in their native language,
sometimes I get these types of “responses” in the data records
Field1 = “????? ????”
Field1 = “ch�quier”
I know this is an Encoding Issue. I am using UTF-8 for everything and that
is what this project was built on. 
The question marks are coming from double byte languages (Japan, Chinese,
Arabic) and the Black diamonds are from Latin markets or languages that use
Tildes, Umlauts, etc.

This wave I cannot save out my syntax file as it says “This file contains
one or more characters not recognized by the selected or default encoding.
The syntax could produce errors or produce unintended results.”

In the syntax file I am using the Recode statement as a way to convert the
strings into a code so we can analyze the data.
Do repeat a= Field1 Field2 Field3 /
b=Field1_coded Field2_coded Field3_coded .
if( a  eq “????? ????” ) b = 998.
if( a  eq “ch�quier”) b = 998.
End repeat.

So how can I work this out in SPSS?  I wanted to weigh in and see what or if
I can do anything in SPSS to work through this.

Thanks in advance for the assistance,
~Kevin.




--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

--
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD