"Text value unmappable to the current locale"

[Albert-Jan Roskam]

Hi,

I want to create a test sav file that contains some text in Arabic (the text between BEGIN DATA - END DATA is some copy-pasted text from http://www.bbc.co.uk/arabic/). When I run the code below, I get the following message: "Text value unmappable to the current locale". What can I do to make this work? I want to use this file to test a Python program that reads spss files. If the program reads a sav file from a foreign locale, I want to temporarily set the locale so the terminal encoding is compatible with the file encoding. (I hope I understood this fuzzy topic!)

preserve.
set unicode = on.
set locale 'Arabic'.
data list free x (500).
begin data
الصومال: حركة الشباب تتهم الأمم المتحدة بالمبالغة في وصف حال المجاعة لأسباب سياسية
منطقة اليورو توافق على حزمة مساعدات جديدة لليونان
العفو الدولية: مشروع قانون الإرهاب السعودي سيجرم المعارضة المشروعة
مصر: شرف يتعهد بتحقيق أهداف الثورة
اقالة صحفي بارز من صحيفة يمتلكها مردوخ على خلفية فضيحة التنصت
end data.
save outfile = "d:/temp/arabic.sav".
restore.

Cheers!!
Albert-Jan

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[Jon K Peck]

This will work correctly with a little caution, but the DATA LIST command below isn't quite right.  It should be something like
data list free/ x (A500).

The problem you are seeing, though, has to do with how input text is processed.  You are doubtless selecting the entire block of syntax in the syntax window and running it.  But the block is read into the input queue before any of it is executed.  So the error is being issued before the unicode and arabic locale settings take effect.

Run the first part,
preserve.
set unicode = on.
set locale 'Arabic'.

and then run the rest.  That way, those commands will have been executed when the data are encountered.

Jon Peck
Senior Software Engineer, IBM
[hidden email]
new phone: [phone]




From:        Albert-Jan Roskam <[hidden email]>
To:        [hidden email]
Date:        07/22/2011 05:57 AM
Subject:        [SPSSX-L] "Text value unmappable to the current locale"
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

Hi,

I want to create a test sav file that contains some text in Arabic (the text between BEGIN DATA - END DATA is some copy-pasted text from http://www.bbc.co.uk/arabic/). When I run the code below, I get the following message: "Text value unmappable to the current locale". What can I do to make this work? I want to use this file to test a Python program that reads spss files. If the program reads a sav file from a foreign locale, I want to temporarily set the locale so the terminal encoding is compatible with the file encoding. (I hope I understood this fuzzy topic!)

preserve.
set unicode = on.
set locale 'Arabic'.
data list free x (500).
begin data
الصومال: حركة الشباب تتهم الأمم المتحدة بالمبالغة في وصف حال المجاعة لأسباب سياسية
منطقة اليورو توافق على حزمة مساعدات جديدة لليونان
العفو الدولية: مشروع قانون الإرهاب السعودي سيجرم المعارضة المشروعة
مصر: شرف يتعهد بتحقيق أهداف الثورة
اقالة صحفي بارز من صحيفة يمتلكها مردوخ على خلفية فضيحة التنصت
end data.
save outfile = "d:/temp/arabic.sav".
restore.

Cheers!!
Albert-Jan

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[Albert-Jan Roskam]

Hi Jon,

aah, how stupid of me, one backslash. Yes, it works indeed. And my also code works! It spits out an arabic csv file! Does Spss have an equivalent to .Rprofile or autoexec.sas? autoscripts perhaps? Would be useful to put the first bit of Spss SET syntax there (for some people at least).

Re: the spssio module: do there happen to be any dependent libraries available on the IBM website? I've been looking for them, but a working libirc.so is impossible to find.

Cheers!!
Albert-Jan

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

--- On Fri, 7/22/11, Jon K Peck <[hidden email]> wrote:

From: Jon K Peck <[hidden email]>
Subject: Re: [SPSSX-L] "Text value unmappable to the current locale"
To: "Albert-Jan Roskam" <[hidden email]>
Cc: [hidden email]
Date: Friday, July 22, 2011, 3:46 PM

This will work correctly with a little caution, but the DATA LIST command below isn't quite right.  It should be something like
data list free/ x (A500).

The problem you are seeing, though, has to do with how input text is processed.  You are doubtless selecting the entire block of syntax in the syntax window and running it.  But the block is read into the input queue before any of it is executed.  So the error is being issued before the unicode and arabic locale settings take effect.

Run the first part,
preserve.
set unicode = on.
set locale 'Arabic'.

and then run the rest.  That way, those commands will have been executed when the data are encountered.

Jon Peck
Senior Software Engineer, IBM
[hidden email]
new phone: [phone]




From:        Albert-Jan Roskam <[hidden email]>
To:        [hidden email]
Date:        07/22/2011 05:57 AM
Subject:        [SPSSX-L] "Text value unmappable to the current locale"
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

Hi,

I want to create a test sav file that contains some text in Arabic (the text between BEGIN DATA - END DATA is some copy-pasted text from http://www.bbc.co.uk/arabic/). When I run the code below, I get the following message: "Text value unmappable to the current locale". What can I do to make this work? I want to use this file to test a Python program that reads spss files. If the program reads a sav file from a foreign locale, I want to temporarily set the locale so the terminal encoding is compatible with the file encoding. (I hope I understood this fuzzy topic!)

preserve.
set unicode = on.
set locale 'Arabic'.
data list free x (500).
begin data
الصومال: حركة الشباب تتهم الأمم المتحدة بالمبالغة في وصف حال المجاعة لأسباب سياسية
منطقة اليورو توافق على حزمة مساعدات جديدة لليونان
العفو الدولية: مشروع قانون الإرهاب السعودي سيجرم المعارضة المشروعة
مصر: شرف يتعهد بتحقيق أهداف الثورة
اقالة صحفي بارز من صحيفة يمتلكها مردوخ على خلفية فضيحة التنصت
end data.
save outfile = "d:/temp/arabic.sav".
restore.

Cheers!!
Albert-Jan

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[Spousta Jan]

Děkuji za Váš e-mail. Budu si jej však moci přečíst nejdříve 1.8.2011. Poštu nemám přesměrovanou. V případě potřeby kontaktujte prosím Michala Seiferta, [hidden email]. S přátelským pozdravem, Jan Spousta

Thank you for your e-mail. However, I'll be able to read it earliest on 8/1. My e-mail box isn't forwarded. In urgent cases contact please Michal Seifert, [hidden email]. Kind regards, Jan Spousta

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD