Locale parsing should be more robust

[wichert@…]

Apple did a somewhat stupid thing on OSX 10.6: it defaults to setting LC_CTYPE=UTF-8. This is obviously a bad value since UTF-8 is a character encoding and not a locale. Unfortunately Babel does not handle this very elegantly: it aborts in parse_locale with ValueError: expected only letters, got 'utf-8'.

Since this makes it impossible for people to run Babel-using applications on OSX 10.6 unless they manually correct LC_CTYPE I suggest that Babel be a bit more relaxed about errors like this: instead of raising an exception it should log a warning and ignore the bogus value.

[cg@…]

This was fixed 8 months ago, so I guess it can be closed

[cg@…]

Oh, sorry wrong ticket, meant #173

[jruigrok]

Trying to get into contact with relevant Apple people about this feature.

Furthermore, need to fix it to make it more tolerant. Problem will be the default locale though. Need to think about this.

[anonymous]

Replying to jruigrok:

Trying to get into contact with relevant Apple people about this feature.

Furthermore, need to fix it to make it more tolerant. Problem will be the default locale though. Need to think about this.

Very bad

[Thijs Triemstra <lists@…>]

Trac 0.12 is one of the packages affected because it now depends on Babel (and I initially opened an ​invalid ticket there). One of my own Pylons apps also depends on Babel and trying to install it on Snow Leopard results in this error:

Traceback (most recent call last):
  File "setup.py", line 84, in <module>
    "Topic :: Software Development :: Libraries :: Python Modules",
  File "/usr/local/lib/python2.6/distutils/core.py", line 113, in setup
    _setup_distribution = dist = klass(attrs)
  File "/usr/local/lib/python2.6/site-packages/distribute-0.6.13-py2.6.egg/setuptools/dist.py", line 225, in __init__
    _Distribution.__init__(self,attrs)
  File "/usr/local/lib/python2.6/distutils/dist.py", line 270, in __init__
    self.finalize_options()
  File "/usr/local/lib/python2.6/site-packages/distribute-0.6.13-py2.6.egg/setuptools/dist.py", line 258, in finalize_options
    ep.load()(self, ep.name, value)
  File "/usr/local/lib/python2.6/site-packages/distribute-0.6.13-py2.6.egg/pkg_resources.py", line 1954, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/usr/local/lib/python2.6/site-packages/Babel-0.9.5-py2.6.egg/babel/messages/__init__.py", line 16, in <module>
    from babel.messages.catalog import *
  File "/usr/local/lib/python2.6/site-packages/Babel-0.9.5-py2.6.egg/babel/messages/catalog.py", line 30, in <module>
    from babel.dates import format_datetime
  File "/usr/local/lib/python2.6/site-packages/Babel-0.9.5-py2.6.egg/babel/dates.py", line 34, in <module>
    LC_TIME = default_locale('LC_TIME')
  File "/usr/local/lib/python2.6/site-packages/Babel-0.9.5-py2.6.egg/babel/core.py", line 642, in default_locale
    return '_'.join(filter(None, parse_locale(locale)))
  File "/usr/local/lib/python2.6/site-packages/Babel-0.9.5-py2.6.egg/babel/core.py", line 763, in parse_locale
    raise ValueError('expected only letters, got %r' % lang)
ValueError: expected only letters, got 'utf-8'

[Thijs Triemstra <lists@…>]

Also see ​http://bugs.python.org/issue2173 for Python's approach to this issue.

[Thijs Triemstra <lists@…>]

This is on Mac OSX 10.6.3 with Python 2.6.5 built from source:

$ locale
LANG=
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

[Thijs Triemstra <lists@…>]

Fixed it with:

export LC_CTYPE="en_GB.utf-8"

[fschwarz]

That error sounds quite severe and I'll fix it before releasing 0.9.6.

[fschwarz]

Patch to ignore bad locale settings

[fschwarz]

I think the correct fix is to catch the exception in default_locale() and to keep parse_locale() as it is currently (including the exception for invalid locale strings).

I'm pretty confident that my patch fixes the actual problem but it'd be nice to get some feedback from a Mac user about that.

Also I found another (smaller) problem: The docstring for default_locale() says that the return type is str. However if none of the environment variables contain a valid locale, the function returns None. Do we need to fix that?

[yota]

For the records, I could catch this bug with :

$ locale
LANG=C
LC_CTYPE=iso_8859_1
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_ALL=

which raise a

ValueError: 'iso_8859_1' is not a valid locale identifier

I agree that the local is weirdly set, but I'm not admin...

[fschwarz]

Replying to yota:

For the records, I could catch this bug with : …

Thanks for your feedback, I just checked and my patch​ fixes your problem. Afterwards it would return the locale string 'en_US_POSIX' which I believe is correct given your settings.

Just waiting for a short review from other developers here :-)

[wichert@…]

Looks fine to me.

[fschwarz]

Ok, fixes committed in r586 (trunk) and r587 (0.9 branch). For the docstring issue I opened ticket #261.