Modify

Ticket #5289 (closed Bugs: fixed)

Opened 3 years ago

Last modified 2 years ago

Library that uses Boost Filesystem would not start when the stdandard C++ library does not support the current locale

Reported by: artyomtnk@… Owned by: bemandawes
Milestone: Boost 1.47.0 Component: filesystem
Version: Boost 1.46.0 Severity: Showstopper
Keywords: Cc:

Description

The standard C++ library required to support only C and POSIX locales, for others it may not be possible to create locale object.

Thus std::locale("") would fail if the default system locale is not C or POSIX on may systems.

For example, GCC support locales only on Linux so basically global_locale() would throw anywhere except Linux, in the Boost.Filesystem's code this means that it would not be possible to even start program under Solaris or FreeBSD.

Even under Linux it is not possible to get to main() when linked with boost.filesystem shared library and the locale (LANG env. variable) is not supported.

Simple test. Link the program with boost.filesystem and try to run the program giving LANG=foo_BAR.UTF-8 - the program would not even start. Of course if may happen even for legal locales like de_AU.UTF-8 that are not configured on this operating system.

I had attached the patch with following changes:

If std::locale("") fails, check if current system locale's charset is UTF-8, if so install utf-8 facet otherwise fallback to global locale.

It would cover most of frequent cases when the standard library does not support the global locale and would not fail on locale generation, it may fail later on conversion if the charset is not supported. But it at least would work for ASCII subset.

The second change in the attached patch is that it always tries to use std::locale(""), for example MSVC fully supports them so why would not use the default one.

This is better then implement codecvt over Wide/Narrow? API as it would not work correctly on boundaries of double width encoding like Shift-JIS or BIG5.

For example of you have sequence "XYYZ" where X, YY and X are "characters" where YY is double width encoded character then if you would do partial conversion of "XY" it would return "error", rather then "partial" which is wrong.

So I think it is always best idea to give a best shot do standard C++ library rather then install your own facets.

The patch is attached.

Artyom Beilis (Author of Boost.Locale)

Attachments

boost_filesystem.patch Download (2.5 KB) - added by artyomtnk@… 3 years ago.

Change History

Changed 3 years ago by artyomtnk@…

comment:1 Changed 3 years ago by Sam Morris <sam@…>

Duplicate of #4688?

comment:2 Changed 3 years ago by artyomtnk@…

It is duplicate but the patch attached there is not correct. As even on Linux it may not start if LANG=xx_YY.UTF-8 that is not configured on this OS (locale -a) then it would fail as well.

So the patch there is wrong.

My patch handles it much better even on system when gcc does not support non-POSIX locales.

comment:3 Changed 3 years ago by bemandawes

  • Status changed from new to assigned

I'm concerned that this patch doesn't include any test cases. I'd really like to first add additional test cases that fail with the current implementation and then pass when the patch is supplied.

Thanks,

--Beman

comment:4 Changed 3 years ago by artyomtnk@…

I'm concerned that this patch doesn't include any test cases. I'd really like to first add additional test cases that fail with the current implementation and then pass when the patch is supplied

The simplest test that can be done under any platform is:

  1. link the library to simple executable that does nothing.
  2. Run it with environment variable "LANG" set to "xx_YY.UTF-8" (illegal locale) and check if the library starts (fails to start without this patch).
  3. Check that the boost::filesystem::path's locale has UTF-8 facet (trivially by converting one wide NON-ASCII character to char.

It is very easy to reproduce this bug (by setting illegal locale environment variable) but it requires some non-trivial scripting with BBv2 (that I unfortunately not too familiar with it)

comment:5 Changed 3 years ago by anonymous

  • Status changed from assigned to closed
  • Resolution set to fixed

Fixed by changeset 72855, Fix problem of locale("") exception being thrown before main() starts on misconfigured (e.g. LANG="bad name") POSIX systems. Resolves the most serious aspect of tickets 4688, 5100, 5289.

Boost 1.47.0 was the first release to include this fix.

--Beman

comment:6 Changed 2 years ago by bbazso@…

I do not think that this issue should be closed or perhaps another one needs to be opened, but there is definitely still a problem associated with this issue. When I call the function:

boost::filesystem::create_directories (iDirectory);

To create a directory, I get the error:

locale::facet::_S_create_c_locale name not valid

I am running Boost 1.47.0 on the following system:

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=9.04
DISTRIB_CODENAME=jaunty
DISTRIB_DESCRIPTION="Ubuntu 9.04"

I manually patched Boost 1.47.0 with the patch in this issue and it solves the issue. So I think that a issue should be opened or this one re-opened.

View

Add a comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
The resolution will be deleted. Next status will be 'reopened'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.