Opened 9 years ago

Closed 8 years ago

#4340 closed Patches (fixed)

property_tree xml parser failed to handle wstream with i18n char

Reported by: zhuo.qiang@… Owned by: Sebastian Redl
Milestone: Boost 1.45.0 Component: property_tree
Version: Boost 1.44.0 Severity: Showstopper
Keywords: property_tree i18n unicode Cc:

Description

boost::property_tree::read_xml(wistream&, wptree&) throw exception if tag/value/attribute has a unicode with '\0' inside,

for example, for chinese character L'\u4E00', an exception will be throwed because the parser treat L'\u4E00' as '\0'.

for example, the following test case won't pass:

BOOST_AUTO_TEST_CASE(test_i18n_xml_tag_name) {

wistringstream in(L"<\u4E00>abc</\u4E00>"); property_tree::wptree pt; BOOST_REQUIRE_NO_THROW(property_tree::read_xml(in, pt)); BOOST_CHECK(L"abc" == pt.get<wstring>(L"\u4E00"));

}

BOOST_AUTO_TEST_CASE(test_i18n_xml_attribute_name) {

wistringstream in(L"<tag \u4E00=\"abc\">def</tag>"); property_tree::wptree pt; BOOST_REQUIRE_NO_THROW(property_tree::read_xml(in, pt)); BOOST_CHECK(L"abc" == pt.get<wstring>(L"tag.<xmlattr>.\u4E00"));

}

BOOST_AUTO_TEST_CASE(test_i18n_xml_attribute_value) {

wistringstream in(L"<tag attribute=\"\u4E00\">def</tag>"); property_tree::wptree pt; BOOST_REQUIRE_NO_THROW(property_tree::read_xml(in, pt)); BOOST_CHECK(L"\u4E00" == pt.get<wstring>(L"tag.<xmlattr>.attribute"));

}

BOOST_AUTO_TEST_CASE(test_i18n_xml_tag_value) {

wistringstream in(L"<tag>\u4E00</tag>"); property_tree::wptree pt; BOOST_REQUIRE_NO_THROW(property_tree::read_xml(in, pt)); BOOST_CHECK(L"\u4E00" == pt.get<wstring>(L"tag"));

}

The fix is treat the all char whose value above 255 same as 'z' when doing semantic action.

Attachments (1)

property_tree_xml_i18n_fix.patch (3.9 KB) - added by zhuo.qiang@… 9 years ago.
Fix for the property_tree xml parser parse i18n stream bug against svn revision 62903

Download all attachments as: .zip

Change History (4)

Changed 9 years ago by zhuo.qiang@…

Fix for the property_tree xml parser parse i18n stream bug against svn revision 62903

comment:1 Changed 8 years ago by zhuo.qiang@…

Milestone: Boost 1.43.0Boost-1.45.0
Severity: ProblemShowstopper
Summary: property_tree xml parser do not handle wstream with i18n char wellproperty_tree xml parser failed to handle wstream with i18n char
Version: Boost Development TrunkBoost 1.44.0

It's better that this small patch (as well as the unit test) could go into boost 1.45

comment:2 Changed 8 years ago by Sebastian Redl

Applied on trunk, waiting for tests to cycle.

comment:3 Changed 8 years ago by Sebastian Redl

Resolution: fixed
Status: newclosed

(In [66479]) Merge simple fix for non-ascii XML parsing to release. Fixes bug 4340.

Note: See TracTickets for help on using tickets.