Modify

Opened 5 years ago

Last modified 5 years ago

#7785 new Bugs

karma real generator limits precision to digits10 no matter exponent

Reported by: strk@… Owned by: Hartmut Kaiser
Milestone: To Be Determined Component: spirit
Version: Boost 1.52.0 Severity: Problem
Keywords: Cc:

Description

The real_inserter::call_n method here is limiting the user-provided precision by the http://svn.boost.org/svn/boost/trunk/boost/spirit/home/karma/numeric/detail/real_utils.hpp

But that value is later used to determine how many decimal digits to print, independently on where the first non-zero digit is. This means overlimiting the actual number of significant digit for no evident reason.

The user provided ::precision() function has access to the full number and thus can judge when to provide more or less precision depending on magnitude. Why not trusting that ?

Attachments (0)

Change History (4)

comment:1 Changed 5 years ago by strk@…

For reference, the problem showed up here: https://github.com/mapnik/mapnik/pull/1632 where we're trying to get an output comparable with that of iostream with setprecision(16)

comment:2 Changed 5 years ago by anonymous

On further analysis I found out that the "fraction_part" function does not even get more than 13 significant digits when the number is lower than 10e-3, so there's nothing an overridden fraction_part method could do

comment:3 Changed 5 years ago by Sandro Santilli <strk@…>

The problem is in call_n, where fractional part is normalized even if the format flag is requesting fixed precision, despite the comment above (saying normalization is only needed for scientific notation).

This patch would fix my case:

--- real_utils.hpp.000  2012-12-13 13:42:01.000000000 +0100
+++ real_utils.hpp      2012-12-13 13:43:02.000000000 +0100
@@ -85,5 +85,5 @@
             // get correct precision for generated number
             unsigned precision = p.precision(n);
-            if (std::numeric_limits<U>::digits10) 
+            if (std::numeric_limits<U>::digits10 && !(Policies::fmtflags::fixed & flags))
             {
                 // limit generated precision to digits10, if defined

comment:4 Changed 5 years ago by Joel de Guzman

Owner: changed from Joel de Guzman to Hartmut Kaiser

Modify Ticket

Change Properties
Set your email in Preferences
Action
as new The owner will remain Hartmut Kaiser.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.