Modify

Ticket #7785 (new Bugs)

Opened 17 months ago

Last modified 17 months ago

karma real generator limits precision to digits10 no matter exponent

Reported by: strk@… Owned by: hkaiser
Milestone: To Be Determined Component: spirit
Version: Boost 1.52.0 Severity: Problem
Keywords: Cc:

Description

The real_inserter::call_n method here is limiting the user-provided precision by the  http://svn.boost.org/svn/boost/trunk/boost/spirit/home/karma/numeric/detail/real_utils.hpp

But that value is later used to determine how many decimal digits to print, independently on where the first non-zero digit is. This means overlimiting the actual number of significant digit for no evident reason.

The user provided ::precision() function has access to the full number and thus can judge when to provide more or less precision depending on magnitude. Why not trusting that ?

Attachments

Change History

comment:1 Changed 17 months ago by strk@…

For reference, the problem showed up here:  https://github.com/mapnik/mapnik/pull/1632 where we're trying to get an output comparable with that of iostream with setprecision(16)

comment:2 Changed 17 months ago by anonymous

On further analysis I found out that the "fraction_part" function does not even get more than 13 significant digits when the number is lower than 10e-3, so there's nothing an overridden fraction_part method could do

comment:3 Changed 17 months ago by Sandro Santilli <strk@…>

The problem is in call_n, where fractional part is normalized even if the format flag is requesting fixed precision, despite the comment above (saying normalization is only needed for scientific notation).

This patch would fix my case:

--- real_utils.hpp.000  2012-12-13 13:42:01.000000000 +0100
+++ real_utils.hpp      2012-12-13 13:43:02.000000000 +0100
@@ -85,5 +85,5 @@
             // get correct precision for generated number
             unsigned precision = p.precision(n);
-            if (std::numeric_limits<U>::digits10) 
+            if (std::numeric_limits<U>::digits10 && !(Policies::fmtflags::fixed & flags))
             {
                 // limit generated precision to digits10, if defined

comment:4 Changed 17 months ago by djowel

  • Owner changed from djowel to hkaiser
View

Add a comment

Modify Ticket

Change Properties
<Author field>
Action
as new
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.