Modify

Ticket #4590 (closed Bugs: fixed)

Opened 4 years ago

Last modified 3 years ago

Flushing a filtering_ostream stopped working in Boost 1.44

Reported by: t0rt1e@… Owned by: turkanis
Milestone: To Be Determined Component: iostreams
Version: Boost 1.44.0 Severity: Problem
Keywords: Cc: duncanphilipnorman@…

Description

Since the update to Boost 1.44, flushing an filtering_ostream containing an output_filter and a sink different from std::cout (tested boost::iostreams::back_inserter(std::string) and boost::test_tools::output_test_stream) using the flush() member function stopped working. The output string stays empty. This used to work until Boost 1.43.

I tried to replace the call of flush() with the sync() member function documented in <http://www.boost.org/doc/libs/1_44_0/libs/iostreams/doc/classes/filtering_stream.html>, which leads to compiler error that filtering_ostream does not have this member function. Only using the member function strict_sync() flushes correctly the filtering_ostream in Boost 1.44.

Attached is a test case based on the back_inserter example in <http://www.boost.org/doc/libs/1_44_0/libs/iostreams/doc/tutorial/container_sink.html> augmented by a transparent filter simplified from the code example in <http://www.boost.org/doc/libs/1_44_0/libs/iostreams/doc/concepts/output_filter.html>.

Attachments

boost_iostreams_filtering_ostream.cpp Download (2.5 KB) - added by t0rt1e@… 4 years ago.
Test case
boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp Download (2.3 KB) - added by duncanphilipnorman@… 4 years ago.
Testcase for linux that shows that even cout does not actually get flushed.
boost_4590_correction.diff Download (539 bytes) - added by Mike Tryhorn <miketryhorn@…> 4 years ago.
Correction to concept_adapter.hpp for boost 1.44.0.

Change History

Changed 4 years ago by t0rt1e@…

Test case

comment:1 Changed 4 years ago by tort1e@…

Maybe I should mention the platform, on which I observed this behavior: Mac OS X 10.6 x86_64 with Boost 1.44 installed through MacPorts?.

Changed 4 years ago by duncanphilipnorman@…

Testcase for linux that shows that even cout does not actually get flushed.

comment:2 follow-ups: ↓ 3 ↓ 5 Changed 4 years ago by duncanphilipnorman@…

  • Cc duncanphilipnorman@… added

I've attached boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp Download, which demonstrates (on Linux) that even std::cout does not get flushed by filtering_ostream.

I'm using:

  • GCC 4.3.4
  • Gentoo Linux
  • Boost 1.43.0

I'm not sure if this is the same problem, but  my post to the mailing list got no response.

  1. The basic recipe:
    filtering_ostream out;
    out.push(any_filter_at_all());
    out.push(std::cout);
    out << "Hello World!" << std::flush;
    sleep(10); // Observe an empty console.
    std::cout << std::flush;
    sleep(10); // Observe printed text on console.
    
  2. This problem is quite bad for me when doing the following on the command-line (for a more complicated program):
     ./program 2>&1 | tee some-file.log
    
    Because stdout is not getting flushed, the stdout and stderr get intermingled strangely.
  3. However, disabling buffering on stdout works around the problem:
     stdbuf --output=0 ./program 2>&1 | tee some-file.log
    

comment:3 in reply to: ↑ 2 Changed 4 years ago by anonymous

Replying to duncanphilipnorman@…:

I've attached boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp Download, which demonstrates (on Linux) that even std::cout does not get flushed by filtering_ostream.

By the way, my test program is much less elegant than the original reporter's testcase. One must observe console output (based on timing due to sleep() calls) to see the problem, since I wasn't sure how to test std::cout otherwise.

comment:4 Changed 4 years ago by Mike Tryhorn <miketryhorn@…>

Dear all,

I recently upgraded from Boost 1.43.0 to 1.44.0 and have experienced the same problem. My personal debug logging library relies upon the proper functioning of the flush (and sync) mechanism in boost::iostreams to produce transactional, orderly message output. Unfortunately, in the latest boost version this flush mechanism is non-functional.

However, I've made an investigation of this problem by putting versions 1.43.0 and 1.44.0 side-by-side. I am relieved to say that I have tracked the problem, and have a simple solution.

I have confirmed that this problem was introduced by the bugfix for #2356, which changed the following code in boost/iostreams/detail/adapter/concept_adapter.hpp:

Boost v1.43.0

  bool flush( BOOST_IOSTREAMS_BASIC_STREAMBUF(char_type,
              BOOST_IOSTREAMS_CHAR_TRAITS(char_type))* sb )
  {
      bool result = any_impl::flush(t_, sb);
      if (sb && sb->BOOST_IOSTREAMS_PUBSYNC() == -1)
          result = false;
      return result;
  }

Boost v1.44.0

  template<typename Device>
  bool flush( Device* dev )
  {
      return any_impl::flush(t_, dev);
  }

(See changeset:  https://svn.boost.org/trac/boost/changeset/63034.)

For some reason, the 'PUBSYNC' operation was removed. I suspect that it was merely mistaken as being unnecessary, however it is the mechanism by which the filtering_ostream's chain is flushed in its entirety.

To give some context: indirect_streambuf (the stream buffer used by filtering_ostream), being a std::streambuf, has a required 'sync' method:

boost::iostreams::indirect_detail::indirect_streambuf::sync

  template<typename T, typename Tr, typename Alloc, typename Mode>
  int indirect_streambuf<T, Tr, Alloc, Mode>::sync()
  {
      try { // sync() is no-throw.
          sync_impl();
          obj().flush(next_);
          return 0;
      } catch (...) { return -1; }
  }

Where 'obj()' returns a 'concept_adapter<T>&'. This 'sync' method's implementation has not changed between versions 1.43.0 and 1.44.0. This sync method is called by ostream::flush, and relies upon the expectation that any 'flush' will indeed itself perform a 'sync' operation (the aforementioned 'PUBSYNC').

However, as 'PUBSYNC' has been removed from concept_adaptor<T>::sync, only the first device or filter in the filtering_ostream's chain will ever be flushed. This means that for any filtering_ostream which a chain of length 2 or more, the stream's output will only reach its destination when a buffer overflows, or the stream is released.

So, I hope that by now the solution is self-evident:

Proposal for Boost 1.45.0

  template<typename Device>
  bool flush( Device* dev )
  {
      bool result = any_impl::flush(t_, dev);
      if (dev && dev->BOOST_IOSTREAMS_PUBSYNC() == -1)
          result = false;
      return result;
  }

I've tested this solution with the test code provided both in this bug report (attachment: boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp), and #2356 (attachment: test.cpp).

I'm not myself a boost contributor (this is my first contribution), so if you believe it to be correct, could someone please submit this change to SVN? Correction diff file to follow.

Many thanks.

Changed 4 years ago by Mike Tryhorn <miketryhorn@…>

Correction to concept_adapter.hpp for boost 1.44.0.

comment:5 in reply to: ↑ 2 Changed 4 years ago by Duncan Exon Smith <duncanphilipnorman@…>

Replying to duncanphilipnorman@…:

I've attached boost_iostreams_filtering_ostream_does_not_flush_at_all.cpp Download, which demonstrates (on Linux) that even std::cout does not get flushed by filtering_ostream.

I believe my issue is distinct from the one described by the original poster -- particularly because my issue is present already in Boost 1.43.0. I have created a new ticket for my issue (see #4728), along with a patch to fix it.

comment:6 Changed 3 years ago by steven_watanabe

  • Status changed from new to closed
  • Resolution set to fixed

(In [68779]) Fix flush regression. Fixes #4590.

View

Add a comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
The resolution will be deleted. Next status will be 'reopened'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.