Modify

Opened 9 years ago

Last modified 5 months ago

#2557 assigned Bugs

iostreams filtering_stream w/ gzip infinite loop when writing to a full drive

Reported by: Tomasz Śniatowski <kailoran@…> Owned by: Jonathan Turkanis
Milestone: Boost 1.38.0 Component: iostreams
Version: Boost 1.55.0 Severity: Problem
Keywords: Cc:

Description

When a filtering_stream with a gzip_compressor is used to write to a full hard drive (i.e. insufficient free space), boost enters an infinite loop in /boost/iostreams/detail/adapter/non_blocking_adapter.hpp:41 because the write function keeps returning zero. This loop happens during the destruction of the stream. I can't seem to find a client-side workaround.

Attached is a test case, point it to a volume with zero space and give some large number of bytes. If there's insufficient space, execution hangs. Tested on mingw/winxp/gcc4.2 but seems to fail on linux/gcc as well.

Attachments (1)

boost-bug.cpp (904 bytes) - added by Tomasz Śniatowski <kailoran@…> 9 years ago.

Download all attachments as: .zip

Change History (11)

Changed 9 years ago by Tomasz Śniatowski <kailoran@…>

Attachment: boost-bug.cpp added

comment:1 Changed 9 years ago by Jonathan Turkanis

Status: newassigned

comment:2 Changed 7 years ago by michal-slizak@…

Has this been fixed? If so, which version of the library contains the fix?

comment:3 Changed 7 years ago by anonymous

No, it hasn't been fixed.

comment:4 Changed 7 years ago by michal-slizak@…

The gzip file consists of a sequence of compressed chunks.

You can manually compress data chunks and append them to a file:

class StreamSink
{
public:
	typedef char char_type;
	typedef boost::iostreams::sink_tag category;
	StreamSink();
	StreamSink(std::ostream * stream);
	std::streamsize write(const char * ptr, std::streamsize n);
	void close();
	bool isOpen() const;
private:
	bool opened;
	std::ostream * stream;
};

StreamSink::StreamSink():
	opened(false),
	stream(NULL)
{
}

StreamSink::StreamSink(std::ostream * stream):
	opened(true),
	stream(stream)
{
	assert(stream != NULL, "StreamSink constructed from NULL");
}
	
std::streamsize StreamSink::write(const char * ptr, std::streamsize n)
{
	assert(this->stream != NULL, "Writing to NULL stream");
	this->stream->write(ptr, n);
	return n;
}
	
void StreamSink::close()
{
	this->opened = false;
	this->stream = NULL;
}
	
bool StreamSink::isOpen() const
{
	return this->opened;
}

void writeGzippedSection(string const & filename, ios_base::openmode mode, string const & data)
{
	try
	{
		std::ofstream out(filename.c_str(), mode | ios::binary);
		out.exceptions(std::ios_base::badbit | std::ios_base::failbit);
		StreamSink sink(&out);
		boost::iostreams::gzip_compressor compressor;
		compressor.write(sink, data.c_str(), data.length());
		compressor.close(sink, BOOST_IOS::out);
	}
	catch (boost::iostreams::gzip_error const & e)
	{
		std::cerr << "gzip error: " << e.error() << ", zlib error: " << e.zlib_error_code() << std::endl;
		throw;
	}
	catch (ofstream::failure const & e)
	{
		std::cerr << "ofstream::failure: " << e.what() << std::endl;
		throw;
	}
}

The code above produces decompressable files (tested with zcat, zless and gunzip). Also, when out-of-space occurs, chunks that were written successfully can be recovered.

The code may not be pretty, but it works.

comment:5 Changed 4 years ago by leyaya_1999@…

The same infinite loop happens if user does not have write access on the file that is being written to. In that case the variable amt receives value -1 in

/include/boost/iostreams/detail/adapter/non_blocking_adapter.hpp line 42

this bug is still here in boost 1.55

comment:6 Changed 4 years ago by leyaya_1999@…

Version: Boost 1.37.0Boost 1.55.0

comment:7 Changed 4 years ago by anonymous

And the same issue happens on Windows if the "filename" is longer then MAX_PATH and not prefixed with "\\?\".

comment:8 Changed 8 months ago by anonymous

There are also hangs if a file_sink could not be opened (though that is at least possible to check manually by calling is_open() before). Also, the "hangs" are not just hangs but infinite loops with 100% CPU usage. It feels like the whole iostream was written without any consideration for error handling at all.

comment:9 Changed 6 months ago by anonymous

Submitted a pull request with fix and test-case: https://github.com/boostorg/iostreams/pull/36

It doesn't check or ensure that the error is properly propagated (I am not sure there even is a proper way to do that), but at least no more infinite loop.

comment:10 Changed 5 months ago by anonymous

Change has been merged, so hopefully the next release will finally have the fix.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as assigned The owner will remain Jonathan Turkanis.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.