Modify

Ticket #2318 (closed Bugs: fixed)

Opened 6 years ago

Last modified 4 years ago

A particular string for which compressing/uncompressing using iostreams/zlib fails.

Reported by: guy_tannenbaum@… Owned by: turkanis
Milestone: Boost 1.37.0 Component: iostreams
Version: Boost 1.36.0 Severity: Problem
Keywords: Cc:

Description

Found a string that compressing and than uncompressing using iostreams and zlib results in an exception being thrown by the uncompress operation. (The same code works fine for all other strings tried). Attaching a sample program + file containing the problematic string.

Not sure if this helps - but take note that the resulting compressed buffer has the pretty unusual size of 20480, which might be a multiple of some constant. A number of different strings which compress to this size however didn't exhibit the same behavior.

Attachments

strange_zlib.cpp Download (1.8 KB) - added by guy_tannenbaum@… 6 years ago.
program which generates the behavior
short_magic.txt Download (125.1 KB) - added by guy_tannenbaum@… 6 years ago.
problematic string (each line is the ascii code of one char)
working_zlib.cpp Download (2.4 KB) - added by rbock@… 6 years ago.
Variants of strange_zlib.cpp showing how to circumvent the problem (and another variant which does fail)
magic.bin Download (32.6 KB) - added by rbock@… 6 years ago.
Binary version of short_magic.txt (easier to read into the program, I think, unless it is damaged by upload/download, md5sum: fd78968e81890bce3cee1c3054b700e9)

Change History

Changed 6 years ago by guy_tannenbaum@…

program which generates the behavior

Changed 6 years ago by guy_tannenbaum@…

problematic string (each line is the ascii code of one char)

comment:1 Changed 6 years ago by guy_tannenbaum@…

Found this on Boost 1.35.0 (not 1.36.0)

comment:2 Changed 6 years ago by rbock@…

Hi,

I found the same problem in 1.36. It is rather rare: In 6.5Million compressed files in my project, it occurred only 441 times. As mentioned by Guy, the size of the damaged zip is a bit peculiar, all sizes I checked are multiples of 4096.

Also, I think it is more related to filtering_istream and filtering_istreambuf than zlib_compression, as it works fine if you use filtering_ostream or filtering_ostreambuf instead.

I am going to attach an extended example with similar code which does not work, too for the given example string and code which does (using filtering_ostream instead of filtering_istream for compression).

I am also attaching a binary version of the sample data, since reading the ascii version did not work for me (I had to replace the lexical_cast by atoi to make it work).

Here is some system information, if this should be required to reproduce the problem (and circumvention):

Ubuntu 8.04 64bit, gcc 4.2.3, boost-1.36, zlib-1g 1.2.3.3

Please do not hesitate to ask for more information. Personally, I consider this a rather severe problem since it can impair data integrity in backups. Lucky me, my software is not in production yet :-)

Regards,

Roland

Changed 6 years ago by rbock@…

Variants of strange_zlib.cpp showing how to circumvent the problem (and another variant which does fail)

Changed 6 years ago by rbock@…

Binary version of short_magic.txt (easier to read into the program, I think, unless it is damaged by upload/download, md5sum: fd78968e81890bce3cee1c3054b700e9)

comment:3 Changed 6 years ago by rbock@…

See also #2411, which shows that the same (or at least very similar) problem occurs for bzip2 compression, too (and much easier to reproduce).

comment:4 Changed 4 years ago by steven_watanabe

The compressed data is invalid because the stream is not properly ended. Still investigating.

comment:5 Changed 4 years ago by steven_watanabe

  • Status changed from new to closed
  • Resolution set to fixed

(In [62952]) Don't end the stream produced by a reading through a symmetric_filter prematurely. Fixes #2318.

View

Add a comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
The resolution will be deleted. Next status will be 'reopened'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.