Modify

Opened 9 years ago

Closed 7 years ago

#2318 closed Bugs (fixed)

A particular string for which compressing/uncompressing using iostreams/zlib fails.

Reported by: guy_tannenbaum@… Owned by: Jonathan Turkanis
Milestone: Boost 1.37.0 Component: iostreams
Version: Boost 1.36.0 Severity: Problem
Keywords: Cc:

Description

Found a string that compressing and than uncompressing using iostreams and zlib results in an exception being thrown by the uncompress operation. (The same code works fine for all other strings tried). Attaching a sample program + file containing the problematic string.

Not sure if this helps - but take note that the resulting compressed buffer has the pretty unusual size of 20480, which might be a multiple of some constant. A number of different strings which compress to this size however didn't exhibit the same behavior.

Attachments (4)

strange_zlib.cpp (1.8 KB) - added by guy_tannenbaum@… 9 years ago.
program which generates the behavior
short_magic.txt (125.1 KB) - added by guy_tannenbaum@… 9 years ago.
problematic string (each line is the ascii code of one char)
working_zlib.cpp (2.4 KB) - added by rbock@… 9 years ago.
Variants of strange_zlib.cpp showing how to circumvent the problem (and another variant which does fail)
magic.bin (32.6 KB) - added by rbock@… 9 years ago.
Binary version of short_magic.txt (easier to read into the program, I think, unless it is damaged by upload/download, md5sum: fd78968e81890bce3cee1c3054b700e9)

Download all attachments as: .zip

Change History (9)

Changed 9 years ago by guy_tannenbaum@…

Attachment: strange_zlib.cpp added

program which generates the behavior

Changed 9 years ago by guy_tannenbaum@…

Attachment: short_magic.txt added

problematic string (each line is the ascii code of one char)

comment:1 Changed 9 years ago by guy_tannenbaum@…

Found this on Boost 1.35.0 (not 1.36.0)

comment:2 Changed 9 years ago by rbock@…

Hi,

I found the same problem in 1.36. It is rather rare: In 6.5Million compressed files in my project, it occurred only 441 times. As mentioned by Guy, the size of the damaged zip is a bit peculiar, all sizes I checked are multiples of 4096.

Also, I think it is more related to filtering_istream and filtering_istreambuf than zlib_compression, as it works fine if you use filtering_ostream or filtering_ostreambuf instead.

I am going to attach an extended example with similar code which does not work, too for the given example string and code which does (using filtering_ostream instead of filtering_istream for compression).

I am also attaching a binary version of the sample data, since reading the ascii version did not work for me (I had to replace the lexical_cast by atoi to make it work).

Here is some system information, if this should be required to reproduce the problem (and circumvention):

Ubuntu 8.04 64bit, gcc 4.2.3, boost-1.36, zlib-1g 1.2.3.3

Please do not hesitate to ask for more information. Personally, I consider this a rather severe problem since it can impair data integrity in backups. Lucky me, my software is not in production yet :-)

Regards,

Roland

Changed 9 years ago by rbock@…

Attachment: working_zlib.cpp added

Variants of strange_zlib.cpp showing how to circumvent the problem (and another variant which does fail)

Changed 9 years ago by rbock@…

Attachment: magic.bin added

Binary version of short_magic.txt (easier to read into the program, I think, unless it is damaged by upload/download, md5sum: fd78968e81890bce3cee1c3054b700e9)

comment:3 Changed 9 years ago by rbock@…

See also #2411, which shows that the same (or at least very similar) problem occurs for bzip2 compression, too (and much easier to reproduce).

comment:4 Changed 7 years ago by Steven Watanabe

The compressed data is invalid because the stream is not properly ended. Still investigating.

comment:5 Changed 7 years ago by Steven Watanabe

Resolution: fixed
Status: newclosed

(In [62952]) Don't end the stream produced by a reading through a symmetric_filter prematurely. Fixes #2318.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Jonathan Turkanis.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.