Opened 10 years ago

Last modified 9 years ago

#2068 new Bugs

Better path comparison for common.mkdir

Reported by: Eric Niebler Owned by: Vladimir Prus
Milestone: Component: build
Version: Boost 1.35.0 Severity: Problem
Keywords: Cc:


The problem is exposed by the accumulators doc build, which fails because bjam tries to create the same target directory twice, once with a relative path and once with an absolute path. bjam doesn't normalize the paths correctly and therefore fails to recognize that the targets are the same. The build fails on the second mkdir attempt because the target directory already exists.

Jurko Gospodnetić tracked the problem down to the following:

Ok, I managed to reproduce an equivalent problem using regular Windows bjam. To reproduce it use the following jamroot.jam file placed in a folder called UUU:

import common ;
install fff                      : jamroot.jam ; <name>file.1 ;
install fff/ggg/..               : jamroot.jam : <name>file.2 ;
install fff/ggg/../ggg/../../fff : jamroot.jam : <name>file.3 ;
install ../UUU/fff               : jamroot.jam : <name>file.4 ;

Boost Build tries to battle this problem by normalizing any paths passed to it (i.e. shortcircuiting any . path elements except and any .. path elements except for the initial series), which makes cases 2 & 3 above not conflict with case 1. However the path passed in case 4. is already normalized and does not get recognized as pointing to the same location as the one in case 1. Therefore the initial build fails, while any repeated builds work as expected.

The problem could be solved by updating the bindtarget() function in Boost Jam's rules.c source file to find some sort of a 'unique' identifier for each of its targets. Any suggestions on what such an identifier might be?

I guess we could attempt to root any non absolute paths from the current folder's absolute path and then normalize that. But is that enough?

Should we be prepared to handle stuff like symbolic links?

I have not researched this much before, but is there any 'standard solution' to such a problem that I am not aware of?

Is there some portable 'file system object identifier' we can use in case the file system already exists? There are some other places in Boost Jam code (e.g. file_query() function in filent.c) that could benefit from such an identifier since they hash their results based on the parameter name and so may cache multiple result sets for the same file in some cases. If there is no portable identifier, then are there at least different identifiers on different systems?

Note that this is not just a MkDir? problem. Any other target build could fail due to similar reasons.

Attachments (1)

common.jam.patch (1.2 KB) - added by Jurko Gospodnetic 10 years ago.
Suggested patch for working around problems detected with the Boost Accumulators library documentation build.

Download all attachments as: .zip

Change History (7)

Changed 10 years ago by Jurko Gospodnetic

Attachment: common.jam.patch added

Suggested patch for working around problems detected with the Boost Accumulators library documentation build.

comment:1 Changed 10 years ago by Jurko Gospodnetic

Suggested common.jam patch, if accepted will lower the severity of this item. It still needs to be dealt with but it will no longer be a showstopped for a Boost library 1.36.0 release.

comment:2 Changed 10 years ago by Jurko Gospodnetic

Status: newassigned

comment:3 Changed 10 years ago by Jurko Gospodnetic

Milestone: Boost 1.36.0Boost 1.37.0
Severity: ShowstopperProblem
Summary: [jam] path normalization is brokenSuggested quick-fix patch applied applied to the trunk.

Suggested quick-fix patch committed to the Boost trunk.

Changeset link:

This makes the problem no longer a showstopper for the Boost 1.36 release.

comment:4 Changed 10 years ago by Vladimir Prus

Summary: Suggested quick-fix patch applied applied to the trunk.Better path comparison for common.mkdir

comment:5 Changed 9 years ago by Vladimir Prus

Owner: changed from Jurko Gospodnetic to Vladimir Prus
Status: assignednew

comment:6 Changed 9 years ago by Vladimir Prus

Component: bjambuild
Milestone: Boost 1.37.0
Note: See TracTickets for help on using tickets.