MantisBT - CMake
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0014963CMakeCMakepublic2014-06-09 09:172015-11-02 09:13
ReporterBrad King 
Assigned ToBrad King 
PrioritynormalSeverityminorReproducibilityhave not tried
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionCMake 3.2Fixed in VersionCMake 3.2 
Summary0014963: Add explicit specification of custom command side effect outputs
DescriptionA common idiom in CMake-based build systems is to use order-only dependencies on high-level targets to implicitly ensure their side effects are up to date, and then to have other build rules depend on those side effects explicitly.

This works well in the VS, Xcode, and Makefile generators because the underlying build systems all evaluate rules at the granularity of the high-level targets so dependencies are not stat-ed from the filesystem until after the order-only dependencies are satisified. It does not work well in the Ninja generator because ninja does not re-stat dependencies of a rule after its order-only dependencies have been satisfied:

 https://github.com/martine/ninja/issues/760 [^]

This behavior is not necessarily wrong because nothing in the generated ninja build system tells ninja what build rules generate the side effects.

With a bit more information from the CMakeLists.txt code we can generate a ninja build system that works correctly for such cases.
Steps To Reproduce
Additional Information
TagsNo tags attached.
Relationships
related to 0014771closed Kitware Robot Ninja generator does not always honor SYMBOLIC rules with no command 
related to 0014972closed Brad King Ninja gen generates invalid phony dependencies for source files with in-source build 
related to 0015256closed Ben Boeckel Ninja handling of CMake depends and output files is not correct 
related to 0014747closed  Target dependencies do not trigget build on Ninja 
related to 0013574closed  Ninja: no transitive dependency for imported libraries 
related to 0013538closed  Ninja: Generated 'build.ninja' seems to be incomplete and fails to build ExternalProjects. 
related to 0015454closed Brad King Ninja generator should not generate phony commands with cyclic dependency 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2014-06-09 09:17Brad KingNew Issue
2014-06-09 09:19Brad KingNote Added: 0036129
2014-06-09 09:20Brad KingNote Added: 0036130
2014-06-09 09:27Brad KingNote Added: 0036131
2014-06-11 08:35Brad KingRelationship addedrelated to 0014771
2014-06-11 08:35Brad KingRelationship addedrelated to 0014747
2014-06-11 08:36Brad KingRelationship addedrelated to 0013574
2014-06-11 08:36Brad KingRelationship addedrelated to 0013538
2014-06-11 08:36Brad KingStatusnew => backlog
2014-06-13 08:51Brad KingRelationship addedrelated to 0014972
2014-10-08 09:40Brad KingNote Added: 0037005
2014-11-17 11:37Brad KingNote Added: 0037230
2014-11-17 11:37Brad KingAssigned To => Brad King
2014-11-17 11:37Brad KingStatusbacklog => assigned
2014-11-17 11:37Brad KingTarget Version => CMake 3.2
2014-11-17 11:49Brad KingNote Added: 0037231
2014-11-17 11:52Brad KingCategory(No Category) => CMake
2014-11-19 10:23Brad KingRelationship addedrelated to 0015256
2014-12-01 14:32Brad KingNote Added: 0037339
2014-12-01 14:33Brad KingStatusassigned => resolved
2014-12-01 14:33Brad KingResolutionopen => fixed
2014-12-01 14:33Brad KingFixed in Version => CMake 3.2
2015-03-18 08:37Brad KingRelationship addedrelated to 0015454
2015-11-02 09:13Robert MaynardNote Added: 0039734
2015-11-02 09:13Robert MaynardStatusresolved => closed

Notes
(0036129)
Brad King   
2014-06-09 09:19   
Here is one example of the case in the description:

add_custom_target(Provider COMMAND some-generator -o side-effect)
add_custom_command(
  OUTPUT explicit-output
  COMMAND cp side-effect explicit-output
  DEPENDS side-effect
  )
add_custom_target(Consumer DEPENDS explicit-output)
add_dependencies(Consumer Provider) # ensure side-effect is up to date


Nothing here tells CMake that Provider also produces "side-effect" as an output, but the author of the code clearly knows this.
(0036130)
Brad King   
2014-06-09 09:20   
Here is another example of the case in the description:

include(ExternalProject)
ExternalProject_Add(OtherProj ...)
add_library(OtherProj::OtherLib STATIC IMPORTED)
set_target_properties(OtherProj::OtherLib PROPERTIES ...)
add_dependencies(OtherProj::OtherLib OtherProj) # ensure OtherProj is up to date
add_executable(MyExe ...)
target_link_libraries(MyExe OtherProj::OtherLib) # gets dependency on OtherProj through imported target


The add_dependencies behavior for imported targets is intended exactly for the case that a custom target produces an imported target's file as a side-effect. This basically provides the information we need but not as clearly as it could.
(0036131)
Brad King   
2014-06-09 09:27   
The example from ninja issue 760 is:

rule R
    command = $COMMAND
build always: R
    COMMAND = echo 1 >> out
build out: phony out
build out-copy: R out || always
    COMMAND = cp out out-copy
build all: phony out-copy always
default all


The problem is that while evaluating out-copy the dependency on "out" is not re-stated after the order-only dependency on "always" is satisfied. Instead we should write:

rule R
    command = $COMMAND
    restat = 1
build always out: R
    COMMAND = echo 1 >> out
build out-copy: R out || always
    COMMAND = cp out out-copy
build all: phony out-copy always
default all


Note that we explicitly list "out" as an output of the rule for "always". By telling ninja to restat after evaluating the rule then if "out" were to not change then its dependents would not necessarily rebuild. This serves the same purpose as the explicit dependency on an implicit (side-effect) output used in 0014963:0036129 and 0014963:0036130. We just need a way for CMake to know enough information to generate this.
(0037005)
Brad King   
2014-10-08 09:40   
A relevant mailing list thread:

 Severe regression caused by 0014972 fixes
 http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/11207/focus=11221 [^]
(0037230)
Brad King   
2014-11-17 11:37   
I've drafted a solution here:

 Add an option for explicit BYPRODUCTS of custom commands
 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=e15a7075 [^]

 ExternalProject: Add options to specify BYPRODUCTS
 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=557aef0b [^]
(0037231)
Brad King   
2014-11-17 11:49   
A mailing list thread pointing at the solution in 0014963:0037230 for review:

 Explicit custom command BYPRODUCTS
 http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/11648 [^]
(0037339)
Brad King   
2014-12-01 14:32   
The changes linked in 0014963:0037230 are now in 'master'.
(0039734)
Robert Maynard   
2015-11-02 09:13   
Closing resolved issues that have not been updated in more than 4 months.