View Issue Details Jump to Notes ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0014963CMakeCMakepublic2014-06-09 09:172015-11-02 09:13
ReporterBrad King 
Assigned ToBrad King 
PrioritynormalSeverityminorReproducibilityhave not tried
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionCMake 3.2Fixed in VersionCMake 3.2 
Summary0014963: Add explicit specification of custom command side effect outputs
DescriptionA common idiom in CMake-based build systems is to use order-only dependencies on high-level targets to implicitly ensure their side effects are up to date, and then to have other build rules depend on those side effects explicitly.

This works well in the VS, Xcode, and Makefile generators because the underlying build systems all evaluate rules at the granularity of the high-level targets so dependencies are not stat-ed from the filesystem until after the order-only dependencies are satisified. It does not work well in the Ninja generator because ninja does not re-stat dependencies of a rule after its order-only dependencies have been satisfied:

 https://github.com/martine/ninja/issues/760 [^]

This behavior is not necessarily wrong because nothing in the generated ninja build system tells ninja what build rules generate the side effects.

With a bit more information from the CMakeLists.txt code we can generate a ninja build system that works correctly for such cases.
TagsNo tags attached.
Attached Files

 Relationships
related to 0014771closedKitware Robot Ninja generator does not always honor SYMBOLIC rules with no command 
related to 0014972closedBrad King Ninja gen generates invalid phony dependencies for source files with in-source build 
related to 0015256closedBen Boeckel Ninja handling of CMake depends and output files is not correct 
related to 0014747closed Target dependencies do not trigget build on Ninja 
related to 0013574closed Ninja: no transitive dependency for imported libraries 
related to 0013538closed Ninja: Generated 'build.ninja' seems to be incomplete and fails to build ExternalProjects. 
related to 0015454closedBrad King Ninja generator should not generate phony commands with cyclic dependency 

  Notes
(0036129)
Brad King (manager)
2014-06-09 09:19

Here is one example of the case in the description:

add_custom_target(Provider COMMAND some-generator -o side-effect)
add_custom_command(
  OUTPUT explicit-output
  COMMAND cp side-effect explicit-output
  DEPENDS side-effect
  )
add_custom_target(Consumer DEPENDS explicit-output)
add_dependencies(Consumer Provider) # ensure side-effect is up to date


Nothing here tells CMake that Provider also produces "side-effect" as an output, but the author of the code clearly knows this.
(0036130)
Brad King (manager)
2014-06-09 09:20

Here is another example of the case in the description:

include(ExternalProject)
ExternalProject_Add(OtherProj ...)
add_library(OtherProj::OtherLib STATIC IMPORTED)
set_target_properties(OtherProj::OtherLib PROPERTIES ...)
add_dependencies(OtherProj::OtherLib OtherProj) # ensure OtherProj is up to date
add_executable(MyExe ...)
target_link_libraries(MyExe OtherProj::OtherLib) # gets dependency on OtherProj through imported target



The add_dependencies behavior for imported targets is intended exactly for the case that a custom target produces an imported target's file as a side-effect. This basically provides the information we need but not as clearly as it could.
(0036131)
Brad King (manager)
2014-06-09 09:27

The example from ninja issue 760 is:

rule R
    command = $COMMAND
build always: R
    COMMAND = echo 1 >> out
build out: phony out
build out-copy: R out || always
    COMMAND = cp out out-copy
build all: phony out-copy always
default all


The problem is that while evaluating out-copy the dependency on "out" is not re-stated after the order-only dependency on "always" is satisfied. Instead we should write:

rule R
    command = $COMMAND
    restat = 1
build always out: R
    COMMAND = echo 1 >> out
build out-copy: R out || always
    COMMAND = cp out out-copy
build all: phony out-copy always
default all


Note that we explicitly list "out" as an output of the rule for "always". By telling ninja to restat after evaluating the rule then if "out" were to not change then its dependents would not necessarily rebuild. This serves the same purpose as the explicit dependency on an implicit (side-effect) output used in 0014963:0036129 and 0014963:0036130. We just need a way for CMake to know enough information to generate this.
(0037005)
Brad King (manager)
2014-10-08 09:40

A relevant mailing list thread:

 Severe regression caused by 0014972 fixes
 http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/11207/focus=11221 [^]
(0037230)
Brad King (manager)
2014-11-17 11:37

I've drafted a solution here:

 Add an option for explicit BYPRODUCTS of custom commands
 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=e15a7075 [^]

 ExternalProject: Add options to specify BYPRODUCTS
 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=557aef0b [^]
(0037231)
Brad King (manager)
2014-11-17 11:49

A mailing list thread pointing at the solution in 0014963:0037230 for review:

 Explicit custom command BYPRODUCTS
 http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/11648 [^]
(0037339)
Brad King (manager)
2014-12-01 14:32

The changes linked in 0014963:0037230 are now in 'master'.
(0039734)
Robert Maynard (manager)
2015-11-02 09:13

Closing resolved issues that have not been updated in more than 4 months.

 Issue History
Date Modified Username Field Change
2014-06-09 09:17 Brad King New Issue
2014-06-09 09:19 Brad King Note Added: 0036129
2014-06-09 09:20 Brad King Note Added: 0036130
2014-06-09 09:27 Brad King Note Added: 0036131
2014-06-11 08:35 Brad King Relationship added related to 0014771
2014-06-11 08:35 Brad King Relationship added related to 0014747
2014-06-11 08:36 Brad King Relationship added related to 0013574
2014-06-11 08:36 Brad King Relationship added related to 0013538
2014-06-11 08:36 Brad King Status new => backlog
2014-06-13 08:51 Brad King Relationship added related to 0014972
2014-10-08 09:40 Brad King Note Added: 0037005
2014-11-17 11:37 Brad King Note Added: 0037230
2014-11-17 11:37 Brad King Assigned To => Brad King
2014-11-17 11:37 Brad King Status backlog => assigned
2014-11-17 11:37 Brad King Target Version => CMake 3.2
2014-11-17 11:49 Brad King Note Added: 0037231
2014-11-17 11:52 Brad King Category (No Category) => CMake
2014-11-19 10:23 Brad King Relationship added related to 0015256
2014-12-01 14:32 Brad King Note Added: 0037339
2014-12-01 14:33 Brad King Status assigned => resolved
2014-12-01 14:33 Brad King Resolution open => fixed
2014-12-01 14:33 Brad King Fixed in Version => CMake 3.2
2015-03-18 08:37 Brad King Relationship added related to 0015454
2015-11-02 09:13 Robert Maynard Note Added: 0039734
2015-11-02 09:13 Robert Maynard Status resolved => closed


Copyright © 2000 - 2018 MantisBT Team