MantisBT - CMake
View Issue Details
0014915CMakeCMakepublic2014-05-13 17:202015-06-01 08:38
mwoehlke 
 
normalfeatureN/A
closedfixed 
 
CMake 3.2CMake 3.2 
0014915: cmake should support ninja's "console" pool
Now that ninja has a "console" pool feature, CMake should be making use of this for certain operations. For example, the install step, and re-run CMake steps should almost certainly use it unconditionally. (Also packaging steps, if it is trivial to identify such.)

It would be helpful to also expose this in a way that users can make selective use of it, e.g. for a custom command. (In particular, I'd like to see it used when building external projects, although this probably needs to be controllable by the user building a project e.g. via a CMake option.)
No tags attached.
Issue History
2014-05-13 17:20mwoehlkeNew Issue
2014-05-13 17:32Ben BoeckelNote Added: 0035873
2014-05-13 17:34Ben BoeckelNote Added: 0035874
2014-05-13 17:44mwoehlkeNote Added: 0035875
2014-10-03 08:48Brad KingNote Added: 0036892
2014-10-03 08:58Brad KingNote Added: 0036893
2014-11-20 07:01raspyNote Added: 0037237
2014-11-20 08:46raspyNote Added: 0037239
2014-11-20 08:46Brad KingNote Added: 0037240
2014-11-20 08:47Brad KingAssigned To => Peter Collingbourne
2014-11-20 08:47Brad KingStatusnew => assigned
2014-11-20 08:47Brad KingTarget Version => CMake 3.2
2014-11-24 03:33JussiPNote Added: 0037259
2015-01-26 10:25Brad KingNote Added: 0037810
2015-01-26 10:25Brad KingAssigned ToPeter Collingbourne =>
2015-01-26 10:25Brad KingStatusassigned => resolved
2015-01-26 10:25Brad KingResolutionopen => fixed
2015-01-26 10:25Brad KingFixed in Version => CMake 3.2
2015-01-27 11:17Brad KingNote Edited: 0037810bug_revision_view_page.php?bugnote_id=37810#r1686
2015-06-01 08:38Robert MaynardNote Added: 0038861
2015-06-01 08:38Robert MaynardStatusresolved => closed

Notes
(0035873)
Ben Boeckel   
2014-05-13 17:32   
There's already a target property for pools (JOB_POOL_COMPILE and JOB_POOL_LINK and corresponding global properties). To set the pool sizes, set JOB_POOLS to a key/value list, e.g. "poola=10;poolb=3".

I don't think the rerun cmake needs to be in the pool since it uses the "generator = 1" rule flag and is already treated specially. Likewise, the "install" target depends on the entire build already, so it probably doesn't need it either.

For custom commands, I don't think there's a way to set the pool; you'd need to attach it to a target and set the target property.
(0035874)
Ben Boeckel   
2014-05-13 17:34   
The only target which should use the console pool unconditionally is the edit_cache target (which probably needs to be added to ninja).
(0035875)
mwoehlke   
2014-05-13 17:44   
Why would you not want to run the re-run CMake step in the console pool? Likewise for install (can this ever meaningfully happen in parallel?)?

The existing support also needs to be usable for external project steps for me to consider this "supported".

The purpose here to using the console pool isn't quite the same as other reasons to use generic pools. It's specifically so that a step that a: takes a long time to run, and b: doesn't make a lot of sense to - or *never is* (e.g. re-running CMake) - run in parallel anyway can output directly to the console without being buffered, as opposed to the current behavior where there is no ability to monitor the task's progress.
(0036892)
Brad King   
2014-10-03 08:48   
Related mailing list thread and patch proposal:

 [PATCH] Ninja: Use 'console' pool for CMake re-run if possible
 http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/11137 [^]
(0036893)
Brad King   
2014-10-03 08:58   
Re 0014915:0036892: Patch applied here:

 Ninja: Use 'console' pool for CMake re-run if possible
 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=9f32a241 [^]

It addresses the case when CMake re-runs.

Future work could use this for the 'install' target too. One could also investigate whether this will allow 'make edit_cache' to run ccmake instead of cmake-gui.
(0037237)
raspy   
2014-11-20 07:01   
I think it is still needed to have a possibility to run a specific custom command in console pool. My use case is like this:

I have a target which requires a (one-time) preparation of a password file. It depends on the file itself, but I also have a custom command to generate this file if needed. If this command is run, it needs to run in console pool so that it gets stdin for user input.

As for now, I can only set pool on the target level, but I don't need whole target run sequentially. If I wrap my custom command in a custom target and start depending on this target, it would be always called (since it's phony) and the file will always be generated, which I don't want. The semantics of a custom command is better here (only run command when output is out of date) but it would definitely make use of specifying a pool.
(0037239)
raspy   
2014-11-20 08:46   
I made a check and I cannot get custom command running in a console pool even with wrapping through a target. Am I missing something?

add_custom_command(
    OUTPUT passfile
    COMMAND important command
)

add_custom_target(generate_passfile
    DEPENDS passfile
)

set_target_properties(generate_passfile PROPERTIES
    JOB_POOL_COMPILE console
    JOB_POOL_LINK console
)

It would also be useful to allow somehow specify to Ninja to emit 'generator = 1' for this command, so that it does not regenerate passfile if command line is missing in ninja's log and the file itself is not removed upon clean.
(0037240)
Brad King   
2014-11-20 08:46   
Support was merged to 'master' for this recently:

 Merge topic 'console-pool'
 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=444f61e0 [^]
(0037259)
JussiP   
2014-11-24 03:33   
In addition to the CMake re-run, the test target should also be in the console pool by default.
(0037810)
Brad King   
2015-01-26 10:25   
(edited on: 2015-01-27 11:17)
A contributed patch should finish this off:

 Configure some CMake-provided targets with USES_TERMINAL
 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=bbffaa0e [^]

(0038861)
Robert Maynard   
2015-06-01 08:38   
Closing resolved issues that have not been updated in more than 4 months.