[CMake] Ninja + parallel linking

[Brad King]

On 07/07/2016 04:59 PM, Robert Dailey wrote:
> I notice on ninja builds generated via CMake 3.6, the link step blocks
> compilation threads. What I expect is that if I have 32 threads
> available for compiling, those are also shared via the link step. In
> other words, if I reach a link step, I want it to consume 1 thread and
> the other 31 threads start compiling translation units for the next
> library.
> 
> Is there a reason it isn't behaving as I describe? Is there a setting
> or CMake option I can set to enable this?

This happens because CMake generates order-only dependencies on the
compilation rules of each exe/lib's objects to make sure the do not
compile until linking of dependencies is done.  This makes the build
semantics match that of other generators where each logical target is
treated as its own isolated build that is evaluated only when its
logical target dependencies have been finished.

This is unfortunately necessary to get correct builds for CMake projects
in general because we support cases where add_custom_command is used
in library "foo" to generate a header file that is included during
compilation in library "bar" that links to "foo", but we have no good
way to express this dependency besides the ordering dependency of bar
on foo.

It is likely possible to detect causes automatically when this is
not necessary, such as when the transitive closure of a target's
dependencies contains no custom commands.  A dedicated effort will be
needed to investigate this further and implement such optimizations
on the build graph generation.

-Brad