[CMake] cmake + ninja how to use several CPU cores?

[Alan W. Irwin]

Hi to Dave and Brad:

@Dave: As you are probably already aware Brad is one of the primary
CMake developers so I think he should be in on this discussion.

@Brad:

Using Dave's simple test project
I confirmed the compilation delay issue below in detail for both the
-G"Ninja" case AND the -G"Unix Makefiles" case on Debian Buster.
Would you please take a look at this simple test case to see if you
can find the explanation for this issue?

<Aside>
After years of ignoring CMake + Ninja because of the Fortran issues
for that combination, I have just realized you have a fork of Ninja
(not yet merged upstream) which apparently works well with CMake for the Fortran
case.  So I am in the process of trying that Ninja fork out for all components of PLplot
(including our Fortran binding and examples), and I plan to report to you those
results in a separate thread.  Which explains my renewed interest in Ninja
(in this case not your fork since no Fortran is involved) for this current thread.
</Aside>

@Both: I also plan to look at whether this issue exists for
the internal library case so more later when I get that
corresponding simple test project implemented.

More details below concerning the external library case ....

On 2019-07-29 03:36+0300 Dave Milter wrote:

> Hi,
>
> On Mon, Jul 29, 2019 at 12:01 AM Alan W. Irwin
> <Alan.W.Irwin1234 at gmail.com> wrote:
>>
>> On 2019-07-28 19:03+0300 Dave Milter wrote:
>>
>> To help answer your question, I see no fundamental issues with your code above.
>>
>> If the latter is really true, that appears to me to be a CMake bug
>> (although I cannot make the final judgement call on that since I am an
>> experienced CMake user rather than a CMake developer).  And if so, does that
>> same problem (unnecessary delay in compilation rather than linking)
>> occur for the following simplified case?
>>
>
> In fact my code in previous email is demo code. It already contains
> "sleep" inside lib_generator.sh to emulate
> real project.

@Dave:

Great minds think alike concerning clearly demonstrating this issue with the
help of the sleep command.  :-)

> I am just not sure is this good idea to send attachment into this mailing list.

Actually, for obvious reasons of convenience, gzipped tarball
attachments should never be an issue on mailing lists (like the CMake
ones) which are relevant to discussion of software.  (There are
typically mailing-list limits with regard to size, but they are
normally quite generous so I have never run into this issue on the
CMake mailing lists.  Therefore, as expected, your attachment got
through to me without issues.  Also, I double-checked your description
of that attachment which is critical for the input mail chains of some
users.  In your case that description (which you can configure with
your mail software) was "application/GZIP" which is obviously fine for
my input mail chain case and presumably most others on this list who
want to look at that attachment.

@Brad: have you had any trouble seeing that attachment without your
input mail chain stripping it out?

> See attachment, it is possible only to build on Linux/Mac, because of
> usage bash/gcc for lib_geneator.sh
>
> And I am pretty sure that problem in cmake (or my usage of cmake),
> because of build.ninja generated by cmake contains such rules:
> ```
> build cmake_object_order_depends_target_app: phony || extern_lib_target
>
> build CMakeFiles/app.dir/main.cpp.o: CXX_COMPILER__app ../main.cpp ||
> cmake_object_order_depends_target_app
> ```
> "||" is "order-only dependencies" according to
> https://ninja-build.org/manual.html#_syntax_example .
> So cmake instruct ninja do not build main.cpp.o untill extern_lib_target build.

@Dave: Thanks very much for that simple test project.

@Both:

I configured and built it using the Debian Buster = Stable system
versions of cmake (version 3.13.4) and ninja (version 1.8.2) using the
following commands

I. The -G"Ninja" case....

# Fresh start:
# IMPORTANT: first cd to the top-level directory of the source tree. Then

# Get rid of the previously generated external library (if any) to
# insure the 5 second pause associated with that generation always
# occurs.
rm -f extern_lib/libextern.so

# Start with an empty build tree

rm -rf build_dir
mkdir build_dir
cd build_dir

# Configure
time cmake -G"Ninja" .. >& cmake.out
==>

real    0m1.084s
user    0m0.845s
sys     0m0.232s

# Just to be specific (although ninja normally figures this out for itself for parallel builds)
# build with -j16 since I have 16-thread hardware.  Also use -v option to create (slightly more)
# verbose output.
time ninja -j16 -v all
==>

[1/3] cd /home/irwin/David.Milter/20190728/test_ninja/add_library_imported/extern_lib && sh ./lib_generator.sh
Generating...
Generating...DONE
[2/3] /usr/bin/c++   -isystem ../extern_lib/gen  -MD -MT CMakeFiles/app.dir/main.cpp.o -MF CMakeFiles/app.dir/main.cpp.o.d -o CMakeFiles/app.dir/main.cpp.o -c ../main.cpp
[3/3] : && /usr/bin/c++     CMakeFiles/app.dir/main.cpp.o  -o app  -Wl,-rpath,/home/irwin/David.Milter/20190728/test_ninja/add_library_imported/extern_lib ../extern_lib/libextern.so && :

real    0m5.354s
user    0m0.281s
sys     0m0.070s

Most of that real time was due to the 5 second pause (generated by
the sleep command) before the "Generating...DONE" message.  And
clearly these results show that main.cpp was generated after that 5
second pause, i.e., the delayed compile issue is completely and unambiguously
confirmed by me and should be equally easy to confirm by Brad.

I also obtained the following result:

irwin at merlin> grep cmake_object_order_depends_target_app build.ninja 
build cmake_object_order_depends_target_app: phony || extern_lib_target
build CMakeFiles/app.dir/main.cpp.o: CXX_COMPILER__app ../main.cpp || cmake_object_order_depends_target_app

which I agree appears to be the reason for the delayed compilation issue in the ninja case.

II. The -G"Unix Makefiles" case....

N.B. It turns out the issue also occurs for the -G"Unix Makefiles"
case as well, i.e.,

# Fresh start:
# IMPORTANT: first cd to the top-level directory of the source tree. Then

# Get rid of the previously generated external library (if any) to
# insure the 5 second pause associated with that generation always
# occurs.
rm -f extern_lib/libextern.so

# Start with an empty build tree
rm -rf build_dir
mkdir build_dir
cd build_dir

# Configure
time cmake -G"Unix Makefiles" .. >& cmake.out
==>

real    0m1.470s
user    0m1.152s
sys     0m0.309s

That's substantially slower (by ~0.4 seconds!) than the Ninja case
which is part of the reason there is so much interest in Ninja these
days compared to (GNU-)make.

# (GNU-make requires a -j option to use parallel builds so make it
# the same -j option as in the Ninja case)
# build with -j16 since I have 16-thread hardware
# Also to keep output simple, do not use VERBOSE=1 option.
time make -j16 all
==>
Scanning dependencies of target extern_lib_target
[ 33%] Generating ../extern_lib/libextern.so
Generating...
Generating...DONE
[ 33%] Built target extern_lib_target
Scanning dependencies of target app
[ 66%] Building CXX object CMakeFiles/app.dir/main.cpp.o
[100%] Linking CXX executable app
[100%] Built target app

real    0m5.482s
user    0m0.397s
sys     0m0.082s

Note, the 5 seconds part of this is due to the pause.  Therefore, the
fact this is ~0.1 seconds slower than the Ninja case is pretty
significant, i.e., this confirms CMake -G"Ninja" provides more nimble
builds than CMake -G"Unix Makefiles" (as expected from the Ninja simplistic
design principles).

More importantly these -G"Unix Makefiles" results confirms that the (compilation delay) issue also
appears for the make case.

I have looked at the various Makefile rules generated by CMake in this
case.  However, they are so complex I am having trouble figuring out
why the compilation of main.cpp is delayed until after the external
library is built.

@Brad: I hope you will be able to satisfy my curiosity about that.

More later for the internal library case.

Alan
__________________________
Alan W. Irwin

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.org); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________