[CMake] cmake + ninja how to use several CPU cores?

[Bruce Stephens]

You might find it interesting to look at build.ninja. By the looks of
it there's a (phony) target guving the dependencies of the library.
(That's called cmake_object_order_depends_target_internal_lib) and
CMakeFiles/app.dir/main.cpp.o is depending on that.

Presumably the idea is that there's no point trying to compile
main.cpp before we could build the library, since (as in this case)
that might require generated headers which main.cpp might need.

If I get rid of the wait5 thing, then main.cpp gets compiled (using
"ninja -v CMakeFiles/app.dir/main.cpp.o") without the library. Just
the headers (and lib/lib1.cpp) get generated.

I think it's reasonable for CMake/Ninja to require the headers be
generated, especially since main.cpp does include one of them (though
CMake/Ninja doesn't know that until later). lib/lib1.cpp is more
arguable, but I imagine there's no real distinction made: main.cpp.o
depends on all the dependencies of the libraries.

On Mon, 29 Jul 2019 at 10:34, Alan W. Irwin <Alan.W.Irwin1234 at gmail.com> wrote:
>
> On 2019-07-28 23:39-0700 Alan W. Irwin wrote:
>
> > @Both: I also plan to look at whether this issue exists for
> > the internal library case so more later when I get that
> > corresponding simple test project implemented.
>
> @Brad and David:
>
> I have now implemented a simple test project for the internal library
> case.  See the attached tarball (and please let me know if your input
> mail software chain allows you to receive that tarball).  In general I
> think this test case improves somewhat on Dave's test project by
> making the sleep more explicit at the CMake level, generating the
> headers explicitly rather than including them in the tarball,
> generating all header and source code files and the library (internal
> in this case) within the build tree rather than source tree (which
> considerably simplifies making a clean start), and by allowing the
> user to specify the -DBUILD_SHARED_LIBS=ON (or OFF) cmake option to
> choose whether the library is built shared on static.  (Of course,
> none of these improvements should affect the delayed compilation
> issue.)
>
> For this new simple test project, here are the results for the -G"Ninja" case (when -DBUILD_SHARED_LIBS=ON is specified):
>
> irwin at merlin> ninja -j16 -v app
> [1/10] cd /home/irwin/David.Milter/20190728/test_ninja/add_internal_library/build_dir/lib && sh /home/irwin/David.Milter/20190728/test_ninja/add_internal_library/lib/include_generator.sh
> [2/10] cd /home/irwin/David.Milter/20190728/test_ninja/add_internal_library/build_dir/lib && sh /home/irwin/David.Milter/20190728/test_ninja/add_internal_library/lib/lib_generator.sh
> [3/10] cd /home/irwin/David.Milter/20190728/test_ninja/add_internal_library/build_dir && sleep 5
> [4/10] /usr/bin/c++  -Dinternal_lib_EXPORTS -Ilib -fPIC -MD -MT CMakeFiles/internal_lib.dir/lib/lib1.cpp.o -MF CMakeFiles/internal_lib.dir/lib/lib1.cpp.o.d -o CMakeFiles/internal_lib.dir/lib/lib1.cpp.o -c lib/lib1.cpp
> [5/10] /usr/bin/c++  -Dinternal_lib_EXPORTS -Ilib -fPIC -MD -MT CMakeFiles/internal_lib.dir/lib/lib3.cpp.o -MF CMakeFiles/internal_lib.dir/lib/lib3.cpp.o.d -o CMakeFiles/internal_lib.dir/lib/lib3.cpp.o -c lib/lib3.cpp
> [6/10] /usr/bin/c++  -Dinternal_lib_EXPORTS -Ilib -fPIC -MD -MT CMakeFiles/internal_lib.dir/lib/lib4.cpp.o -MF CMakeFiles/internal_lib.dir/lib/lib4.cpp.o.d -o CMakeFiles/internal_lib.dir/lib/lib4.cpp.o -c lib/lib4.cpp
> [7/10] /usr/bin/c++  -Dinternal_lib_EXPORTS -Ilib -fPIC -MD -MT CMakeFiles/internal_lib.dir/lib/lib2.cpp.o -MF CMakeFiles/internal_lib.dir/lib/lib2.cpp.o.d -o CMakeFiles/internal_lib.dir/lib/lib2.cpp.o -c lib/lib2.cpp
> [8/10] : && /usr/bin/c++ -fPIC    -shared -Wl,-soname,libinternal_lib.so -o libinternal_lib.so CMakeFiles/internal_lib.dir/lib/lib1.cpp.o CMakeFiles/internal_lib.dir/lib/lib2.cpp.o CMakeFiles/internal_lib.dir/lib/lib3.cpp.o CMakeFiles/internal_lib.dir/lib/lib4.cpp.o   && :
> [9/10] /usr/bin/c++   -Ilib  -MD -MT CMakeFiles/app.dir/main.cpp.o -MF CMakeFiles/app.dir/main.cpp.o.d -o CMakeFiles/app.dir/main.cpp.o -c ../main.cpp
> [10/10] : && /usr/bin/c++     CMakeFiles/app.dir/main.cpp.o  -o app  -Wl,-rpath,/home/irwin/David.Milter/20190728/test_ninja/add_internal_library/build_dir libinternal_lib.so && :
>
> Note that the build of main.cpp.o is (unreasonably in my opinion) delayed to step 9.
>
> And that build is similarly delayed in the -G"Unix Makefiles" case.
> That is this newly implemented test case for the internal library
> build shows exactly the same unexplained compilation delays for
> main.cpp for both -G"Ninja" and -G"Unix Makefiles" as David's test
> case for the external library build.
>
> @Brad and other CMake developers who might be lurking here:
>
> Can anybody explain why CMake has this delayed compilation "feature"
> for apps that depend on internal or external libraries, and if there
> is no compelling reason for it would it be straightforward to remove
> it?
>
> Alan
> __________________________
> Alan W. Irwin
>
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.org); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __________________________
>
> Linux-powered Science
> __________________________--
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> https://cmake.org/mailman/listinfo/cmake