[CMake] Parallel build setup question

Sun Feb 18 11:36:33 EST 2018

Just to close this thread: Also the ninja tipp turned out to be a very
good one! I did the same full built (clear - rebuild) with ninja
instead of make -j4 and times were more or less comparable: from
initially 23 minutes down to 13-14 with make, and 12-13 with ninja.
However, what is much more helpful in practical programming work is the
fact that if I change only one or two files, ninja "knows" this
instantaneously, i.e. no perceivable overhead to find out about
outdated files, while make still takes maybe half a minute or such for
checking.

And since this is a very common case during development I decided to
stay with the QtCreator - CMake - ninja "team of tools"!

Regards, Cornelis

Am Samstag, den 17.02.2018, 08:06 +0100 schrieb Cornelis Bockemühl:
> Thanks also for this hint: I will give it a try!
> 
> Regards, Cornelis
> 
> Am Freitag, den 16.02.2018, 21:35 +0300 schrieb Konstantin Tokarev:
> > > > 16.02.2018, 21:32, "Cornelis Bockemühl" <cornelis at bockemuehl.ch>:
> > > Thanks - that did the trick!
> > > 
> > > > > > > > > > > > And indeed by entering -j4 in the "Tool arguments" field, the
build command changes "magically" into "cmake --build . --target
all -- -j4" - with the "--" to pass the argument further to
gmake.
> > > 
> > > > > > > > > > > > > > > > > > Actually compile time of my project after a "clean" went down
from 23 to 14 min, thus reduction factor 1.6 with 4 processors: I
tend to attribute the diffference to "almost 4" to the nature of
my project where gmake seems not to see too many opportunities
for parallel processing. Maybe it works always module by module,
or whatever.
> > > 
> > > Anyway, my question is answered - thanks again!
> > 
> > > > Note that if you used Ninja generator, it would use all cores by
default.
> > 
> > > 
> > > Regards,
> > > Cornelis
> > > 
> > > > > > Am Freitag, den 16.02.2018, 20:25 +0300 schrieb Konstantin
Tokarev:
> > > > > > > > 16.02.2018, 20:20, "Cornelis Bockemühl" <cornelis at bockemuehl.ch
>:
> > > > > > > > > > > > > > > Thanks for your hints! And you are right: it is still
interesting to know how QtCreator is actually invoking and
using cmake.
> > > > 
> > > > > > > > You should go to project settings and change cmake --build
arguments there
> > > > 
> > > > > > > > > > > > > > > > > > > > > > > > > This is the beginning of a log that the QtCreator was
generating during a build. Actually with two modules where
nothing had to be done, but still showing how cmake and gmake
are being invoked - and all a bit above my understanding of
the two tools:
> > > > > 
> > > > > ===================================
> > > > > > > > > > > > > > > /usr/bin/cmake -H/Sources/GitLinux/RMT
-B/BinariesLinux/RMT/Debug --check-build-system
CMakeFiles/Makefile.cmake 0
> > > > > > > > > > > > > > > /usr/bin/cmake -E cmake_progress_start
/BinariesLinux/RMT/Debug/CMakeFiles
/BinariesLinux/RMT/Debug/CMakeFiles/progress.marks
> > > > > /usr/bin/gmake -f CMakeFiles/Makefile2 all
> > > > > gmake[1]: Entering directory '/BinariesLinux/RMT/Debug'
> > > > > > > > > > /usr/bin/gmake -f CMakeFiles/FirstModule.dir/build.make
CMakeFiles/FirstModule.dir/depend
> > > > > gmake[2]: Entering directory '/BinariesLinux/RMT/Debug'
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > cd /BinariesLinux/RMT/Debug && /usr/bin/cmake -E
cmake_depends "Unix Makefiles" /Sources/GitLinux/RMT
/Sources/GitLinux/RMT /BinariesLinux/RMT/Debug
/BinariesLinux/RMT/Debug
/BinariesLinux/RMT/Debug/CMakeFiles/FirstModule.dir/DependInf
o.cmake --color=
> > > > > gmake[2]: Leaving directory '/BinariesLinux/RMT/Debug'
> > > > > > > > > > /usr/bin/gmake -f CMakeFiles/FirstModule.dir/build.make
CMakeFiles/FirstModule.dir/build
> > > > > gmake[2]: Entering directory '/BinariesLinux/RMT/Debug'
> > > > > > > > > > gmake[2]: Nothing to be done for
'CMakeFiles/FirstModule.dir/build'.
> > > > > gmake[2]: Leaving directory '/BinariesLinux/RMT/Debug'
> > > > > [  3%] Built target FirstModule
> > > > > > > > > > /usr/bin/gmake -f CMakeFiles/SecondModule.dir/build.make
CMakeFiles/SecondModule.dir/depend
> > > > > gmake[2]: Entering directory '/BinariesLinux/RMT/Debug'
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > cd /BinariesLinux/RMT/Debug && /usr/bin/cmake -E
cmake_depends "Unix Makefiles" /Sources/GitLinux/RMT
/Sources/GitLinux/RMT /BinariesLinux/RMT/Debug
/BinariesLinux/RMT/Debug
/BinariesLinux/RMT/Debug/CMakeFiles/SecondModule.dir/DependIn
fo.cmake --color=
> > > > > gmake[2]: Leaving directory '/BinariesLinux/RMT/Debug'
> > > > > > > > > > /usr/bin/gmake -f CMakeFiles/SecondModule.dir/build.make
CMakeFiles/SecondModule.dir/build
> > > > > gmake[2]: Entering directory '/BinariesLinux/RMT/Debug'
> > > > > > > > > > gmake[2]: Nothing to be done for
'CMakeFiles/SecondModule.dir/build'.
> > > > > gmake[2]: Leaving directory '/BinariesLinux/RMT/Debug'
> > > > > [  6%] Built target SecondModule
> > > > > ...
> > > > > ===========================================
> > > > > 
> > > > > > > > > > > > > > > For example I do not really understand all the parameters
that are passed to cmake - and I do not find them in the
documentation either (like -H, -B or --check-build-system...
> > > > > 
> > > > > Regards,
> > > > > Cornelis
> > > > > 
> > > > > > > > > > Am Freitag, den 16.02.2018, 08:59 -0800 schrieb Michael
Ellery:
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don’t know anything about QTCreator, but the question
really boils down to how to get your IDE to pass extra args
to the build tool - it’s not really a cmake issue except to
the extent that CMake can be used to invoke the build step
(after it has been used to generate the build). It sounds
like QT uses “cmake --build” to do the build phase (which
makes sense), so you just need to add your “-j N” argument
to the build step invocation. If you believe this: https://
doc.qt.io/qtcreator/creator-build-settings.html, then it
sounds like maybe “Tool Arguments” is what you want (see
“CMake Build Steps”) ? BTW, the way I do this myself (since
I don’t use an IDE) is something like “cmake --build . --
-j 8” where the double dash stops shell argument processing
and effectively passes everything else on to the build tool
that cmake ultimately invokes (e.g. make or ninja or
msbuild, etc.) HTH, Mike On Feb 16, 2018, at 8:31 AM,
Cornelis Bockemühl <cornelis at bockemuehl.ch> wrote: Hello,
Somehow I seem to miss some crucial point regarding setup
for parallel build with CMake, so I would be happy if
somebody can push me the last few millimeters to hit my
target! My configuration is on OpenSuse Linux (Leap - 64-
bit), working with QtCreator and CMake, using the "make"
configuration. I learned from "the internet" that actually
I would have to pass a -jN option to make or gmake (with N
being the max number of processors I want to use). Or even
more elegant: go for a -lN option to somehow balance the
load. Sounds good! But what I do not see is how to pass
these nice options to make through CMake and QtCreator! One
attempt was to specify CMAKE_CXX_FLAGS=-j4 or the like
because some comment "in the internet" seemed to suggest
this. However, like I already feared: this passes the
option to the compiler call - and that is the wrong address
for it, so I get an error message. Basically I think that
it is 99% a CMake question and maximum 1% a QtCreator
problem, so I am asking this question here. And now I am
stuck - if not some friendly and knowing person can help me
out of this! Thanks for any helpful hint! Cornelis
> > 
> > 
> > -- 
> > Regards,
> > Konstantin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://cmake.org/pipermail/cmake/attachments/20180218/65dee9d5/attachment-0001.html>