[CMake] Building all tests in parallel (whole subtrees)

[Alan W. Irwin]

On 2013-01-24 12:26+0100 wmknapik at gmail.com wrote:

> Could you describe your approach in more detail ? If I understand it correctly, it requires quite a bit of manual maintenance...

It does require initial work to set up the dependencies (see below)
properly to avoid race conditions.  But there should be no on-going
maintenance burden after that.

>
> I've tried the following so far:
> 1) Created a "ut" target on top and for every unit test did add_dependency(ut that_ut). That works fine in parallel, but builds *all* the tests - not enough granularity.
> 2) Created a "ut" target at the top (with no dependencies) and set the global property ALLOW_DUPLICATE_CUSTOM_TARGETS and USE_SUBDIRS to true. In every directory with tests in it, I created a "ut" target that depends on building the test. That works exactly how I want it to, but breaks when building in parallel (race conditions in writing to files that are common dependencies). This approach would require zero maintenance, so it would be my preferred one, but I don't know if the parallelism issue can be fixed. Ideas ?
> 3) I managed to force ctest to build the tests in parallel, but when calling ctest with -j (multiple tests building in parallel), the same issues cropped up as in 2) and I prefer 2) anyway.
>
> Hints on fixing 2) would be very much appreciated. Time is running out... ;)

I was unaware of the ALLOW_DUPLICATE_CUSTOM_TARGETS property, but
looking that up in the documentation I think the caveats for that are
pretty big so I would never recommend that.  You may have no interest
in any backend other than GNU make for now, but you and your users may
have different ideas later.  Therefore, I advice using unique target
names (just like I do).  If you happen to like a particular target
name such as "ut", then append a suffix to it (such as subdirectory
name) to make that target easy to identify and globally unique.

I don't understand _why_ you are running into race conditions with
parallel builds.  Sure that is the fundamental issue with parallel
builds, but that is addressed by implementing proper dependencies. 
You used a pure Makefile approach previously.  Didn't you have the
same problem there until you fixed all dependencies?  Assuming your
pure Makefile system did have proper dependencies, all you have to do
is also implement those same dependencies for your CMake-based system.

Note, CMake has both file dependencies and target dependencies, and
both have to be set up correctly in order for parallel builds to work
properly.  CMake often does the right thing here, but there are also
cases where you have to explicitly tell it about dependencies. For
more detail on the two kinds of dependencies look extremely carefully
at the documentation of add_custom_command, add_custom_target, and
add_dependencies.  I tend to carefully re-read that documentation
every time I implement a new test target.

The principal advantage of add_test and ctest is you can implement
automatic testing and reporting of those tests.  So in principle (I
have never tried this, but it is on my agenda for all my projects) you
could add ctest-style tests corresponding to each of your test
targets.  For example, I think you could add one ctest-style test
corresponding to running make for your overall "ut" target and/or add
multiple ctest-style tests corresponding to running make for each "ut"
target in each subdirectory.  But none of this will work properly
until you have implemented the correct dependencies for your many test
targets so ctest-style tests are something to think about for later.

I believe I have responded to all your questions above and also
questions implied by your posts before and after the above one.
However, often the example of an implemented build system for a
real-world project is a quicker method than question and answer to
learn about CMake. Therefore, I suggest you take a look at the
CMake-based build systems of the recently released ephcom-3.0.0 and
te_gen-2.0.0 projects where I implemented a fair number of test
targets with proper dependencies between them.  See timephem.sf.net
for access to the release announcements which should give you further
links to download or browse those projects.  By the way,
timephem.sf.net is outdated by those new releases, and I plan to
update it appropriately soon. But until I do that, trust the release
announcements more than timeephem.sf.net.  If you have further
questions about the ephcom and/or te_gen build systems, don't hesitate
to contact me again on or off list as you think appropriate.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
-------------- next part --------------
--



Powered by www.kitware.com



Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html



Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ



Follow this link to subscribe/unsubscribe:

http://www.cmake.org/mailman/listinfo/cmake