[CMake] custom commands, targets and parallel builds

[Kris Thielemans]

> From: Alan W. Irwin,  Sent: 25 November 2018 01:48

Many thanks Alan, answers and questions below

> On 2018-11-25 00:23-0000 Kris Thielemans wrote:
> 
> > I'm trying to add doxygen-generated comments to my Python module build
> > via SWIG using https://github.com/m7thon/doxy2swig. This means I need
> > to run doxygen first, then run doxy2swig, and only then run swig.
> > However, I'm getting reports that parallel builds are failing
(sometimes).
> >
> >
> >
> > My strategy is based on CMake advice and some blogs (e.g. at
> >
https://samthursfield.wordpress.com/2015/11/21/cmake-dependencies-between-ta
rgets-and-files-and-custom-commands/ ) and relies on creating
> > custom commands to generate the files and custom targets to be able to
> > specify dependencies on those generated files. Somewhat simplified
> > code is below (original is at https://github.com/UCL/STIR/pull/280)
> >
> >
> >
> > # command that runs doxygen
> >
> > add_custom_command(
> >
> > OUTPUT doxygen.stamp
> >
> > DEPENDS ${doxyfile}
> >
> > COMMAND ${DOXYGEN_EXECUTABLE} ${doxyfile}
> >
> > COMMAND cmake -E touch doxygen.stamp
> >
> > .)
> >
> > # corresponding target
> >
> > add_custom_target( doc ALL DEPENDS doxygen.stamp)
> >
> >
> >
> > # command that runs doxy2swig
> >
> > add_custom_command(
> >
> > OUTPUT STIR_DOXY2SWIG.i
> >
> > DEPENDS doc
> >
> > COMMAND ${PYTHON_EXECUTABLE} doxy2swig.py -c index.xml
> > STIR_DOXY2SWIG.i
> >
> > )
> >
> > # corresponding target
> >
> > add_custom_target(doxy2swig DEPENDS STIR_DOXY2SWIG.i)
> >
> >
> >
> > # now add that target to the SIWG module
> >
> > add_dependencies(${SWIG_MODULE_stir_REAL_NAME} doxy2swig)
> >
> >
> >
> > Any suggestions on what I'm doing wrong?
> >
> >
> >
> > Many thanks!
> >
> > Kris
> >
> >
> >
> > PS: I note that FindDoxygen's doxygen_add_docs merges the
> > add_custom_command and target into one add_custom_target, see
> >
>
https://github.com/Kitware/CMake/blob/a44191abc489373d295ecaeb5c2eb153c876a1
a/Modules/FindDoxygen.cmake#L1104
> >
> > I thought that couldn't be done but I guess I was wrong.
> >
> > PS: I also note that doxygen_add_docs doesn't create a doxygen.stamp
> > file to prevent re-runs though, is that not needed then?
> 
> Hi Kris:
> 
> To answer your last set of questions first, you can execute all desired
> commands with add_custom_target, but the issue is that COMMAND always
> reruns.  So your way of doing it above (with paired
> add_custom_command/add_custom_target) is the recommended procedure
> which should only re-run if the OUTPUT file is non-existent or older than
a
> dependent file.
> 

Ok. That makes sense to me. thanks!

> Anyhow, I think what you have outlined above is generally correct.
> For example, your "DEPENDS doc" argument to add_custom_command where
> doc is a custom target argument is correct according to the documentation
at
> <https://cmake.org/cmake/help/latest/command/add_custom_command.html
> >.
> However, your simplified code above has several deviations from the
pattern I
> always use.  I know my pattern generally works, and I think at least your
first
> two deviations from it are important.
> 
> 1.  I always like to specify the full pathname for all files (e.g.,
> ${CMAKE_CURRENT_BINARY_DIR}/doxygen.stamp and
> ${CMAKE_CURRENT_BINARY_DIR}/STIR_DOXY2SWIG.i.  This is true not only for
> OUTPUT files and the corresponding DEPENDS but also for input (which
> typically start with ${CMAKE_CURRENT_SOURCE_DIR} and output files for
> COMMANDs.
> 
> 2. At the same time I make sure I run all commands in
> ${CMAKE_CURRENT_BINARY_DIR} by specifying that as the working directory.
> So that all unmentioned files on the command line are generated in the
build
> tree to keep the source tree as clean as possible.
> 
> 3. I don't think it matters above but I always use the VERBATIM attribute
for
> COMMANDS.
> 

Thanks for these suggestions. RE 2 and 3, yes, I did the same. For the sake
of brevity I cut some of the relevant bits. Sorry! (full code at
https://github.com/UCL/STIR/pull/280)

I'm indeed not using the full pathnames. However, given the
WORKING_DIRECTORY option, all files are "local" as far as CMake concerns, it
seems to me this is not relevant, although I should give it try.

> I have found one of my most difficult CMake tasks is to avoid build race
> conditions that can occur for parallel builds.  So your real code is most
> important in that regard rather than a simplified example of your CMake
logic.
> Perhaps others here would like to comment on the most reliable way to find
> such race conditions.  But what I do is process "make -j<jobs>
<target_name>"
> output results to remove the progress percentage marks, sort those
results,
> and look for anything that is repeated that shouldn't be.  

Sounds painful :-; The person who reported the problem can currently not
provide me with a log file. He's on OSX but I don't know yet what CMake/make
versions he's using.

So instead I did some more digging in the generated makefiles. From these,
it seems all is ok (see end of email). I'm now puzzled of course.

> Finally, although I have found parallel builds to be reliable on Linux,
Arjen
> Markus, one of the PLplot developers, has discovered that parallel builds
are
> currently unreliable for all Unix-like Windows platforms, e.g., classical
> MinGW/MSYS, its modern replacement MinGW-w64/MSYS2, and Cygwin. So if
> some of your users are reporting parallel build troubles on any of those
> platforms it is likely an issue with the make command on those platforms
> rather than your build system.
>

That is pretty amazing. Who knows, maybe "make" on OSX should be added to
the list of problematic "make" implementations as well.

If I can get dig out anything else on reproducing the problem or debugging,
I'll let you know. Thanks again for your help!

Kris

PS: some checks on build files

== Visual Studio 2015 generated files (with CMake 3.11.4):

My swig module _stir has a reference "doxy2swig", which has a reference
"doc", so that seems fine. (I'm assuming that a reference means it'll check
dependencies).

== Unix Makefiles generated by CMake 3.7 and 3.13.0:
I first got confused by the fact that cross-directory dependencies in the
makefiles in each sub-dir are non-existent, but I realised that "make" is
called for each of these from the "master" makefile. Checking
CMakeFiles/Makefile2 in the root of my build, everything seems fine as this
contains dependencies:

src/swig/CMakeFiles/doxy2swig.dir/all: src/CMakeFiles/doc.dir/all
src/swig/CMakeFiles/_stir.dir/all: src/swig/CMakeFiles/doxy2swig.dir/all

(I've now learned that using "make" from in one of the subdirs is a
dangerous thing to do as it doesn't check dependencies properly, but that
makes some sense.)