[CMake] Introducing the "build_projects" project and discussion of the Ninja efficiency/reliability I get for that project

[Alan W. Irwin]

The "build_projects" project (which for now can be obtained by

svn checkout \
svn://svn.code.sf.net/p/plplot/code/trunk/cmake/build_projects

) grew out of a casual discussion on the cmake-devel mailing list a
few weeks ago.

The issue is that most free software developers working on the Windows
platform need to build their own free software libraries with a
coherent tool chain.  To automate that process, the idea is to build
the desired free software projects systematically using the CMake
ExternalProject_Add command.  Currently I only have 5 such builds
configured (mostly soft dependencies of PLplot) for the build_projects
project as a proof of concept.  But that has been such a success, I
have plans to greatly expand that list to additional soft dependencies
of PLplot (such as wxwidgets, the Qt4 library and the pango and
dependencies subset of the GTK-related libraries), PLplot itself, and
also some additional projects of interest to me (such as ephcom,
te_gen, and FreeEOS and their dependencies). I encourage others to
contribute build configurations as well of their favorite must-have
free software on Windows.

Adding a basic configuration without build dependencies should be
extremely straightforward.  Using shapelib/bp.cmake from the project
as a template it should only be a matter of a few minutes to write
down the download location, the appropriate checksum for that
download, the configuration command, the build command, the test
command, and the install command that should be used by
ExternalProject_Add.

The CMake ExternalProject_Add command is quite general and allows use
of essentially any valid combination of configurator (such as
autotools or CMake) and build tool (such as gmake or ninja) on any
platform.  So far, the 5 projects I have configured happen to have
CMake-based build systems (although the next one (wxwidgets) will be
autotools based.) So I can try a number of different generators for
those builds.  One of those "NMake Makefiles JOM" for the jom build
tool failed utterly because it currently appears to not be compatible
with the MinGW compiler suite. (I have collected all the information
Bill Hoffman requested in a simple test case so once he runs that he
may be able to diagnose and fix this issue.)

Here are the current results I have for those builds for the remaining 5
different generators that I have tried on Linux and the Wine version
of Windows.  All of these produced perfect build results for the 5
different projects (which for the record are currently ndiff, libqhull,
shapelib, libagg, and libharu, but that list is about to be greatly expanded).

"Unix Makefiles" on linux
wine at raven> time make build_all >& build_all.out

real    0m17.614s
user    0m20.421s
sys     0m4.104s

"Ninja" on linux
wine at raven> time ninja build_all >& build_all.out

real    0m15.490s
user    0m20.633s
sys     0m3.764s


"MSYS Makefiles" on wine-git
bash.exe-3.1$ time make build_all >& build_all.out

real    4m17.073s
user    0m20.040s
sys     0m48.780s

"MinGW Makefiles" on wine-git
bash.exe-3.1$ time mingw32-make build_all >& build_all.out

real    2m31.398s
user    0m0.000s
sys     0m0.020s

"Ninja" on wine-git
bash.exe-3.1$ time ninja build_all >& build_all.out

real    1m31.620s
user    0m0.000s
sys     0m0.040s

All builds (and tests) for the 5 different projects showed no issues
except for the combinations of "Ninja" on wine-git where one of the
projects (libqhull) had a broken build for the special case of the
combination of a shared library and an executable that was renamed to
have the same name kernel as the library.  I implemented a simple test
case to demonstrate that issue which is currently being discussed on
the CMake devel mailing list, but I temporarily worked around the
issue here by using a static library for the libqhull build.

The ninja build on Linux only provides a tiny speed boost for that
platform, but I still like ninja for that case because I was able to
diagnose some dependency issues in my initial attempt at
build_projects using ninja that gmake ignored.  As a result of
changing the build_projects logic so that ninja was satisfied on the
dependency question, the whole build speeded up considerably for all
generators and platforms.  For example, both "MinGW Makefiles" and
"MSYS Makefiles" speeded up but the former by a larger factor so that
it is now clearly superior in speed to "MSYS Makefiles" whereas for my
first attempt at build_projects "MinGW Makefiles" and "MSYS Makefiles"
had very similar speeds.

Once the bug I discovered for the "Ninja" on wine-git case is sorted
out I will also prefer that over either "MinGW Makefiles" or "MSYS
Makefiles" on Wine because of its superiority in speed. If Bill is
able to fix the "NMake Makefiles JOM" generator so that it works
properly with MinGW than I should also be able to compare the speed
and reliability of build results from that generator compared to the
other three generators I am currently testing on Wine.

In general, as you can see above, the Wine platform is a slow build
platform (six times slower for the ninja case) because of the large
startup latency costs on that platform due to the inefficiency of fork
implementations on Windows platforms.  If a project build consists of
compilations of a lot of small files, then the startup latency will
dominate the efficiency question and the resulting builds will take an
order of magnitude longer (the typical ratio of the startup speeds) to
finish than the Linux case.  But if longer compilations or tests are
part of the build, then startup latency doesn't dominate the
efficiency question quite so much. Typically, I have seen Wine
slowdowns for builds and tests ranging from a factor of two (for
te_gen) to the current factor of 6 for the average of the 5 projects
currently configured in build_projects. But as far as I am concerned I
will put up with that cost (whatever the slowdown factor happens to be
in the range from two to six) for the huge benefit of having a free
(in both senses) Windows platform where I can automatically test lots
of different software builds and tests against the equivalent Linux
results.

The last time I tried to build software (by hand) on Wine was for
wine-1.5.19, and that proved to be an extremely reliable platform for
that purpose.  In the series of experimental 1.5.x releases since then
two regressions crept into Wine that I discovered with the
build_projects package.  Fortunately, the Wine developers have quickly
responded to those regression reports so that wine-git (and presumably
wine-1.6_rc1 which is almost the same as my wine-git snapshot) has
those regressions fixed. So through the expansion of the packages
configured to build by build_projects I hope not only to complete in
an automated way the builds I did by hand for wine-1.5.19 but also go
far beyond those so that wine-1.6 (when it comes out) should have an
extremely solid reputation as a build platform.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________