[Insight-developers] Parallel builds with CMake?

Thu Oct 21 05:31:48 EDT 2010

On Tue, Oct 19, 2010 at 09:08:04AM -0400, David Cole wrote:
> CMake generates makefiles that are capable of parallel build with "-j N"
> flags to make...
> 
> ParaView, VTK and ITK all should work with -j N.

I have had little success building ITK plus wrappers in parallel.

This is my experience both with official ITK releases (e.g. ITK 3.20
fails to build in parallel) and nightly building (riemann.sumost.ca).
I believe it is always the wrappers that fail.

I have been building ITK 3.x packages for Debian for some years [1].
In December 2008 I enabled parallel building for the first time (ITK
3.10.1) and removed it due to failing builds of ITK 3.12.0 [2,3].
After being advised it was fixed [4] I re-enabled parallel building
for 3.20.0, but it is still broken [5].  If you examine that build
log, you see:

  cd /build/buildd-insighttoolkit_3.20.0-1-i386-z9iqD8/insighttoolkit-3.20.0/obj-i486-linux-gnu/Wrapping/CSwig/VXLNumerics && /usr/bin/cableidx /build/buildd-insighttoolkit_3.20.0-1-i386-z9iqD8/insighttoolkit-3.20.0/obj-i486-linux-gnu/Wrapping/CSwig/VXLNumerics/wrap_vnl_fortran_copy.xml /build/buildd-insighttoolkit_3.20.0-1-i386-z9iqD8/insighttoolkit-3.20.0/obj-i486-linux-gnu/Wrapping/CSwig/VXLNumerics/wrap_vnl_fortran_copy.idx
  ERROR: In /build/buildd-cableswig_0.1.0+cvs20100501-1-i386-7Ju1kS/cableswig-0.1.0+cvs20100501/Cable/Parsers/cableXMLParser.cxx, line 175
  XMLSourceParser (0x832d4c8): Error parsing XML in stream at line 181: unclosed token

  Errors occurred during parsing of GCC-XML output: /build/buildd-insighttoolkit_3.20.0-1-i386-z9iqD8/insighttoolkit-3.20.0/obj-i486-linux-gnu/Wrapping/CSwig/VXLNumerics/wrap_vnl_fortran_copy.xml

I have reproduced this on my machine.  The error always goes away when
built serially.  This is using cswig wrapping.

For the nightly dashboard build [6] I was building in parallel for
quite some time WITHOUT building wrappers.  I enabled wrappers in
2010-04-18 [7] and after some initial difficulties I did manage to get
the nightly of ITK 3.x to build in parallel with wrappers.

After switching to ITK v4, I got a nightly to build SERIALLY with
wrappers at least once [8] (although 16 java tests failed).  The next
night [9] the java wrapping failed to build.  This was fixed, then a
similar error appeared on August 12 [10] -- two missing "SetRadius()"
methods -- an error which persists to this day [11].

Despite the two missing SetRadius() methods, I tried enabling parallel
building on the ITK v4 dashboard August 18 [12] but 70 new errors
appeared [13].  After switched back to serial building, the errors
went back down to the two [14].  So I have to conclude that parallel
building with wrapping was to blame.  That was the status at the end
of August.  

Again, this is with cswig wrapping.  I haven't followed the wrapping
discussions as closely as I might but I have been using cswig because
that is/was the one supported in the main ITK sources and the one that
builds Tcl, Java, and Python.  I'm willing to switch to the new
wrapITK if/when it is the official supported wrapping.

I have not been pro-active in pointing out the build failures, much
less fixing them.  This is unfortunate, but simply due to lack of time
on my part.

I'm willing to try again with the v4 nightly if the developers believe
it worthwhile to have a parallel build with wrapping.  

Cheers,
-Steve

 [1] http://packages.debian.org/changelogs/pool/main/i/insighttoolkit/current/changelog
 [2] http://bugs.debian.org/536999
 [3] http://www.cmake.org/pipermail/insight-users/2009-October/033352.html
 [4] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=578313#55
 [5] https://buildd.debian.org/fetch.cgi?pkg=insighttoolkit&arch=i386&ver=3.20.0-1&stamp=1287604473&file=log&as=raw
 [6] http://www.cdash.org/CDash/viewNotes.php?buildid=754470
 [7] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=578313#35
 [8] http://www.cdash.org/CDash/index.php?project=Insight&date=2010-07-31
 [9] http://www.cdash.org/CDash/viewBuildError.php?buildid=681969
[10] http://www.cdash.org/CDash/viewBuildError.php?buildid=692255
[11] http://www.cdash.org/CDash/viewBuildError.php?buildid=754470
[12] http://www.cdash.org/CDash/viewNotes.php?buildid=697718
[13] http://www.cdash.org/CDash/viewBuildError.php?buildid=697718
[14] http://www.cdash.org/CDash/viewBuildError.php?buildid=701438

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
URL: <http://www.itk.org/mailman/private/insight-developers/attachments/20101021/a8570c47/attachment.pgp>