[vtkusers] faster compilation (was ANSI C++ stdlib issue (windows), precompiled header (fixed))

Tue Oct 10 08:39:20 EDT 2000

At 16/09/00 09:06, Will Schroeder wrote:

>The bad news is that enabling VTK ANSI stdlib has, to say the least, a 
>disastrous impact on compile time.
>Example. 8 classes, on a Pentium II @ 300 Mhz, 200 RAM, NT4 + MSVC 6.0
>
>without VTK ANSI stdlib support: DLL build time : 45 s.
>with VTK ANSI stdlib support: DLL build time : 86 s.
>
>Yes, my dear fellows, compilation is *two times* slower on my computer 
>with ANSI support

For these interested in *faster* compilation, I finally managed to reduce 
compile time to :

without VTK ANSI stdlib support: DLL build time : 25 s. (was 45 s.)
with VTK ANSI stdlib support: DLL build time : 34 s. (was 86 s.)

I divided VTK ANSI compile time approx. by 3 by using the "precompiled 
header" feature of MSVC, although I previously said  that :

>I guess it could compile faster if the precompiled header was handled 
>correctly, but it's rebuilt for each file on my configuration, which 
>actually slows done the whole stuff ! (flags /YX). The Consistency Rules 
>for the /YX flag are not very clear, I do not know what I'm doing wrong

Now I guess I understand how to solve that issue (the headers in the 
precompiled file shall really be found in the same order in each source 
file). I browsed the archive and found John Biddiscombe's article :

>From: "John Biddiscombe" <j.biddiscombe at rl.ac.uk>
>Subject: Re: [TIP] windows build : low priority
>Date: Tue, 30 May 2000 22:29:28 +0100
>
>I experimented some time back with Borland's Precompiled headers [...]
>I managed to reduce my "full build" of vtk from something
>like an hour to about 5 minutes, by
>
>#include "Precompiled_header.h"
>#pragma hdrstop
>
>at the start of all *.cxx files.
>
>[
>Precompiled_header.h contained
>#include "vtkCommon.h"
>#include "vtkGraphics.h"
>#include "vtkImaging.h"
>]
>
>As a regular CVS user now, I've abandoned this because it's too
>much trouble maintaining two copies of everything.
>
>How about a petition to add
>#ifdef VTK_USE_PRECOMPILED_HEADERS
>#include "Precompiled_header.h"
>#pragma hdrstop
>#endif

I do agree with him  (regarding the petition :)

The Borland Compiler has always been faster than the Microsoft Compiler, 
therefore I wasn't expecting to cut down compile time from "an hour to 5 
minute" :), but two or three times faster is OK for me :)

Hopefully, the Microsoft Compiler solution is the same as the Borland stuff.

I put :

#ifdef USE_PCH
#  ifdef _MSC_VER
#    include "vtkPrecompiledHeaders.h"
#    pragma hdrstop
#  endif
#endif

at the beginning of each *.cxx.
_MSC_VER is set by the Microsoft Compiler, thus Unix users won't bother 
precompiled stuff. There shall be a corresponding directive for Borland 
compiler, thus adding it after _MSC_VER will do the trick too.

Then I compile them using these flags :
/YX /Fp$(LIBDIR)\vc60.pch /D "USE_PCH"
(LIBDIR is the path where my libs are stored, and /YX triggers precompiled 
header).

The important thing is : what shall I put in vtkPrecompiledHeaders.h ?

I tried :

#ifndef __vtkPrecompiledHeaders_h
#define __vtkPrecompiledHeaders_h

#include "vtkCommon.h"
#include "vtkGraphics.h"
#include "vtkImaging.h"

#endif /* __vtkPrecompiledHeaders_h */

but the compiler crashed and burned : out of heap space, and various other 
memory problems. Setting /Zm to the max did not help, although I have 200 
RAM and enough swap space. This is a bad news, the Microsoft Compiler seems 
not to be able to handle big precompiled headers.

Thus, I removed each header one by one, until it worked with :

#include "vtkCommon.h"

which is better than nothing.
At this time, my compile time was 69 s., instead of the previous 85 s. with 
no precompiled headers.

But not satisfying me :)
I wrote a Perl script that automatically generate 
local/vtkPrecompiledHeaders.h with the 'n' most dependent header files 
(parsed from the vtkdll/makefile). For example, the 10 most are :

#include "vtkWin32Header.h"  // 589
#include "vtkSetGet.h" // 589
#include "vtkSystemIncludes.h" // 589
#include "vtkTimeStamp.h" // 589
#include "vtkIndent.h" // 589
#include "vtkObject.h" // 589
#include "vtkDataArray.h" // 543
#include "vtkIdList.h" // 527
#include "vtkObjectFactory.h" // 520
#include "vtkAttributeData.h" // 505

Then I tried with different 'n'.
Here are the results :

n / compile time / size of the dll / size of the precompiled header / total 
size of the .obj
********************************************
10 / 71 s. / 512 Ko / 6157 Ko / 3243 Ko
20 / 57 s. / 512 Ko / 7189 Ko / 3243 Ko
40 / 36 s. / 512 Ko / 9794 Ko / 3725 Ko
50 / 34 s. / 520 Ko / 10257 Ko / 4157 Ko
60 / 37 s. / 565 Ko / 10928 Ko / 4852 Ko
75 / 59 s. / 819 Ko / 16732 Ko / 7984 Ko
100 / 83 s. / 1069 Ko / 21308 Ko / 11048 Ko

As you see, the more headers in the precompiled header file, the larger it 
grows, and the larger the .obj and the .dll are. Thus, at a given point the 
compiler spends lots of time reading and writing big amount of data to the 
disk, and it slows done the compilation process, disabling the precompiled 
header benefit.

On my configuration, the best threshold is 50 headers in 
vtkPrecompiledHeaders.h :
with VTK ANSI stdlib support: DLL build time : 34 s. (was 86 s.)
without VTK ANSI stdlib support: DLL build time : 25 s. (was 45 s.)

Needless to say that it would be nice to see this feature in the VTK source 
tree :), but it requires adding a couple of line at the beginning of each 
.cxx. This would have no influence until the user choose to compile VTK 
with the /YX /D "USE_PCH" flags, and provides a "vtkPrecompiledHeaders.h" 
somewhere in the include directories.

The Perl script is here :
ftp://sic.sp2mi.univ-poitiers.fr/pub/barre/vtk/build_pch.pl
and shall be run in the vtk source tree, after pcmaker has been run. Use 
--from to specify where your build directory is (the script needs to parse 
vtkdll/makefile). By default it dumps vtkPrecompiledHeaders.h to the local/ 
directory (add the according -I flag to the compilation options, or check 
the "local" box in pcmaker). The --limit n parameters might be used to 
write at most 'n' headers. Use --help anyway.

Hope this helps.