From Victor.Poughon at cnes.fr  Fri Jul  6 08:21:19 2018
From: Victor.Poughon at cnes.fr (Poughon Victor)
Date: Fri, 6 Jul 2018 12:21:19 +0000
Subject: [ITK-dev] extern templates to reduce build time
Message-ID: <3E55146A6A81B44A9CB69CAB65908CEA53F33449@TW-MBX-P01.cnesnet.ad.cnes.fr>

Hello,

In OTB (which is heavily based on ITK) we have been looking at the possibility of introducing explicit instantiation of common template classes in the dynamic libraries we build, together with "extern template" declarations in headers to reduce build times significantly. A quick grep in ITK source shows that this C++11 feature is barely used in ITK, and it seems like it is for another purpose (dynamic_cast issues).

After a few test on our side, the best approach seems to be:

- In the header file: declare common template classes specializations as "extern template"
- In the cxx file: explicitely instantiate the same specializations

This means that client code will not need to instantiate a template type (reducing build time significantly). It will need however to link with the corresponding lib. The benefit of this approach is supported by doing a quick:

 $ nm -g --demangle $(find . -name "*.o") | sort | uniq -c | sort -n | grep " W "

in a build directory, and noticing that in a full build, the compiler instantiates the same template types tens of thousands of times. (why it does not keep a cache is beyond me...). This can be seen with the compiler option -ftime-report, also.

This approach is promising (about ~15% reduction in build time in my initial tests, more can be achieved I think), but raises two issues which are probably worth discussing with ITK devs:

1. How to handle specialization on a type that is in another module, or another library? For example in OTB we have some:
itk::ImageSource<otb::VectorImage<double, 2u> >

(and many more). Where should the "extern template" declarations and explicit instantiations for this type be in the code? extern template declarations require the header of the type on which the template is specialized the be included (I think). This poses no issues for itk::Image<double>, but some issues for more complex templating.

2. How to handle MSVC and gcc incompatibilities regarding library export keywords (decltype, dllexport, _EXPORT macros & co). This issue is explained well by this page:

https://github.com/RobotLocomotion/drake/issues/3305

>From my research I found that ITK handles that somewhat with EXPORT_EXPLICIT macros. Other C++ projects out there have tackled that issue as well. See for example:

https://bugs.chromium.org/p/chromium/issues/detail?id=420695
https://chromium.googlesource.com/chromium/src/+/66.0.3359.158/base/export_template.h

So, questions:
- Would ITK be interested in using extern templates to reduce build time? Which approach would you take?
- How to make extern templates portable accross MSVC and gcc/clang? Something like Chromium's export_template.h? How does this interact with CMake's generation of _EXPORT macros? Maybe it needs to be a patch all the way in CMake's generate_export_header?

Thanks,

Victor Poughon

From blowekamp at mail.nih.gov  Fri Jul  6 09:00:54 2018
From: blowekamp at mail.nih.gov (Lowekamp, Bradley (NIH/NLM/LHC) [C])
Date: Fri, 6 Jul 2018 13:00:54 +0000
Subject: [ITK-dev] [ITK]  extern templates to reduce build time
Message-ID: <F208EFEB-7220-421B-BAF3-72606B2D86C6@mail.nih.gov>

Can you please post this topic in discourse?

Brad

?On 7/6/18, 8:31 AM, "Poughon Victor" <Victor.Poughon at cnes.fr> wrote:

    Hello,
    
    In OTB (which is heavily based on ITK) we have been looking at the possibility of introducing explicit instantiation of common template classes in the dynamic libraries we build, together with "extern template" declarations in headers to reduce build times significantly. A quick grep in ITK source shows that this C++11 feature is barely used in ITK, and it seems like it is for another purpose (dynamic_cast issues).
    
    After a few test on our side, the best approach seems to be:
    
    - In the header file: declare common template classes specializations as "extern template"
    - In the cxx file: explicitely instantiate the same specializations
    
    This means that client code will not need to instantiate a template type (reducing build time significantly). It will need however to link with the corresponding lib. The benefit of this approach is supported by doing a quick:
    
     $ nm -g --demangle $(find . -name "*.o") | sort | uniq -c | sort -n | grep " W "
    
    in a build directory, and noticing that in a full build, the compiler instantiates the same template types tens of thousands of times. (why it does not keep a cache is beyond me...). This can be seen with the compiler option -ftime-report, also.
    
    This approach is promising (about ~15% reduction in build time in my initial tests, more can be achieved I think), but raises two issues which are probably worth discussing with ITK devs:
    
    1. How to handle specialization on a type that is in another module, or another library? For example in OTB we have some:
    itk::ImageSource<otb::VectorImage<double, 2u> >
    
    (and many more). Where should the "extern template" declarations and explicit instantiations for this type be in the code? extern template declarations require the header of the type on which the template is specialized the be included (I think). This poses no issues for itk::Image<double>, but some issues for more complex templating.
    
    2. How to handle MSVC and gcc incompatibilities regarding library export keywords (decltype, dllexport, _EXPORT macros & co). This issue is explained well by this page:
    
    https://github.com/RobotLocomotion/drake/issues/3305
    
    From my research I found that ITK handles that somewhat with EXPORT_EXPLICIT macros. Other C++ projects out there have tackled that issue as well. See for example:
    
    https://bugs.chromium.org/p/chromium/issues/detail?id=420695
    https://chromium.googlesource.com/chromium/src/+/66.0.3359.158/base/export_template.h
    
    So, questions:
    - Would ITK be interested in using extern templates to reduce build time? Which approach would you take?
    - How to make extern templates portable accross MSVC and gcc/clang? Something like Chromium's export_template.h? How does this interact with CMake's generation of _EXPORT macros? Maybe it needs to be a patch all the way in CMake's generate_export_header?
    
    Thanks,
    
    Victor Poughon
    The ITK community is transitioning from this mailing list to discourse.itk.org. Please join us there!
    ______________________________________
    Powered by www.kitware.com
    
    Visit other Kitware open-source projects at
    http://www.kitware.com/opensource/opensource.html
    
    Kitware offers ITK Training Courses, for more information visit:
    http://kitware.com/products/protraining.php
    
    Please keep messages on-topic and check the ITK FAQ at:
    http://www.itk.org/Wiki/ITK_FAQ
    
    Follow this link to subscribe/unsubscribe:
    https://itk.org/mailman/listinfo/insight-developers
    The ITK community is transitioning from this mailing list to discourse.itk.org. Please join us there!
    ________________________________________
    Community mailing list
    Community at itk.org
    https://itk.org/mailman/listinfo/community