[vtk-developers] vtk-8.2.0-rc2 problem building wheels

[Ben Boeckel]

On Wed, Dec 12, 2018 at 14:35:37 -0500, Jean-Christophe Fillion-Robin wrote:
> >> module can be built inside and outside of VTK
> > But the better question is "should we?".
> 
> Generally speaking yes, supporting the composition of CMake projects by
> both (1) "aggregation" (add_subdirectory)  and (2) "finding dependency" is
> important.

I'm not seeing a rationale for this though. "Because it's important"
seems awfully tautological to me. VTK and other projects we maintain are
the only ones I'm aware of which support this kind of stuff. I don't see
it in the wider FOSS ecosystem. A similar situation happens nginx adding
modules to build by extracting into a certain subdirectory, but then
again, that's the *only* supported way of building nginx modules.

> This applies to first class modules (e.g VTK module, ITK module, ... ) and
> also third-party project (e.g zlib, hdf5, ...)

Third party projects are different. It's either given an already built
copy or it builds its own. It is done, basically, because Windows. If it
weren't for that, I'd be really partial to just say "use apt or brew".
Even there, the third party projects that get built as part of VTK are
pretty set-in-stone. I've removed all user-facing variables from them.
For example, if one wants an MPI-aware HDF5, it must be provided
externally; VTK's hdf5 is never MPI-aware. Similarly, VTK's png has been
stripped of its vectorized code (due to it simplifying the build by
getting rid of lots of try_compile checks), and other projects only
build what VTK needs from it.

> Also, by having no differences between the build system of "module or
> project" that can be either be aggregated or built as dependency, it is
> easy to maintain and communicate a set of best practices, streamline use of
> projects in various scenarios without having to maintain forks.

There's always going to be a difference. I'd rather just document how to
use VTK via `find_package(VTK)`.

> > The thing is that the module is incompatible from a consumer point of
> view in the two builds. One makes `VTK::RenderingOpenVR` and is provided
> via `find_package(VTK COMPONENTS RenderingOpenVR)`.
> 
> I think this is something that we probably need to address in CMake. In the
> mean time, we could have a "vtk_ns" and vtk_ns_colon variables and
> reference target with "${vtk_ns_colon}RenderingOpenVR". Also wondering if
> introducing some new genex could also help.

This sounds super hacky to me. The module system certainly isn't going
to support it; it's complex enough as it is to deal with namespaces.
This is the best I can see right now:

    if (VTK_HAS_OPENVR) # variable provided by the user
      find_package(VTK REQUIRED COMPONENTS RenderingOpenVR)
      set(openvr_target VTK::RenderingOpenVR)
    else ()
      find_package(VTKRenderingOpenVR REQUIRED)
      # It really shouldn't provide VTK::RenderingOpenVR.
      set(openvr_target VTKRenderingOpenVR)
    endif ()

> > Would this outside-VTK build expect to put its Python module under
> `vtkmodules` still?
> 
> Good question. When building wheels, the answer is yes. That way, the
> content of all "VTK" wheels would end up in the same folder "vtkmodules"
> and shared library are in the same directory. Installing/uninstalling
> wheels also work well because pip has a manifest and only removes the files
> it needs to.

The problem is that this breaks `vtkmodules.all.vtkOpenVROverlay`
because `vtkmodules.all` only includes modules built by VTK itself. It
means that we really shouldn't provide it at all if users have to know
the module name anyways.

> > Does Slicer make VTK modules itself or just consume them?
> 
> Almost both.
> 
> Consume them: yes
> 
> For the "make VTK modules", there are few cases:
> - reuse of VTK wrapping infrastructure through
> Slicer/CMake/vtkMacroKitPythonWrap.cmake
> <https://github.com/Slicer/Slicer/blob/master/CMake/vtkMacroKitPythonWrap.cmake>

The new wrapping code assumes VTK modules (i.e., built via
`vtk_module_scan` and `vtk_module_build`). If the relevant properties
aren't set on the target to be wrapped, they'll not do the right things.

> - we have a lot of "pure" VTK library that currently use their own build
> system. In the future, it would be nice to standardize on reusing VTK build
> system.

That should be doable.

> - building of VTK/Rendering/OpenVR into a Slicer extension.

Why is it not just used as a dependency like this?

    find_package(VTK COMPONENTS RenderingOpenVR)
    if (VTK_RenderingOpenVR_FOUND)
      add_subdirectory(openvr_ext)
    endif ()

--Ben