[vtk-developers] vtk-8.2.0-rc2 problem building wheels

[Ben Boeckel]

On Wed, Dec 12, 2018 at 19:08:30 -0500, Jean-Christophe Fillion-Robin wrote:
> > I'm not seeing a rationale
> 
> By ensuring the build system of a VTK/ITK/Slicer... modules is the same if
> it is built inside or outside the "main" application/libraries allows the
> following:

It *cannot* be the same. `find_package(VTK)` only contains the
information VTK had at the time of *its* build. Modules built outside of
it won't show up. If a module is part of VTK, VTK does lots of heavy
lifting one-time-setup things. Install directories, minimum CMake
version, CMake policies, etc. Any standalone module will have to copy
that information. I suspect that's at least 100 lines of CMake code
(some of which can't be abstracted out).

> - avoid special case, tribal knowledge, ... : a module layout is consistent

This support itself is a special case itself AFAICS. Module layouts are
consistent right now. Modules which do this are going to be the
different ones.

> - support for packaging independently some part

That's just more code that modules will have to duplicate from VTK.

> @Matt: Could you share your experience within the ITK community ?

<Matt's reply>

This seems more like remote modules as a path for migrating *into* VTK.
Building OpenVR separately is like moving it *out* of VTK. *That* is
what I don't understand.

> > > > Would this outside-VTK build expect to put its Python module under
> `vtkmodules` still?
> > > Good question. When building wheels, the answer is yes. [...]
> > The problem is that this breaks `vtkmodules.all.vtkOpenVROverlay` because
> `vtkmodules.all` only includes modules built by VTK itself.
> > It means that we really shouldn't provide it at all if users have to know
> the module name anyways.
> 
> In a nutshell, python support the concept of "entry_points". Each time a
> vtk wheel associated with a module is installed, the namespace
> "vtk.modules" could automatically be updated with a new entry (e.g
> "vtk.modules.vtkopenvr"). Then, in the "all.py" we would make use of
> "pkg_resources.iter_entry_points" to import the associated classes.
> 
> As similar approach was adopted by Girder, see here
> <https://github.com/girder/girder/blob/cfb58ef245384fff7f90cf4cd6c8cab94510b9f6/plugins/homepage/setup.py#L47-L51>

OK, maybe you've solved Python. Now what of CMake (`find_package`) and
Java?

> > The new wrapping code assumes VTK modules (i.e., built via
> `vtk_module_scan` and `vtk_module_build`). If the relevant properties
> aren't set on the target to be wrapped, they'll not do the right things.
> 
> I see. By copying the relevant CMake functions from VTK 8.2, we should be
> able to keep things working.

What? I'm not maintaining that. The information that is used by the old
wrapping macros doesn't exist anymore either; it's all stored on
properites of the targets in which case it's basically the new wrapping
functions anyways. In addition, since PythonD libraries are now gone,
modules must import dependent modules first. Here's vtkPVCommon.py from
ParaView:

    from __future__ import absolute_import
    from . import vtkClientServer
    from vtkmodules import vtkCommonCore
    from vtkmodules import vtkIOXMLParser
    from .vtkPVCommonPython import *

If a module depends on one not using the standard macros, where the
Python bindings for the module is stored needs to be accessible somehow.

> > maybe RenderingOpenVR should just become a remote module instead?
> 
> Since adding a remote module simply means download the source, then call
> "add_subdirectory()" with two parameters. Unless a VTK module can be
> implemented so that it can be built inside and outside of the source tree,
> I don't think it will be supported.

Remote modules with the new system would not be `add_subdirectory`. The
downloaded source directory is added to the `vtk_modules_find_modules`
function. What doesn't work right now is that it assumes everything is
in the source tree.

> > > - building of VTK/Rendering/OpenVR into a Slicer extension.
> 
> > Why is it not just used as a dependency like this?
> >    find_package(VTK COMPONENTS RenderingOpenVR)
> >    if (VTK_RenderingOpenVR_FOUND)
> >      add_subdirectory(openvr_ext)
> >    endif ()
> 
> Because:
> * In order to reuse the install rules of the module and package it, we need
> the associated build tree.

Not in general, you don't. For example, the paraview-superbuild makes
packages from the install trees of all of its projects by doing the
proper dependency traversal of the libraries. The new CMake install
rules work should make it easy for Slicer to package it up from a
separate build/install.

> * The build of VTK in Slicer proper has no dependency on openvr external
> project, this is lever to the SlicerOpenVR extension. See "SuperBuild"
> folder in https://github.com/KitwareMedical/SlicerVirtualReality

Again, this, to me, just points to OpenVR maybe being a separate
repository completely and optionally a remote module. Personally, I'd
have the superbuild turn on the OpenVR dependency in VTK if the
extension is wanted. If it isn't there, the VTK can't be used to build
the SlicerOpenVR extension.

--Ben