[vtk-developers] New module system preview

[Ben Boeckel]

On Thu, Nov 08, 2018 at 11:32:59 -0500, Jean-Christophe Fillion-Robin wrote:
> Thanks for the update. This look very promising.
> 
> Also glad to see the "UseVTK.cmake" disappear, the use of directory
> properties to handle VTK_DEFINITIONS has been the source of a lot of
> trouble.
> 
> Will the new system support the following:
> * Creating a VTK SDK installed in a read-only location and support building
> multiple interdependent VTK external modules against it ?

This should work with the current branch.

> * Building any VTK module either externally or along with the VTK project
> ?  (e.g this will be useful for creating python wheels)

I'm not sure the use case is clear to me. How are Python wheels related
to remote modules?

Remote modules have not been ported (neither has its VTK-side
infrastructure) yet. ParaView is a use case of how it would work since
ParaView won't be doing `add_subdirectory(VTK)` at all, but instead use
the module system to scan VTK itself (or, if using an external VTK,
`find_package(VTK)`).

As for the Python wheel part, in the future, I'd like to have
`vtk-superbuild` creating Python wheel packages.

> * Re-using the VTK python wrapping facility without having to buy in the
> VTK internal build system.

It has been tied tighter, if anything, than before. Python wrapping is
given a list of modules from which it extracts the information it needs.
It assumes that hierarchy files have already been made (this is a
private function in `vtkModule.cmake` right now). But, if the hierarchy
file is made outside the module system itself, making the wrapping work
is a matter of writing some target properties:

  - `INTERFACE_vtk_module_depends`
  - `INTERFACE_vtk_module_implements` (though if you're doing this
    without the module system…)
  - `INTERFACE_vtk_module_hierarchy`
  - `INTERFACE_vtk_module_headers`

But, not using the module system to run the wrapping isn't something I
designed for (it's complicated enough as-is).

Note that it should be possible to wrap VTK from outside the project
building VTK using the properties the module system exports. For
example, ActiViz for C# and ClientServer for ParaView with an external
VTK.

> These are the limitations of the current infrastructure that I would also
> like to see addressed by such a drastic change.
> 
> > build against VTK
> 
> Will the way to use find_package(VTK ...) and to specify required module
> change ?

The component names might be different now:

    find_package(VTK COMPONENTS CommonCore OPTIONAL_COMPONENTS ParallelMPI)

> I also think updating VTK example to work with the new build system would
> be a good test case, this will provide an path forward for project that
> need to support both.

Updating Bill Lorensen's repository is on the list. Updating ParaView is
more urgent though since it needs to be updated before VTK can land. We
can land the updated examples after the initial merge though.

> > `vtk.module` / `vtk.kit`  files
> 
> While this is an implementation details, would it be more sensible to
> standardize on friendly for human, readable by machine and declarative by
> nature format ? E.g json, toml, ..
> Or may be something like the "Common Package Specification" (see
> https://mwoehlke.github.io/cps/) ?

It has to be read in CMake code…so something that is CMake-like is
easier. If CMake were to add JSON support to the CMake language, then we
could also look for `vtk.module.json` files. Until that happens, I'm not
writing a JSON or TOML parser in CMake.

> > rename OpenGL2 back to OpenGL as was the original intent?
> 
> I would prefer we do NOT to that for the next release, it's gonna make work
> of user of VTK even more difficult. We are still working our way to handle
> the switch of VTK major version from 9 to 8 ...

I agree with Ken. CMake code already needs a rework. I think we should
only do this once. However, it will be renamed after the initial
landing. The branch is big enough already: 300+ commits and diffstat
below. This is because renaming modules requires touching *every* header
in the module (for the export macro) whereas the module system is
largely just CMake changes (there are minor code updates, mainly in
testing code to fix some data paths).

    3736 files changed, 343695 insertions(+), 466352 deletions(-)

> Also in the future, I anticipate we will have new version of backend.
> Should the module be named after the minimum version of OpenGL it support
> ?  (e.g OpenGL3.2)

Ken already addressed this.

> Or since one backend can be built at a time, should alias target be created
> instead ?

Only building a single backend should no longer be a thing. Backend
selection should be done through `vtkObjectFactory` and applications
only use the base classes implemented by backends (unless they need the
specific implementations in which case they just use the subclasses
directly). See this issue:

    https://gitlab.kitware.com/vtk/vtk/issues/17218

--Ben