[vtk-developers] New module system preview

[Ben Boeckel]

On Mon, Oct 29, 2018 at 14:23:27 -0400, David Thompson wrote:
> >>> - Instead of `module.cmake`, there are `vtk.module` and `vtk.kit`
> >>>   files. These are basically CMake argument lists, but no variable
> >>>   expansion is allowed. If there are optional dependencies, they must
> >>>   be private dependencies. Optional public dependencies indicate that
> >>>   a new module should be made instead.
> >> 
> >> Is there a reason these things need to be separate files
> >> (vtk.module/vtk.kit) at all?
> > 
> > The `vtk.kit` declares a kit. Membership into a kit is via `vtk.module`.
> > Example `vtk.kit`:
> 
> Yes, but why can't these declarations for modules and kits be inside
> CMakeLists.txt as function calls instead of separate files?

There's still a two-pass build pipeline. First is to get the declaration
of the modules and build up the dependency graph. There is then a build
step which builds modules in dependency order. Unlike the old module
system, these steps are now distinct:

```cmake
vtk_module_find_modules(vtk_module_files ${vtk_source_directories})
vtk_module_find_kits(vtk_kit_files ${vtk_source_directories})
vtk_module_scan(
  MODULE_FILES        ${vtk_module_files}
  KIT_FILES           ${vtk_kit_files}
  REQUEST_MODULES     ${vtk_requested_modules}
  REJECT_MODULES      ${vtk_rejected_modules}
  PROVIDES_MODULES    vtk_modules
  PROVIDES_KITS       vtk_kits
  REQUIRES_MODULES    vtk_required_modules
  UNRECOGNIZED_MODULES vtk_unrecognized_modules
  WANT_BY_DEFAULT     ON
  ENABLE_TESTS        DEFAULT)

# Check to see if any modules were mentioned that we don't know how to
# build.
if (vtk_required_modules OR vtk_unrecognized_modules)
  message(FATAL_ERROR
    "The following modules were requested or required, but not found: "
    "${vtk_required_modules};${vtk_unrecognized_modules}.")
endif ()

vtk_module_build(
  MODULES             ${vtk_modules}
  KITS                ${vtk_kits}
  BUILD_WITH_KITS     ${VTK_ENABLE_KITS}
  INSTALL_EXPORT      VTK
  HEADERS_DESTINATION "include/vtk-${VTK_MAJOR_VERSION}.${VTK_MINOR_VERSION}"
  CMAKE_DESTINATION   "lib/cmake/vtk-${VTK_MAJOR_VERSION}.${VTK_MINOR_VERSION}"
  LIBRARY_NAME_SUFFIX "${VTK_MAJOR_VERSION}.${VTK_MINOR_VERSION}"
  VERSION             "${VTK_VERSION}"
  SOVERSION           "1"
  TEST_DATA_TARGET    VTKData
  TEST_INPUT_DATA_DIRECTORY   "${CMAKE_CURRENT_SOURCE_DIR}/Testing"
  TEST_OUTPUT_DATA_DIRECTORY  "${CMAKE_CURRENT_BINARY_DIR}/ExternalData/Testing")
```

The `scan` step processes dependencies found in the specified module and
kit files. This is also where we find out what modules we're actually
going to build. Once we have the list, we can then build them (assuming
that all mentioned modules actually exist). The reason for the two-phase
is to support embedding module-using projects (e.g., VTK in ParaView).

ParaView works something like this:

```cmake
vtk_module_find_modules(pv_module_files ${pv_source_directories})
vtk_module_find_kits(pv_kit_files ${pv_source_directories})
vtk_module_scan(
  MODULE_FILES        ${pv_module_files}
  KIT_FILES           ${pv_kit_files}
  REQUEST_MODULES     ${pv_requested_modules}
  REJECT_MODULES      ${pv_rejected_modules}
  PROVIDES_MODULES    pv_modules
  PROVIDES_KITS       pv_kits
  REQUIRES_MODULES    pv_required_modules
  UNRECOGNIZED_MODULES pv_unrecognized_modules
  ENABLE_TESTS        DEFAULT)

# Now build VTK modules required by the enabled ParaView modules.
# Alternatively, using an external VTK is now just crafting the
# appropriate `find_package(VTK)` component list.
vtk_module_find_modules(vtk_module_files ${vtk_source_directories})
vtk_module_find_kits(vtk_kit_files ${vtk_source_directories})
vtk_module_scan(
  MODULE_FILES        ${vtk_module_files}
  KIT_FILES           ${vtk_kit_files}
  REQUEST_MODULES     ${pv_required_modules}
                      ${pv_unrecognized_modules}
  PROVIDES_MODULES    vtk_modules
  PROVIDES_KITS       vtk_kits
  REQUIRES_MODULES    vtk_required_modules
  UNRECOGNIZED_MODULES vtk_unrecognized_modules
  WANT_BY_DEFAULT     OFF
  HIDE_MODULES_FROM_CACHE ON
  ENABLE_TESTS        DEFAULT)

# Check to see if any modules were mentioned that we don't know how to
# build.
if (vtk_required_modules OR vtk_unrecognized_modules)
  message(FATAL_ERROR
    "The following modules were requested or required, but not found: "
    "${vtk_required_modules};${vtk_unrecognized_modules}.")
endif ()

# Build VTK modules.
vtk_module_build(
  MODULES             ${vtk_modules}
  KITS                ${vtk_kits}
  BUILD_WITH_KITS     ${PARAVIEW_ENABLE_KITS}
  INSTALL_EXPORT      VTK
  HEADERS_DESTINATION "include/paraview-${PARAVIEW_MAJOR_VERSION}.${PARAVIEW_MINOR_VERSION}"
  CMAKE_DESTINATION   "lib/cmake/paraview-${PARAVIEW_MAJOR_VERSION}.${PARAVIEW_MINOR_VERSION}"
  LIBRARY_NAME_SUFFIX "-pv${PARAVIEW_MAJOR_VERSION}.${PARAVIEW_MINOR_VERSION}"
  VERSION             "${PARAVIEW_VERSION}"
  SOVERSION           "1"
  TEST_DATA_TARGET    ParaViewData
  TEST_INPUT_DATA_DIRECTORY   "${CMAKE_CURRENT_SOURCE_DIR}/VTK/Testing"
  TEST_OUTPUT_DATA_DIRECTORY  "${CMAKE_CURRENT_BINARY_DIR}/ExternalData/VTK/Testing")

# Build ParaView modules.
vtk_module_build(
  MODULES             ${pv_modules}
  KITS                ${pv_kits}
  BUILD_WITH_KITS     ${PARAVIEW_ENABLE_KITS}
  INSTALL_EXPORT      ParaView
  HEADERS_DESTINATION "include/paraview-${PARAVIEW_MAJOR_VERSION}.${PARAVIEW_MINOR_VERSION}"
  CMAKE_DESTINATION   "lib/cmake/paraview-${PARAVIEW_MAJOR_VERSION}.${PARAVIEW_MINOR_VERSION}"
  LIBRARY_NAME_SUFFIX "-pv${PARAVIEW_MAJOR_VERSION}.${PARAVIEW_MINOR_VERSION}"
  VERSION             "${PARAVIEW_VERSION}"
  SOVERSION           "1"
  TEST_DATA_TARGET    ParaViewData
  TEST_INPUT_DATA_DIRECTORY   "${CMAKE_CURRENT_SOURCE_DIR}/Testing"
  TEST_OUTPUT_DATA_DIRECTORY  "${CMAKE_CURRENT_BINARY_DIR}/ExternalData/Testing")
```

The no-extra-files approach is what the superbuild has. All code in
`CMakeLists.txt` now needs to handle being called twice which breaks
things like `cmake_dependent_option` and also means that
`configure_file` should be guarded by the appropriate phase. However,
the superbuild has a *much* simpler API and is smaller to boot.

--Ben