[vtk-developers] [ANNOUNCE] New module system landing
Marcus D. Hanwell
marcus.hanwell at kitware.com
Mon Feb 4 11:04:16 EST 2019
I share your feelings on only wanting YES, and DONT_WANT. I think most
application developers just want a way of turning off everything by
default, and then specifically turning ON those modules they require, and
having the build system handle the rest. When odd dependencies creep in I
don't expect I want the kind of output you show to debug where dependency X
came from. Showing that output all of the time is probably a mistake (but
in some sense what is required, how minimal should configure be), but being
able to get at it when you need it is invaluable.
I think the other states are useful internally, for testing, to ensure
people don't add in unnecessary dependencies etc, but are not as useful
from an application developer's perspective.
My $0.42...
On Mon, Feb 4, 2019 at 10:31 AM Andras Lasso <lasso at queensu.ca> wrote:
> I just wanted to clarify that application developers only need YES and
> DONT_WANT states for VTK modules. If the other states are required
> internally in VTK (for example, for VTK automatic tests) then it's fine,
> but please try to keep the VTK's public CMake configuration API simple.
>
> > Right now it says "X requires disabled module Y". What's missing is
> information of the link between X and Y for figuring out how to fix it.
>
> Dependency graph printing feature of CTK fixes this. See an example below
> how external project dependency graph is printed during application
> configuration. @jcfr can give you more details on how this works and could
> be adapted to VTK modules.
>
> Andras
>
>
> External project dependency graph printed during CMake configuration of
> Slicer:
>
> -- SuperBuild - Slicer => Requires curl, CTKAppLauncherLib, teem, VTK,
> ITK, CTK, LibArchive, RapidJSON, SimpleITK, SlicerExecutionModel, qRestAPI,
> DCMTK, python-pydicom, python-packaging, python-chardet, python-couchdb,
> python-GitPython, python-pip, python-six, python-PyGithub, CTKAPPLAUNCHER,
> python, NUMPY, tbb[INCLUDED],
> -- SuperBuild - curl => Requires zlib, OpenSSL,
> -- SuperBuild - zlib[OK]
> -- SuperBuild - OpenSSL[OK]
> -- SuperBuild - curl[OK]
> -- SuperBuild - CTKAppLauncherLib[OK]
> -- SuperBuild - teem => Requires zlib[INCLUDED], VTK,
> -- SuperBuild - VTK => Requires zlib[INCLUDED], python, tbb[INCLUDED],
> -- SuperBuild - python => Requires bzip2, CTKAPPLAUNCHER,
> zlib[INCLUDED], OpenSSL[INCLUDED],
> -- SuperBuild - bzip2[OK]
> -- SuperBuild - CTKAPPLAUNCHER => Requires CTKResEdit,
> -- SuperBuild - CTKResEdit[OK]
> -- SuperBuild - CTKAPPLAUNCHER[OK]
> -- SuperBuild - python[OK]
> -- SuperBuild - VTK[OK]
> -- SuperBuild - teem[OK]
> -- SuperBuild - ITK => Requires zlib[INCLUDED], VTK[INCLUDED], DCMTK,
> -- SuperBuild - DCMTK => Requires zlib[INCLUDED],
> -- SuperBuild - DCMTK[OK]
> -- SuperBuild - ITK[OK]
> -- SuperBuild - CTK => Requires VTK[INCLUDED], ITK[INCLUDED],
> python[INCLUDED], DCMTK[INCLUDED],
> -- SuperBuild - CTK[OK]
> -- SuperBuild - LibArchive => Requires zlib[INCLUDED], zlib[INCLUDED],
> -- SuperBuild - LibArchive[OK]
> -- SuperBuild - RapidJSON[OK]
> -- SuperBuild - SimpleITK => Requires ITK[INCLUDED], Swig[INCLUDED],
> python[INCLUDED], python-setuptools,
> -- SuperBuild - python-setuptools => Requires python[INCLUDED],
> -- SuperBuild - python-setuptools[OK]
> -- SuperBuild - SimpleITK[OK]
> -- SuperBuild - SlicerExecutionModel => Requires ITK[INCLUDED], JsonCpp,
> ParameterSerializer,
> -- SuperBuild - JsonCpp[OK]
> -- SuperBuild - ParameterSerializer => Requires JsonCpp[INCLUDED],
> ITK[INCLUDED],
> -- SuperBuild - ParameterSerializer[OK]
> -- SuperBuild - SlicerExecutionModel[OK]
> -- SuperBuild - qRestAPI[OK]
> -- SuperBuild - python-pydicom => Requires python[INCLUDED],
> python-setuptools[INCLUDED],
> -- SuperBuild - python-pydicom[OK]
> -- SuperBuild - python-packaging => Requires python[INCLUDED],
> python-pyparsing,
> -- SuperBuild - python-pyparsing => Requires python[INCLUDED],
> -- SuperBuild - python-pyparsing[OK]
> -- SuperBuild - python-packaging[OK]
> -- SuperBuild - python-chardet => Requires python[INCLUDED],
> python-setuptools[INCLUDED],
> -- SuperBuild - python-chardet[OK]
> -- SuperBuild - python-couchdb => Requires python[INCLUDED],
> python-setuptools[INCLUDED],
> -- SuperBuild - python-couchdb[OK]
> -- SuperBuild - python-GitPython => Requires python[INCLUDED],
> python-setuptools[INCLUDED], python-gitdb,
> -- SuperBuild - python-gitdb => Requires python[INCLUDED],
> python-setuptools[INCLUDED], python-smmap,
> -- SuperBuild - python-smmap => Requires python[INCLUDED],
> python-setuptools[INCLUDED],
> -- SuperBuild - python-smmap[OK]
> -- SuperBuild - python-gitdb[OK]
> -- SuperBuild - python-GitPython[OK]
> -- SuperBuild - python-pip => Requires python[INCLUDED], python-wheel,
> -- SuperBuild - python-wheel => Requires python[INCLUDED],
> python-setuptools[INCLUDED],
> -- SuperBuild - python-wheel[OK]
> -- SuperBuild - python-pip[OK]
> -- SuperBuild - python-six => Requires python[INCLUDED],
> -- SuperBuild - python-six[OK]
> -- SuperBuild - python-PyGithub => Requires python[INCLUDED],
> python-setuptools[INCLUDED], python-pyjwt,
> -- SuperBuild - python-pyjwt => Requires python[INCLUDED],
> python-setuptools[INCLUDED],
> -- SuperBuild - python-pyjwt[OK]
> -- SuperBuild - python-PyGithub[OK]
> -- SuperBuild - NUMPY => Requires python[INCLUDED],
> python-setuptools[INCLUDED], python-nose,
> -- SuperBuild - python-nose => Requires python[INCLUDED],
> python-setuptools[INCLUDED],
> -- SuperBuild - python-nose[OK]
> -- SuperBuild - NUMPY[OK]
> -- SuperBuild - Slicer[OK]
> -- Configuring done
> -- Generating done
>
>
> -----Original Message-----
> From: Ben Boeckel <ben.boeckel at kitware.com>
> Sent: Monday, February 4, 2019 9:49 AM
> To: Andras Lasso <lasso at queensu.ca>
> Cc: David Thompson <david.thompson at kitware.com>; VTK Developers <
> vtk-developers at vtk.org>; David Gobbi <david.gobbi at gmail.com>
> Subject: Re: [vtk-developers] [ANNOUNCE] New module system landing
>
> On Sun, Feb 03, 2019 at 12:09:10 +0000, Andras Lasso wrote:
> > I've used two large CMake-based projects in the past that tried to do
> > something similar - guess which features to enable/disable based on a
> > number of require/prohibit CMake flags. Both worked very poorly,
> > because it always required lengthy experimentation with the flags to
> > get the list of features that we actually needed.
>
> With the VTK module system, you say `YES` to modules you require. That
> guarantees you the features you want. You may optionally say `NO` to
> modules that you don't want for some reason (e.g., MPI or Python) which
> guarantees that they do not build even via dependency resolution.
>
> > The main issue is that positive (require) and negative (prohibit)
> > requirements are obviously often contradict and it is not clear which
> > one is stronger, why exactly they conflict, and what are the options
> > to resolve the conflict. Weak require and weak prohibit further
> > complicate things with questionable benefits.
>
> Both are exactly as strong and the module system refuses to continue if
> the graph containing either cannot be made disjoint (mainly by disabling
> testing bits of modules). Having only strong YES/NO makes it easy to
> accidentally create a situation where I can't build anything without
> turning MPI on (currently a supported configuration) since I can't say
> "turn off MPI completely" if anything requires it since there's no "weak
> requires" semantics.
>
> > I would recommend to reconsider the introduction of this unusual and
> > complicated 5-state system and only have binary REQUIRE on/off flag
> > for each module. The build system would only include required modules
> > and their prerequisites (everything else would be disabled).
>
> This is what we had before and telling VTK to *not* build something (e.g.,
> I don't have an external dependency) was painful since it was only ever a
> weak disable.
>
> > If there
> > was any conflict of required modules then the build system would print
> > an error message explaining why the combination is not feasible and
> > the the developer would decide how to modify the requirements.
>
> Right now it says "X requires disabled module Y". What's missing is
> information of the link between X and Y for figuring out how to fix it.
>
> > Instead of VTK checking PROHIBITED modules, the external project that
> > uses VTK can inspect the list of modules and throw a fatal CMake error
> > if it finds that a prohibited module got enabled (it can display a
> > much more informative message than VTK could, explaining why that
> > module is not supported and what to do about it). ATTEMPT and AVOID
> > can be similarly handled in the external project by logging a warning
> > if the resulting module list is not optimal.
>
> Why would any project care that VTK has modules that it doesn't use? The
> only reason I can think of off-hand is a project not caring about such
> things would be if it is providing conflicting symbols (e.g., the same
> class) that VTK does in one of those modules. That's already fragile and
> broken. For object factory issues, this issue should resolve things like
> that:
>
>
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.kitware.com%2Fvtk%2Fvtk%2Fissues%2F17218&data=02%7C01%7Classo%40queensu.ca%7C40363f7095fe40e5ee1f08d68aaff357%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636848885639538130&sdata=fs6slxAXqlZDD79GDuxiqBKeEbz0Vic2EPQyEY6Mzj4%3D&reserved=0
>
> > During configuration, a dependency graph can be printed (already
> > available in CTK and used in most CTK-based projects for external
> > project dependencies) to show how a component brings in other
> > components. This helps the developer in deciding which modules to
> > disable to get rid of prohibited or undesired modules.
>
> Better debugging is on my list, but other tasks have higher priority right
> now (though I can review other MRs doing this).
>
> --Ben
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> https://vtk.org/mailman/listinfo/vtk-developers
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://vtk.org/pipermail/vtk-developers/attachments/20190204/198b88d1/attachment-0001.html>
More information about the vtk-developers
mailing list