[vtk-developers] Driving away your existing developers

Thu Aug 28 10:44:58 EDT 2014

> I agree with Steve that vtk proper should get smaller.

I agree with the spirit of this. In fact, I get very exciting thinking
about how this would work and how nice and
useful the infrastructure would be. However, I don't think it is very
feasible to use such a mechanism to
pull out parts of current VTK. It is a maintenance issue. We have had years
of experience of maintaining
tightly coupled projects that are in separate repositories (VTK and
ParaView). And it is not easy. It requires
team dedicated to maintaining each separately, with separate dashboard
architecture, git branches, code
review processes etc. If we had, say, 10 small projects that tightly
depended on VTK and was developed
by roughly the same team, there would be a significant overhead to
development and maintenance. What
would the development workflow be to keep these in sync? Having the
"remote" modules depend on a release
version of VTK is not tightly coupled enough and would hamper development
significantly. So we would
have to manage some version dependency mechanism that would be at best hard.

I can see making a handful of modules that hardly change external. Or some
modules that change more
regularly but depend less on changes to core features. Some of the IO
modules with external dependencies,
maybe VTK Web, some filtering module such as Matlab, R, Reebgraph etc. But
is it really worth the trouble?
It would shrink VTK slightly but would make testing, integration etc.
harder for everyone.

I would rather focus on implementing this for new projects / modules.
Specially those that are developed
in a way that's less coupled with VTK. Some Slicer code could be organized
this way for example. Slicer
depends on a release version of VTK and has its own testing infrastructure
so it should be easy. Things
like ITK-VTK adapters could probably be organized the same way.

What would the mechanism for indicating version dependencies, whether a
remote module is tested
regularly and things like that?

Best,
-berk

On Wed, Aug 27, 2014 at 8:58 PM, Bill Lorensen <bill.lorensen at gmail.com>
wrote:

> ITK's  remote module mechanism is a great way to add domain specific
> modules. I agree with Steve that vtk proper should get smaller.
>  On Aug 27, 2014 5:04 PM, "Steve Pieper" <pieper at isomics.com> wrote:
>
>> I would love to see more of your excellent work, David, and I'd want to
>> make use of it, but I'd argue that at least some of it, perhaps
>> registration, may not belong in VTK itself.
>>
>> Speaking for myself from my experience in the slicer project, I can say
>> that we end up putting a lot of our functionality in classes that don't
>> belong back into VTK proper, such as vtkITK, vtkTeem, and vtkMRML, each of
>> which adds functionality that is independent of our slicer application, but
>> adds a layer of dependencies that nobody would want to see in VTK itself.
>>
>> On my wishlist for VTK would be a better way for new developers to
>> contribute their work so that it's easily usable by others but doesn't have
>> to pass the gauntlet of reviews and comments and be in VTK before it is
>> easily usable by others.  I'd like to see an experience more like npm for
>> node, where anyone can publish a domain-specific set of code and other
>> people can use it or not as they see fit.  Node itself remains small and it
>> rarely changes, but the ecosystem is quite vibrant.  It allows a lot of
>> diversity, but amazing interoperability and reusability.
>>
>> In slicer we have something similar now with the Extension Catalog, where
>> developers can offer code (either pure VTK or things that work at the
>> VTK/ITK/Qt/Slicer level) that is optionally installed.  Since we perform
>> nightly builds and test it on all platforms, this code is in reasonable
>> shape for other people to draw from, either in slicer or elsewhere.
>>  Typically these extensions are in github repositories, so they have their
>> own wikis, issue lists, pull requests, etc.  We hope that slicer itself
>> actually gets smaller over time and generally doesn't need to change very
>> much or very often.
>>
>> I would love to see something similar in VTK proper.  Right now VTK has a
>> lot of modules that are quite domain specific, such as Chemistry and Geovis
>> which I do not envision using myself.  I would have preferred to see that
>> code exist in a different repository where people who needed it would have
>> the option of including it in their superbuild process.  Others could
>> safely ignore it.  I would even argue that half (or more) of the current
>> modules don't really belong in the 'actual' VTK and should be moved out.
>>  If there were a good place to put some of this kind of code it might also
>> be a good spot for libraries like vtkITK and vtkTeem.  In fact I would
>> argue that if it weren't actually in VTK's repository, a project like
>> vtkChemistry might attract more participants and have more features (but I
>> can't prove that of course).
>>
>> So basically I'd say that VTK is currently too big and should get
>> smaller, while the community is too small and should get bigger.
>>
>> -Steve
>>
>>
>> On Wed, Aug 27, 2014 at 12:00 PM, David Gobbi <david.gobbi at gmail.com>
>> wrote:
>>
>>> On Wed, Aug 27, 2014 at 7:10 AM, Berk Geveci <berk.geveci at kitware.com>
>>> wrote:
>>>
>>> > One final note. Everyone please keep in mind that while there are
>>> quite a
>>> > lot of contributors to VTK, certain parts of it are developed by only
>>> a few
>>> > people. Core pipeline: I am it at this point. Rendering: by next year,
>>> it
>>> > will be Ken and Marcus. Imaging: hardly anyone in the scientific vis
>>> team at
>>> > Kitware does anything with imaging filters. We update them when we make
>>> > changes to the pipeline or data model but that's it. So some of those
>>> are
>>> > maintained and developed by others or they are some orphaned. This can
>>> only
>>> > be addressed by either encouraging new (or existing) developers to take
>>> > ownership or by removing those components all together.
>>>
>>> I have tons of plans for the VTK imaging classes, but unfortunately I
>>> have
>>> almost no resources so progress is slow.  Some examples:
>>>
>>> 1) Improving both the API and the performance for multi-threading.
>>> Last year I worked with an intern to improve load sharing.  Never had
>>> time to finish, because it wasn't the intern's main project.
>>>
>>> 2) Basic linear image registration in VTK.  I've finished it and it's
>>> very robust (been using it for years), but refactoring so that it can
>>> be contributed to VTK is an evening/weekend project and hence is
>>> moving slowly.
>>>
>>> 3) Better image iterators.  I have a nice image iterator that iterates
>>> over arbitrary shapes and also provides coordinate info while it
>>> iterates.  I'd love to develop this further and contribute it, and I'd
>>> especially love to wrap it, but it's out in left field as far as my
>>> day job goes.
>>>
>>> 4) Better filters for going back and forth between image data and
>>> polydata.  Lots of people use my vtkPolyDataToImageStencil filter,
>>> but it has faults and I know exactly how to fix them, but I'll have to
>>> wait until it comes up as a priority for one of my work projects.
>>>
>>> 5) Finally, a better overall structure for the imaging pipeline,
>>> focused on the basics:
>>>  a) scalar operation "functors" that can be plugged into filters
>>>  b) interpolators, which I've already committed
>>>  c) good image "region" descriptors (like my vtkImageStencilData)
>>>  d) connectivity & morphological operations, I've got lots of
>>> un-committed stuff
>>>  e) a replacement for the lousy vtkImageMathematics filter
>>>
>>> Ideally I should write out a big roadmap (the items mentioned above
>>> is just a fraction of the total), but it would be depressing because so
>>> much of it feels like it's nothing more than a pipe dream.
>>>
>>> And, as Berk mentioned, there isn't anyone in Kitware for me to
>>> directly work with on this stuff.  And all the other VTK imaging
>>> developers outside of Kitware are content to keep their work to
>>> themselves, without contributing back to the VTK core.  Or at least,
>>> that's the way it looks from my vantage point.
>>>
>>>  - David
>>> _______________________________________________
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>>
>>>
>>>
>>> The information in this e-mail is intended only for the person to whom
>>> it is
>>> addressed. If you believe this e-mail was sent to you in error and the
>>> e-mail
>>> contains patient information, please contact the Partners Compliance
>>> HelpLine at
>>> http://www.partners.org/complianceline . If the e-mail was sent to you
>>> in error
>>> but does not contain patient information, please contact the sender and
>>> properly
>>> dispose of the e-mail.
>>>
>>>
>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20140828/bbc4590a/attachment-0002.html>