[Insight-developers] IJ Volunteers : THE MATRIX : RED PILL

[Torsten Rohlfing]

Luis --

A couple of things to clarify:

Luis Ibanez wrote:
>         How do we (the ITK community) ensure the quality
>         of new code that goes into the toolkit ?
As we discussed by private email a few weeks ago, I completely agree - 
unchecked contributions will reduce code quality, and there's a 
trade-off between ensuring quality and maintaining the willingness of 
contributors to deal with the QA hurdles. The decision what is tolerable 
and what isn't is very personal and different for each developer. You 
will never get perfect quality control AND the maximum number of 
contributions. BUT it's important to keep this trade-off in mind when 
designing the submission system and tuning it's parameters.

Along these lines, I am wondering -- ITK is not the only large 
collaborative open source projects, and it's definitely not the only one 
faced with such issues. Does anyone have any experience/knowledge how 
other large projects (excluding the Linux kernel of course ;)) handle 
decisions about what to include in the official code base, and how that 
works out for them? Are there any that have a similarly open and 
democratic process with no designated authority implemented?
>    We were expecting that "community reviews" was a good answer to
>    this issue. The assumption was that from 1,200 (known) users
>    who enjoy using the toolkit for free, triplets of them will be
>    willing to review a paper from time to time. After all, they are
>    the ones that will use that contributed code later on.
>
>    Unfortunately, what we have learned so far is that it takes a
>    lot of energy to motivate users to participate of the peer-review
>    process. From the social point of view, this is has hard as
>    getting people out of their houses and bringing them to vote
>    on election day.
>
... and we all know how badly that lack of motivation can play out in 
the end ;)

Seriously, I don't know what the solution here is, but I have a few 
thoughts:

a) Someone mentioned feature requests as a possible criterion for code 
inclusion. I would think that if someone requested a certain feature 
through mailing list or bugtracker, that person would be a natural (and, 
presumably, interested) candidate to provide a review of such function 
one it is submitted to the IJ.

b) How about earmarking developers who submit contributions in certain 
fields to review contributions in the same field? Maybe they could be 
sent emails to the effect that a paper related to their own 
contributions was submitted, and they might want to have a look at it 
(without any commitment, just so no one feels pressured). Having an 
option in the account preferences to opt out of such notification would 
of course be a nice touch.

I think I am forgetting some other ideas I had, but I'll just send them 
when I get back on caffeine tomorrow morning.
> 2) I may have to disagree with the statement that writing tests and
>    documentation is "different" from writing code. Source code that
>    has not been tested and documented is not "complete" source code,
>    and will cost many times its development price in maintenance
>    headaches and Hara-Kiris of future developers and users. It is a
>    well known fact in software engineering that 80% of the cost of
>    software development goes in maintenance, after the code has been
>    declared to be "completed", and most of this maintenance cost is
>    spent by "other" developers reading the code and trying to
>    understand what the code does, even before they can retouch it.
>
>    The simple rule is :  If it is not tested, it does not work.
>                          If it not documented, it will be misused.
If you read my mail carefully, you will see that I agree completely - 
tests and documentation are an absolute necessity, and in fact I am 
personally employing both even in my private code base, which no one 
other than me really uses. What I do maintain, though, is that tests and 
documentation are a pain in the neck for most programmers (it's delayed 
gratification after all). As such, it should be taken into consideration 
when creating additional hurdles for the code submission process (I 
didn't say this before, it just came to me), because it probably reduces 
the willingness to comply with additional "bureaucratic" requirements.
> 3) They IJ has some contributions that are unrelated to ITK and
>    it will be desirable to create a separate section only for ITK.
>
>    I agree with this suggestion. This will also make easier for
>    other toolkits such as VTK and IGSTK to use the IJ as community
>    resource.
Exactly. And if it turns out that papers in the "ITK code contrib" 
section get more attention than others, maybe that would be a little 
extra motivation for people to make _all_ their code available?

Best,
  Torsten

-- 
Torsten Rohlfing, PhD          SRI International, Neuroscience Program
 Research Scientist             333 Ravenswood Ave, Menlo Park, CA 94025
  Phone: ++1 (650) 859-3379      Fax: ++1 (650) 859-2743
   torsten at synapse.sri.com        http://www.stanford.edu/~rohlfing/

     "Though this be madness, yet there is a method in't"

-------------- next part --------------
A non-text attachment was scrubbed...
Name: torsten.vcf
Type: text/x-vcard
Size: 366 bytes
Desc: not available
Url : http://www.itk.org/mailman/private/insight-developers/attachments/20060917/67a2eaa3/torsten.vcf