[Insight-developers] Ramblin' on: Another concern regarding IJ and ITK code contributions

[Zachary Pincus]

As a data point, the procedure Luis outlined below more or less  
describes (a) how I found that Gaëtan was working on updated  
wrappers, and (b) how we started working on that together. It worked  
out really well, and ended up with an IJ submission (and integration  
of the code into ITK -- thanks everyone for their hard work on that!).

Perhaps the IJ is just exposing the duplication of work that's  
already going on and not promoting it explicitly... In that case,  
hopefully the IJ will lead to less such duplication in the future  
when people can see exactly what others are working on.

Zach


On Sep 21, 2006, at 1:53 PM, Luis Ibanez wrote:

>
> Hi Torsten,
>
> That's an interesting point.
>
> However, note that the addition of the IJ has not eliminated the
> availability of all other community resources. In particular,
> the mailing lists and the Wiki.
>
>
> In other words, the IJ is not a replacement for the mailing lists
> and the Wiki. Instead, it is intended to be used along with them.
>
> ---
>
> In the scenario of DTI, the way these different institutions became
> aware of each other efforts was by posting emails to the developers
> list. This still remains an opportunity for initiating collaborations,
> and presumably this sort of contact will happen very early in the
> process.
>
>
>
> Here is an "ideal" timeline for a contribution to ITK
>
>
> 1) User/Developer "X" identifies a need for an algorithm "A"
>
> 2) She/He posts an email to the developers list asking if "A"
>    or something similar to "A" exists in ITK.
>
> 3) Other developers/users will answer saying that "A" does not
>    exist, but it will be great to have it.
>
> 4) User/Developer "X" may answer saying that because she/he needs it,
>    she/he will be willing to implement it and contribute it to ITK.
>    At that point she/he should ask if somebody else is interested
>    in such effort.
>
> 5) Users/Developers "Y" and "Z" may replay saying that they are
>    interested in collaborating.
>
> 6) "X","Y" and "Z" will go on exchanging emails among them and
>    with the developers list brainstorming on how to implement "A".
>
> 7) "X", "Y" and "Z" will then use the Wiki proposals page:
>
>      http://www.itk.org/Wiki/ITK_Oversight_Committee#Proposals
>
>    (which is open in write access to anybody) for posting a detailed
>    plan on how they would like to implement "A".
>    (e.g. look at the details in the Explicit Instantiation proposal:
>    http://www.itk.org/Wiki/Proposals:Explicit_Instantiation)
>
> 8) "X", "Y" and "Z" develop a working version of "A", and together,
>    (as coauthors) they write a paper to the Insight Journal.
>
> 9) The IJ submission system automatically post email to the users
>    and developers list notifying of the new submission.
>
> 10) Users in the list, take a look at the submission, download the
>     source code from the IJ and give it a try.
>
> 11) These users *write reviews* back to the paper in the IJ, and
>     in the process they *rate* the paper.
>
> 12) When it comes the time for preparing the following release of
>     ITK, the developers in charge of the release go to the IJ and
>     look at the paper with higher ratings, take their code and
>     commit it to CVS in the (Insight/Code/Review) directory.
>
> 13) The release is cut, and these filter are distributes as
>     "candidates" to be added to ITK. The code still in the
>     (Insight/Code/Review) directory.
>
> 14) During the three months of the release life cycle, this code
>     is hardened, better tested and maybe even refactored, until
>     it is mature enough for the developers to commit themselves
>     to maintain it in its current form for the next 10 years.
>
> 15) At the next release cycle the mature code is moved to its
>     definite place inside ITK (e.g. Code/Algorithms....)
>
> 16) The codes continue growing and evolving for years, but under
>     the constrains given by the backward compatibility policy.
>
>
>
>
> To summarize, I agree with you that it is not after a paper is
> published to the IJ that we would expect collaborations to flourish.
>
>
> Instead, we would anticipate that collaborations are initiated by
> exchanges in the developers mailing list, and that the results of
> such collaborations will be crystallized in a joint submission to
> the Insight Journal.
>
> In other words, by the time a paper is submitted to the IJ,
> I would assume that we have heard weeks, maybe months, in advance
> from the authors.  This has been indeed the case for WrapITK,
> and for the most recent itkQMesh paper.
>
>
>
> Just for the record, note that what makes this pre-submission dialog
> possible is the fact that the IJ is not interested in "original"
> contributions, as most decadent Journals and Conferences are.
>
>
> Instead, the IJ is interested in contributions that *actually work*.
>
>
> Again, the point is that scientific research is not at all about
> "originality". Scientific research is about "reproducibility".
>
>
> We shouldn't care about the ideas being new or old, we should care
> about whether the ideas are confirmed in practical experiments or not.
>
>
> Anyone who has read a book about epistemology will know that
> "originality" of ideas is irrelevant in the quest for knowledge.
>
> What matter is whether the ideas are consistent with the physical
> reality or not. Aristotle was *very original* but he turned out
> to be *wrong* in almost all his ideas in physics and biology.
>
> Aristitle's image of authority stagnated scientific development for
> 1,800 years!, until Galileo decided that he actually wanted to test
> whether Aristotle's ideas hold true in practical experiments.
>
>
> The notion that papers must be "original" is a result of having
> under-educated managers in charge of evaluating scientific work.
>
>
> Any Journal that "call-for-original-papers" *is not* a scientific
> journal. Any Journal that *do not* publish "reproduction" of work
> made by others *is not a scientific journal*.
>
>
> Such "originality" policies are today one of the strongest obstacles
> for the open exchange of ideas in the scientific community, and
> for the establishment of inter and intra institution collaborations.
>
>
> You will hear colleagues saying that they may not want to talk about
> the details of their "unpublished" work, for fear that others may
> steal their ideas and publish them first.
>
>
> This happens when researchers get confused and believe that they
> are "inventors", which is actually a quite different profession.
>
>
>
>
>     Regards,
>
>
>
>        Luis
>
>
>
> ------------------------
> Torsten Rohlfing wrote:
>> Greetings everyone --
>> I would like to add to the general discussion another issue that I  
>> see in the current model of ITK code contributions through the IJ.  
>> But to not create any unwarranted suspense, let me say one thing  
>> beforehand: I have, again, no idea what a possible solution might  
>> look like, and maybe I am the only one who even thinks that this  
>> is a problem in the first place.
>> Anyway, here we go: in my opinion, channeling code contributions  
>> to ITK through the IJ makes it very hard to create a cross- 
>> institutional collaborative effort for larger ITK extensions and  
>> for the implementation of new concepts.
>> Here's an example from my personal experience: the code and data  
>> structures for diffusion tensor images came out of a community  
>> discussion and at least three sample implementations (by Jeffrey  
>> Duda at Penn, Martin Styner at UNC, and myself). Before this  
>> effort, I didn't know Jeff, and I had no idea Martin was  
>> interested in DTI. So in the IJ submission model, each of us might  
>> have submitted their own implementation, with their own data  
>> structures, and some application code around them. Assuming that  
>> all three implementations would have attracted sufficiently many  
>> favorable reviews, it is unclear to me, how from there one would  
>> have gotten to a single "official" implementation (and when that  
>> might've happened).
>> A more recent example: I noticed two interesting contributions in  
>> the IJ -- a "general" image fusion framework ("GIFT"), and a  
>> contribution to create overlays of label maps onto corresponding  
>> intensity images. It occurred to me that it would probably make  
>> sense to unite these two in a common framework for image fusion  
>> (or, to quote a Monty Pythons sketch: "for putting things on top  
>> of other things").
>> It also occurred to me that there are probably quite a few people  
>> out there who'd be interested in such a general image fusion  
>> framework, only they're scattered across many different  
>> institutions and would probably not find each other easily. So in  
>> the best case, each of them will write an IJ submission with  
>> somewhat incompatible and somewhat redundant code. In the worst  
>> case, many of these folks will find the "entry barrier" of getting  
>> everything coded and documented and polished and written up for an  
>> IJ submission to high, and only a fraction of the possible  
>> contributions will actually happen. And then these would probably  
>> still be incompatible and redundant.
>> So that's basically my criticism and my question - it seems that  
>> the previously practiced model of ITK development would have  
>> accommodated this situation quite nicely, and in my own experience  
>> it did so for the DTI code. I am wondering if there are any ideas  
>> how one could maintain the undoubted benefits of the IJ submission  
>> model (thorough review; stable interfaces) could be extended to  
>> encourage, or at least enable, dynamic forming of cross- 
>> institutional development efforts that are beyond the capabilities  
>> of any one group alone?
>> Cheers!
>>  Torsten
>> _______________________________________________
>> Insight-developers mailing list
>> Insight-developers at itk.org
>> http://www.itk.org/mailman/listinfo/insight-developers
>
>
>
>
> _______________________________________________
> Insight-developers mailing list
> Insight-developers at itk.org
> http://www.itk.org/mailman/listinfo/insight-developers