[Insight-developers] Ramblin' on: Another concern regarding IJ and ITK code contributions

[Luis Ibanez]

Hi Torsten,

That's an interesting point.

However, note that the addition of the IJ has not eliminated the
availability of all other community resources. In particular,
the mailing lists and the Wiki.


In other words, the IJ is not a replacement for the mailing lists
and the Wiki. Instead, it is intended to be used along with them.

---

In the scenario of DTI, the way these different institutions became
aware of each other efforts was by posting emails to the developers
list. This still remains an opportunity for initiating collaborations,
and presumably this sort of contact will happen very early in the
process.



Here is an "ideal" timeline for a contribution to ITK


1) User/Developer "X" identifies a need for an algorithm "A"

2) She/He posts an email to the developers list asking if "A"
    or something similar to "A" exists in ITK.

3) Other developers/users will answer saying that "A" does not
    exist, but it will be great to have it.

4) User/Developer "X" may answer saying that because she/he needs it,
    she/he will be willing to implement it and contribute it to ITK.
    At that point she/he should ask if somebody else is interested
    in such effort.

5) Users/Developers "Y" and "Z" may replay saying that they are
    interested in collaborating.

6) "X","Y" and "Z" will go on exchanging emails among them and
    with the developers list brainstorming on how to implement "A".

7) "X", "Y" and "Z" will then use the Wiki proposals page:

      http://www.itk.org/Wiki/ITK_Oversight_Committee#Proposals

    (which is open in write access to anybody) for posting a detailed
    plan on how they would like to implement "A".
    (e.g. look at the details in the Explicit Instantiation proposal:
    http://www.itk.org/Wiki/Proposals:Explicit_Instantiation)

8) "X", "Y" and "Z" develop a working version of "A", and together,
    (as coauthors) they write a paper to the Insight Journal.

9) The IJ submission system automatically post email to the users
    and developers list notifying of the new submission.

10) Users in the list, take a look at the submission, download the
     source code from the IJ and give it a try.

11) These users *write reviews* back to the paper in the IJ, and
     in the process they *rate* the paper.

12) When it comes the time for preparing the following release of
     ITK, the developers in charge of the release go to the IJ and
     look at the paper with higher ratings, take their code and
     commit it to CVS in the (Insight/Code/Review) directory.

13) The release is cut, and these filter are distributes as
     "candidates" to be added to ITK. The code still in the
     (Insight/Code/Review) directory.

14) During the three months of the release life cycle, this code
     is hardened, better tested and maybe even refactored, until
     it is mature enough for the developers to commit themselves
     to maintain it in its current form for the next 10 years.

15) At the next release cycle the mature code is moved to its
     definite place inside ITK (e.g. Code/Algorithms....)

16) The codes continue growing and evolving for years, but under
     the constrains given by the backward compatibility policy.




To summarize, I agree with you that it is not after a paper is
published to the IJ that we would expect collaborations to flourish.


Instead, we would anticipate that collaborations are initiated by
exchanges in the developers mailing list, and that the results of
such collaborations will be crystallized in a joint submission to
the Insight Journal.

In other words, by the time a paper is submitted to the IJ,
I would assume that we have heard weeks, maybe months, in advance
from the authors.  This has been indeed the case for WrapITK,
and for the most recent itkQMesh paper.



Just for the record, note that what makes this pre-submission dialog
possible is the fact that the IJ is not interested in "original"
contributions, as most decadent Journals and Conferences are.


Instead, the IJ is interested in contributions that *actually work*.


Again, the point is that scientific research is not at all about
"originality". Scientific research is about "reproducibility".


We shouldn't care about the ideas being new or old, we should care
about whether the ideas are confirmed in practical experiments or not.


Anyone who has read a book about epistemology will know that
"originality" of ideas is irrelevant in the quest for knowledge.

What matter is whether the ideas are consistent with the physical
reality or not. Aristotle was *very original* but he turned out
to be *wrong* in almost all his ideas in physics and biology.

Aristitle's image of authority stagnated scientific development for
1,800 years!, until Galileo decided that he actually wanted to test
whether Aristotle's ideas hold true in practical experiments.


The notion that papers must be "original" is a result of having
under-educated managers in charge of evaluating scientific work.


Any Journal that "call-for-original-papers" *is not* a scientific
journal. Any Journal that *do not* publish "reproduction" of work
made by others *is not a scientific journal*.


Such "originality" policies are today one of the strongest obstacles
for the open exchange of ideas in the scientific community, and
for the establishment of inter and intra institution collaborations.


You will hear colleagues saying that they may not want to talk about
the details of their "unpublished" work, for fear that others may
steal their ideas and publish them first.


This happens when researchers get confused and believe that they
are "inventors", which is actually a quite different profession.




     Regards,



        Luis



------------------------
Torsten Rohlfing wrote:
> 
> Greetings everyone --
> 
> I would like to add to the general discussion another issue that I see 
> in the current model of ITK code contributions through the IJ. But to 
> not create any unwarranted suspense, let me say one thing beforehand: I 
> have, again, no idea what a possible solution might look like, and maybe 
> I am the only one who even thinks that this is a problem in the first 
> place.
> 
> Anyway, here we go: in my opinion, channeling code contributions to ITK 
> through the IJ makes it very hard to create a cross-institutional 
> collaborative effort for larger ITK extensions and for the 
> implementation of new concepts.
> 
> Here's an example from my personal experience: the code and data 
> structures for diffusion tensor images came out of a community 
> discussion and at least three sample implementations (by Jeffrey Duda at 
> Penn, Martin Styner at UNC, and myself). Before this effort, I didn't 
> know Jeff, and I had no idea Martin was interested in DTI. So in the IJ 
> submission model, each of us might have submitted their own 
> implementation, with their own data structures, and some application 
> code around them. Assuming that all three implementations would have 
> attracted sufficiently many favorable reviews, it is unclear to me, how 
> from there one would have gotten to a single "official" implementation 
> (and when that might've happened).
> 
> A more recent example: I noticed two interesting contributions in the IJ 
> -- a "general" image fusion framework ("GIFT"), and a contribution to 
> create overlays of label maps onto corresponding intensity images. It 
> occurred to me that it would probably make sense to unite these two in a 
> common framework for image fusion (or, to quote a Monty Pythons sketch: 
> "for putting things on top of other things").
> 
> It also occurred to me that there are probably quite a few people out 
> there who'd be interested in such a general image fusion framework, only 
> they're scattered across many different institutions and would probably 
> not find each other easily. So in the best case, each of them will write 
> an IJ submission with somewhat incompatible and somewhat redundant code. 
> In the worst case, many of these folks will find the "entry barrier" of 
> getting everything coded and documented and polished and written up for 
> an IJ submission to high, and only a fraction of the possible 
> contributions will actually happen. And then these would probably still 
> be incompatible and redundant.
> 
> So that's basically my criticism and my question - it seems that the 
> previously practiced model of ITK development would have accommodated 
> this situation quite nicely, and in my own experience it did so for the 
> DTI code. I am wondering if there are any ideas how one could maintain 
> the undoubted benefits of the IJ submission model (thorough review; 
> stable interfaces) could be extended to encourage, or at least enable, 
> dynamic forming of cross-institutional development efforts that are 
> beyond the capabilities of any one group alone?
> 
> Cheers!
>  Torsten
> 
> _______________________________________________
> Insight-developers mailing list
> Insight-developers at itk.org
> http://www.itk.org/mailman/listinfo/insight-developers