[Insight-developers] IJ Volunteers : THE MATRIX : RED PILL
Zachary Pincus
zpincus at stanford.edu
Sat Sep 16 22:39:06 EDT 2006
I don't think anyone is suggesting that there is a small circle of
"all-powerful developers". However, there is a circle of developers,
who, in an admirably open fashion, minister to ITK. This circle is
overseen by the ITK Oversight Committee ( http://www.itk.org/Wiki/
ITK_Oversight_Committee ).
It was some subset of these developers and the oversight committee
who decided upon (again in an open fashion) the precise algorithm for
transferring work from the IJ to ITK, and the exact parameter
settings for that algorithm. (Of which there are many! What's the
threshold of review-goodness for accepting a work? How many reviews
are needed? Are works from old "editions" of the IJ re-visited?)
Thus, when Gaëtan suggested a change to some of the algorithm
settings, it is reasonable and natural that that suggestion be sent
to the itk-dev list, where, by and large, the people who decided upon
the algorithm and settings (in an open and democratic fashion) and
may continue to refine them, can be reached. Before today's emails, I
would have presumed that the open nature of ITK would encourage
suggestions of this sort.
Again, to review:
There is a problem in getting work from the IJ to ITK via the
algorithm sketched below. This can be addressed by:
(a) addressing at the algorithm. In which case, addressing the people
involved in setting and changing that algorithm is appropriate.
(b) addressing the IJ. In which case, "complaining" to the itk-dev
list is not appropriate, but volunteering to do reviews, to be an
associate editor, and many other such things *is* appropriate.
Gaëtan and I think that perhaps (a) *and* (b) need to be addressed.
And I find it wholly inappropriate to castigate him, repeatedly and
publicly, about failing to do the right things, regarding (b), when
in fact his concern was regarding (a) to begin with! (And not to
mention that few people have been doing more, (b)-wise, than he.)
Please do not conflate issues (a) and (b). I'll chalk this
misunderstanding up to the nature of email and language barriers.
Let me make some suggestions, regarding these separate issues:
Regarding (a):
- Perhaps three reviews is too many to require? A single review from
a highly-trusted source might be enough in some cases.
- Perhaps in cases when repeatedly-requested or otherwise desired
functionality for ITK is in the IJ, reviews could be directly solicited.
- Perhaps during the quarterly review sessions, submissions that
appear to provide desired functionality but which lack reviews could
be noted and reviews requested. (Similar to above suggestion.)
- I don't know if submissions from "old editions" of the IJ are ever
re-visited. Perhaps they should be.
Regarding (b):
- I'm not sure what the point of the different "editions" of the
journal is. Why should time-of-submission matter at all? It seems to
serve to bury older work.
Regarding (a) and (b) together:
- Perhaps there isn't sufficient motivation for people like me to
write reviews? Maybe no submissions should be accepted in the ITK
unless the submitter has made three reviews of his or her own? This
might not provide the best kind of motivation (!), but it seems that
somehow making good reviews needs to be better-incentivized.
- I'd like to see more of the submissions put into the ITK
distribution somewhere under the "review" directory. Maybe if (1) the
submitter signs over the copyright, and (2) the buildbot can build
it, the submission could be deposited there.
You will note that most of these suggestions have to do with
procedure for ITK, or for the interaction of ITK and the IJ. Issues
which might best be discussed and resolved by the denizens of the itk-
dev list, and especially those on the oversight committee and the
cast of characters who makes up the regular conference-call attendees.
Zach
On Sep 16, 2006, at 5:39 PM, Luis Ibanez wrote:
>
>
> Hi Gaëtan, Zach
>
>
> It seems that the fundamental misperception still remains
> in this discussion:
>
>
>
> The notion that there is a small circle of
> all-powerful "developers" who decides the
> destiny of the toolkit.
>
>
>
> That shows that the process for adding new code into ITK
> has not been communicated properly to the community.
>
>
>
>
> The
>
> "...people in ITK developers who are choosing
> how to get code contributions in the toolkit..."
>
>
>
> is simply the following *Algorithm*:
>
>
> 1) Sort the contributions on the Insight Journal
> based on their ratings as provided by reviews.
> Reviews that are public and that can be posted
> by anybody.
>
>
> 2) Request copyright transfers of the source code
> to the Insight Software Consortium.
>
>
>
>
> This means that the "readers/reviewers/raters" of papers
> in the Insight Journal are *the ones who decide* what
> gets included in the toolkit.
>
>
>
> There is nothing in this process that is closed or that
> excludes anybody.
>
>
> There are no different grades of "developers".
> There is no "root" user in ITK.
>
>
>
>
> The fact that even though you have write access to the
> CVS repository, you will:
>
>
> "...not take the freedom to put in the
> toolkit all the nice work we can
> find in the IJ, even if cvs would
> let me do that..."
>
>
> Simply means that you are as responsible as all other ITK
> developers. We all share that same feeling and that same
> responsibility. And curiously enough, is this careful
> consideration and respect for the toolkit, one of the
> elements that makes so slow the process of integrating
> new code into the toolkit.
>
> In the same way that you will not dare to include code out of
> your own initiative, anyone of us older developers, will not
> dare either to make arbitrary unilateral decisions on what to
> include in the toolkit.
>
>
>
> It is not the fact that some "privileged group" of "developers"
> are the only ones that make arbitrary decisions on what to include
> in the toolkit. Instead, it is the fact that in order to include
> a piece of code into a toolkit that is used by a more than 2,000
> people in more than 40 different countries, and that should be
> maintained working for the next ten or so years, we want to get
> *at least* the opinions of *three* people by rating a contribution
> submitted to the Insight Journal.
>
>
> It is simply an *Open Process*
> that requires a *Participatory Community*.
>
>
>
> *EVERYTHING* about ITK is Open. Every document, from every
> meeting since 1999 is available on the Web. The agendas and
> minutes from every weekly conference are posted on the Wiki.
> The plans for releases, the bugs, the strengths and weaknesses
> of the toolkit. Everything is openly and publicly discussed.
>
>
> I would be surprised to find an Open Source project that has
> a more public and open process than ITK.
>
>
>
>
>
>
> That the process of moving source code
> from the Insight Journal to the Toolkit
> is slow....?
>
>
> YES,
>
>
> You are right. It is painfully slow.
>
>
>
> It was not supposed to be slow.
> It was not supposed to be painful.
>
>
>
> Why is it slow?
> and how it could be improved ?
>
>
>
> That's the *positive* discussion
> that we should be holding.
>
>
>
>
> Luis
>
>
>
>
> ======================
> Gaëtan Lehmann wrote:
>> Hi Luis,
>> You're wrong, I'm not more a developer than before, and the write
>> access to ITK cvs is just shortening the time needed to fix some
>> bugs, but it doesn't make any difference regarding the code
>> contribution process. I will not take the freedom to put in the
>> toolkit all the nice work we can find in the IJ, even if cvs
>> would let me do that. That's something only you can do. "The term
>> developers" was surely badly chosen - please replace it with
>> something better, like "people in ITK developers who are choosing
>> how to get code contibutions in the toolkit".
>> For me, open access and peer review are an evidence: they are the
>> future of science publication. In the mean time, ITK is loosing a
>> part of its active community.
>> The red pill can be difficult to take if too few people are taking
>> it at the same time
>> Gaetan
>> Le Sat, 16 Sep 2006 21:06:53 +0200, Luis Ibanez
>> <luis.ibanez at kitware.com> a écrit:
>>>
>>>
>>> Hi Gaetan,
>>>
>>>
>>> I would suggest that you approach this situation with a more
>>> positive
>>> attitude.
>>>
>>>
>>> You are referring to the IJ the same way a tax-payer will complain
>>> about the government.
>>>
>>>
>>>
>>> Complaining and whining is not an effective strategy when you are
>>> trying
>>> to change the status quo of a community that has been mushy-
>>> minded by
>>> decades of "Publish or Perish" propaganda.
>>>
>>>
>>>
>>> The Insight Journal *is not* an Institution,
>>> The Insight Journal *is* a Community Resource.
>>>
>>>
>>>
>>> There is no such a thing as "The Owners" of the Insight Journal.
>>>
>>>
>>>
>>> If you want to define such concept, probably the closest thing to
>>> the
>>> "Owners" of the Journal are the 1,200 subscribers to the ITK
>>> users list.
>>>
>>>
>>>
>>> This is a Journal intended to be supported and managed by the
>>> community.
>>> This is why it is poised to be a real *peer-review* Journal.
>>>
>>>
>>>
>>> The concept of "Freedom" is too new for our community, because the
>>> traditional peer-review process treats authors and readers as
>>> children
>>> who are supervised by the "adult" reviewers. The "adult"
>>> reviewers are
>>> the ones who decide what is good for the children to read. Such
>>> roles,
>>> atrophied critical thinking in the readers and lead them to
>>> passively
>>> accept any published paper as "absolute truth", because it has
>>> passed
>>> through the supposedly "holy" process of review by the "adults".
>>>
>>>
>>>
>>> Members of our community read technical journals with the same level
>>> of critical thinking that a watcher of TV-reality-shows exercise
>>> while
>>> sitting empty-minded for hours in front of the TV-set. The
>>> concept that
>>> they are actually entitled to question the content of papers is too
>>> alien for them. The notion that a published paper may contain
>>> mistakes,
>>> is a sacrilege for them. The notion that the claims made in a paper
>>> are supposed to be verified, is a blasphemy to them.
>>>
>>>
>>>
>>> After all,
>>>
>>> "Who are readers to question the judgment of the holy
>>> reviewers" ?
>>>
>>>
>>>
>>> As a consequence our community has developed the laziness of the kid
>>> that doesn't do his homework until the parents tell them to do so.
>>>
>>>
>>> The concept that *everybody* is entitled to write a review, is
>>> too new
>>> for our community. Most readers and authors assume that
>>> "reviewing" is
>>> an activity exclusively reserved for some "supernatural" beings
>>> who are
>>> the "Chosen Ones", the "Holy Reviewers".
>>>
>>>
>>> Authors are still trained to submit papers to Journals and "Pray"
>>> for
>>> the papers to be accepted by the "Chosen Ones".
>>>
>>>
>>> It takes a lot of education, motivation and *repetition* to
>>> rehabilitate
>>> readers and authors from the damage that the "Publish or Perish"
>>> disease
>>> has inflicted on their minds.
>>>
>>>
>>>
>>> In the meantime,
>>> You are discharging your frustration on the wrong crowd.
>>>
>>>
>>> Instead of complaining about the management of the Journal and its
>>> supposed "Owners", you should brainstorm on ways of encouraging all
>>> under-graduate and graduate students to act as reviewers. They
>>> should
>>> learn that they *ARE* qualified to be reviewers. They should
>>> learn that
>>> they *are* the "PEERS" that the term "Peer-Review" refers to:
>>>
>>>
>>> It is not "Supervisor-Review"
>>> It is not "Nobel-Prize-Review"
>>> It is not "Chosen-One-Review"
>>> It is not "Friend-of-The-Journal-Editor-Review"
>>> It is not "Owner-of-The-Journal-Review"
>>> It is not "Anonymous-Competitor-Review"
>>>
>>>
>>> It is "PEER-Review"
>>>
>>>
>>>
>>> Definition of "PEER":
>>>
>>> One that is of equal standing with another :
>>> EQUAL; one belonging to the same societal group
>>> especially based on *age*, *grade*, or *status*
>>>
>>>
>>>
>>> The "PEERS" of Graduate Students are *OTHER GRADUATE STUDENTS*.
>>>
>>>
>>>
>>> *EVERY* person that uses ITK *IS* qualified to act as a reviewer
>>> of the Insight Journal.
>>>
>>>
>>>
>>> There is no need for being "Friend" of anybody.
>>> There is no need for being consecrated through some "Holy
>>> Process".
>>> There is no need for belonging to some influence group.
>>> There is no need for being member of any social circle.
>>> There should not be a need to be pressured by an associate
>>> editor.
>>>
>>>
>>> That is the message that we must pass across the community.
>>>
>>> Having 1,200 users in the mailing list, there is no shortage of
>>> brains for reviewing less than 100 papers.
>>>
>>>
>>>
>>> Do not make the mistake of thinking
>>> that the Insight Journal is simply the
>>> gate for bringing source code into ITK.
>>>
>>>
>>>
>>> The Insight Journal is the RED PILL that may help our community
>>> to wake up from THE MATRIX of the "Publish or Perish" stratagem.
>>>
>>>
>>> As a community resource, the *Exercise of participating* in the
>>> review
>>> process is *MORE* important than the act of bringing code into ITK.
>>>
>>>
>>> It is a simple matter of teaching readers to stand on their own
>>> feet,
>>> to judge with their own minds, and to take responsibility for their
>>> own future.
>>>
>>>
>>>
>>> "Publish or Perish" was invented by budget managers and
>>> administrators
>>> who didn't wanted, or were incapable of understanding the science
>>> they
>>> were managing. It was an easy trick that made possible for them to
>>> "count numbers of papers" in an annual report instead of having to
>>> "read those papers" and try to understand their implications.
>>>
>>>
>>>
>>> We as a community, are all guilty of nourishing this primitive and
>>> decadent practice. We still assume that "number of papers" equals
>>> "productivity", regardless of what the paper content is.
>>>
>>>
>>>
>>> As dogs in training, the "Publish or Perish" stratagem, educated our
>>> community members to receive cookies (productivity credits) when
>>> they
>>> bark (publish), no matter what the barking was about. In the
>>> "Publish
>>> or Perish" MATRIX, there are no cookies for the ones who listen
>>> to the
>>> barking (read the publications). There are no cookies for the
>>> ones who
>>> attempt to repeat the barking of others (reproduce publications).
>>> Only
>>> "original barking" is supposed to be worth of cookies.
>>>
>>>
>>>
>>> The Insight Journal is like a public library. It is there, open for
>>> everybody. However, we can not force people to go to this library,
>>> we can not force people to contribute reviews.
>>>
>>>
>>> The challenge we face at this point is to untrain the damage that
>>> years
>>> of traditional publishing system has made. Every time that we hear a
>>> senior researcher telling a junior researcher "You should
>>> publish" we
>>> must jump in alert and examine "WHY" and "HOW" publishing should be
>>> done.
>>>
>>>
>>>
>>>
>>> You are now *an ITK developer*, at this point, you have access to
>>> all
>>> the same resources and information that any other "ITK developer"
>>> has.
>>> You have write access to the CVS repository, access to the mailing
>>> lists, access to the Wiki, access to the bug tracker. As you can see
>>> there is no magic aura that suddenly appears, no sparkly dust
>>> falling
>>> down while you walk. It is just one more entry among many other
>>> things
>>> in your to-do list.
>>>
>>>
>>> You can complain about other "ITK developers" not doing enough, but
>>> that's not going to make the world move any faster. The place where
>>> you could invest your energy in a positive way is by going to all
>>> junior researchers around you and letting them know that "Publish or
>>> Perish" is a lie, and that if they don't take responsibility for the
>>> quality of work shared in their community, nobody else will. At
>>> least
>>> nobody without an agenda of their own...
>>>
>>>
>>>
>>>
>>> Luis
>>>
>>>
>>>
>>> =======================
>>> Gaëtan Lehmann wrote:
>>>
>>>> Stephen,
>>>> Have I said or done something wrong ?
>>>> I'm writting a review for each new paper I'm able to review -
>>>> perhaps I'm late of 1 review or 2 - but I'll not submit 3
>>>> reviews per paper, and will not submit reviews for my own ones.
>>>> I'm asking about reviewer associate because their role is to
>>>> assign some reviewers to a paper, and most of the papers in
>>>> the IJ are still lacking of reviews - Note that I'm only
>>>> talking about the contributions to ITK, not about the articles
>>>> for the MICCAI. I would be pleased to incite potential
>>>> reviewers to write more reviews, but again, I have seen
>>>> nothing about the editor associate since your last mail.
>>>> Alsso, it seem that the ITK developers are not getting that
>>>> letting a contribution for 6 or 9 months in the insight
>>>> journal without much activity is like saying
>>>> "we don't care about your work and the time you spent to
>>>> write your article".
>>>> If there can't be more reviewers for the contributions to the
>>>> insight journal, do not require 3 reviews to integrate it in
>>>> the toolkit, are stop asking people to post their code to the
>>>> IJ. If you (developers) really don't care, just keep things
>>>> like that - that's fine.
>>>> Some days like this one, I'm really not sure why I'm still
>>>> using the IJ, and warning their owners about what is wrong -
>>>> it also look like a huge waste of time
>>>> Gaetan
>>>> Le Sat, 16 Sep 2006 01:52:17 +0200, Stephen R. Aylward
>>>> <Stephen.Aylward at kitware.com> a écrit:
>>>>
>>>>> Hi
>>>>>
>>>>> Thanks for volunteering to provide more reviews! There are
>>>>> instructions on the wiki on how to write reviews and
>>>>> contribute to the IJ.
>>>>> As each new paper arrives - I am assigning it to reviewers, but
>>>>> we always welcome additional reviews.
>>>>>
>>>>> Thanks,
>>>>> Stephen
>>>>>
>>>>> ___
>>>>> Sent with SnapperMail
>>>>> www.snappermail.com
>>>>>
>>>>> ..... Original Message .......
>>>>> On Fri, 15 Sep 2006 22:34:56 +0200 Gaëtan Lehmann
>>>>> <gaetan.lehmann at jouy.inra.fr> wrote:
>>>>>
>>>>>> Le Mon, 21 Aug 2006 18:19:37 +0200, Stephen R. Aylward
>>>>>> <Stephen.Aylward at Kitware.com> a écrit:
>>>>>>
>>>>>>> Thanks for all who have already volunteered to be Associate
>>>>>>> Editors for
>>>>>>> the Insight Journal!
>>>>>>> http://insightsoftwareconsortium.org/wiki/index.php/IJ-Whos-Who
>>>>>>>
>>>>>>> We are still looking for a few more volunteers. You've
>>>>>>> contributed
>>>>>>> code to open-source medical image analysis...why not help to
>>>>>>> spread the
>>>>>>> news of your contribution and the contributions of others...
>>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> What is the status of this initiative ?
>>>>>> I've seen nothing about it since this mail, and there is lots of
>>>>>> contributions in the IJ which again will not get their 3
>>>>>> reviews before
>>>>>> the sept 20 (http://www.itk.org/Wiki/ITK_Release_Schedule).
>>>>>> If things stay
>>>>>> like that, some of them will be more than one year old without
>>>>>> getting
>>>>>> their 3 reviews.
>>>>>> If things stay like that, lots of contributors will be
>>>>>> discouraged
>>>>>>
>>>>>> Believe me, I'm a contributor
>>>>>>
>>>>>> And what a waste of time.
>>>>>> If things have been done earlier, some classes can have been
>>>>>> integrated in
>>>>>> the toolkit 2 releases ago
>>>>>>
>>>>>> Gaetan
>>>>>>
>>>>>>
>>>>>>
>>>>>> -- Gaëtan Lehmann
>>>>>> Biologie du Développement et de la Reproduction
>>>>>> INRA de Jouy-en-Josas (France)
>>>>>> tel: +33 1 34 65 29 66 fax: 01 34 65 29 09
>>>>>> http://voxel.jouy.inra.fr
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>>
>
>
>
>
>
>
>
>
> _______________________________________________
> Insight-developers mailing list
> Insight-developers at itk.org
> http://www.itk.org/mailman/listinfo/insight-developers
More information about the Insight-developers
mailing list