[Insight-developers] OPEN SCIENCE: IEEE Editor in Chief : on Reproducible Research

[Luis Ibanez]

Some interesting excerpts from:


http://ieeexplore.ieee.org/iel5/83/4011946/04011955.pdf


    From the Editor-in-Chief
    by Jelena Kovacevic
    IEEE Transactions on Image Processing,
    Page(s): 3625 - 3626
    Dec. 2006
    Volume: 15 Issue: 12



This is an excellent paper that anybody interested
on scientific research should read. The author nicely
describes what should be considered the "sine-qua-non"
of publishing scientific work in the domain of
computation:


   The fact that the described work must be reproducible,
   and that the purpose of a publication should be to
   facilitate others to exert such reproducibility.




      Luis



=====================================================


<Excerpts Begin>


Reproducible Research (RR):

...What is it, why do we do it, how do we get to RR, and,
finally, how we publish true RR. RR refers to the idea that, in
‘‘computational’’ sciences, the ultimate product is not a published
paper but, rather, the entire environment used to produce the results
in the paper (data, software, etc.). While it might sound natural and
obvious, how many of us in signal processing really do it that way?

...


...Of course, to truthfully replicate the experiment, the paper has
to provide enough specific detail about the experiment to allow
another group to mimic it—the reproducibility criterion. While the
above criteria seem simple in theory, in practice, things do not
always work smoothly, as alluded to in the above headlines...


....


Donald Knuth,...was followed by Claerbout and his Stanford Exploration
Project, in whose view a scientific article is merely advertisement of
scholarship; the real scholarship includes software and data which went
into producing the article,...


.....


...We could encourage authors to publish first-class, experimental
work, as well as submit work that uses a known algorithm in a new
setting or with a different type of data. ...


.....


...Finally, how do we write RR papers? We could establish a
template of what we think are essential parts of such work.
For example: Every algorithm should have a block-diagram and
pseudo-code; all the parameters needed to run that algorithm
should be clearly listed; software and data should be available
online.



<Excerpts End>