[Insight-developers] OPEN ACCESS : Reproducible Research

[Luis Ibanez]

     "For a PhD thesis to be accepted by me,
      I routinely remove the computed figures and recompute them."


                                     Jon Claerbout


       http://sepwww.stanford.edu/sep/jon/

------------------------



      http://sepwww.stanford.edu/research/redoc/IRIS.html



"I have three textbooks on reflection seismology containing 243
illustrations that are computed from data and theory.
I routinely erase all these 243 figures, and then I recompute them."



"The purpose of reproducible research is to facilitate someone going a
step further by changing something. The first step that someone will
want to make is to be sure that your work is reproducible before they
change and improve upon it."


--


"On average, two PhD students graduate each year from our laboratory,
the Stanford Exploration Project. Years ago, junior students who built
on their seniors' work often spent a considerable effort to merely
reproduce their colleagues' old computational results."



"Indeed, the problem occurs wherever *traditional methods* of
*scientific publication* are used to describe computational research.

In a traditional article the author merely outlines the relevant
computations: the limitations of a paper medium prohibit a complete
documentation including experimental data, parameter values, and the
author's programs. Consequently, the reader has painfully to
re-implement the author's work before verifying and utilizing it."

"Even if the reader receives the author's source files (a feasible
assumption considering the recent progress in electronic publishing),
the results can be recomputed only if the various programs are invoked
exactly as in the original publication. The reader must spend valuable
time merely rediscovering minutiae, which the author was unable to
communicate conveniently. "


----


"Some people see reproducibility of research as a troublesome ethical
issue. We see it as an everpresent, irritating practical concern."

"We are annoyed when we cannot easily reproduce our own work. Some of us
are irritated when other professors publish grandiloquent papers whose
conclusions require too much time to confirm. And we are further
frustrated when new students cannot reproduce (in a year) the work of
other recently graduated students. Although each student prepares a
doctoral dissertation that should enable others to reproduce that work,
the reality is that paper documents are outdated; they no longer fulfill
the role they did a generation ago when data consisted of photographs
and pencil marks on a piece of paper---when theory was several pages of
Greek symbols."


"*Paper no longer serves us well.* Our data now resides in a computer,
and our theory is embodied in computer programs. Many libraries overflow
and they cannot afford the materials they want. [but] At a million
characters per book, over 600 books could be on a CD-ROM with a retail
cost below 2 cents per book..."


----


Guidelines on how to make reproducible research:
http://sepwww.stanford.edu/research/redoc/IRIS.html


Other followers of this "eccentric" idea:
http://lcavwww.epfl.ch/reproducible_research/