[vtkusers] [query] good venue for publishing a dataset?

[Will Ray]

In response to my query, Mathieu Malaterre and Luis Ibanez wrote:


> Ok I think before hurting people feeling we need more info, Where 
> exactly did you go ? I am looking at:
> http://www.insight-journal.org/
> 
> 
> And here are the latest publications...

I'm going to answer you and Luis here, not because I want to start
a political argument, but because I really /want/ your proposed
publishing model to survive and gain credibility, and I can't be
the only one with these concerns, or who lives in an environment where
they are concerns.  

My point, before I start, is /not/ to hurt feelings.  I'd love to see
this model become the standard for publication and data sharing.  I've
personally always tried to operate as though it is (often much to my
employer's chagrin), and I've always insisted on making the code, data
and methods openly available for anything I've ever published anywhere.

Partly this is because I've been living in the bio/life-sciences world
for a while, where it's just gauche to not publish necessary and
sufficient data and code, and partly this is because growing up in the
era of Richard Stallman, one develops the sense that this is just the
right thing to do.

Still, it's not much use to anyone, to publish good data in venues where
it simply will not be seen, good intentions on the part of the venue, or
not.

(and as an aside, I don't see volume dataset publications in the
existing TIJ articles lists, so partly my question is whether this
is an appropriate venue for such a publication anyway)

You're free to convince me that TIJ has greater penetration than it
looks from the web page - once you do, put that convincing argument in
great big letters on the front page, because I guarantee that there are
other people with data and methods who would be willing to publish in your
venue, if they weren't worried that the statistics didn't add up to
much readership.

> So either we don't have the same definition of 'much life' or I am 
> missing something, but seriously the last 5 publications are from this 
> month.


From the same page, you'll see that TIJ sports 16 articles from the
August-December issue, and that your 5 articles from this month, are
five of six total, between this month and last month.

So, no, we don't have the same definition of "much life".  My typical venues
are Nucleic Acids Research, and Bioinformatics, where I spend $1500 to
$2500 per article, so that I can get them into their "open access"
(equals instantly online, no subscription required, in their model) section.

30-50 original (not revision - or rather, only a single version, revision
or not, is ever published) peer-reviewed articles per month are about par.
Not that that's a necessity for "life", but the reality is that high and
lofty goals aside (good though they may be), it's difficult to risk putting
a year's work into a venue where one can't tell if anyone's ever going to
see it.

Young journal or not, you might want to put some statistics on the TIJ
web-page that indicate how many views/downloads the articles are getting,
etc - something to inspire some confidence that things are going well.

> Is anybody still reading papers published in closed access journals ?

It's a nice quote, but... What does CiteSeer have to say about this?
I know the statistics suggest that web-access articles are cited more
frequently than dead-tree-access.  How are comparable articles in
"closed" publications doing compared to articles in TIJ in terms of
actually being used by other people to develop and further their own
research?  I know the model sounds like a good idea.  The tools exist
to generate a convincing statistical argument that it /is/ a good idea,
if the statistics bear out that assertion.

Again, if they _do_ show that TIJ's impact exceeds other start-up
journals with a closed model and traditional reviews, you /really/
need to publish this on your web page - leaving it out, almost implies
that the statistics disagree with the supposed benefits.

> Papers submitted to the Insight Journal are published in a matter of
> hours and are available in Open Access (gratis) to anybody in the world.

That is part of the problem, rather than the solution...  Whether
you like it or not, the lack of a peer-review mechanism that can prevent
a bad publication from ever seeing the light of day, devaluates
publications in TIJ considerably.   The fact that a bad article can be
clobbered with publicly visible bad reviews, doesn't change the fact that
few people will risk publishing their work in a venue where it might appear
beside something that's utter bunk.  It also doesn't change the fact that
research published in such an "un-reviewed" format is all but valueless
when it comes to supporting grant applications, etc.

As far as NIH reviewers are concerned, for example, you'd be better
off not admitting to having done preliminary experiments, as compared
to publishing your results in a non-reviewed journal.

I'd love to see TIJ demonstrate that these articles have significant
impact, and that the open review process works.  Right now, I don't
see it - I see arguments that it _should_ work, not facts that
it _does_.

> Note also that the Insight Journal is not intended to be the typical
> vanity-science journal where people publish just to met their annual
> quota of papers required to account for their "productivity". That
> empty practice is void of any scientific value and only demonstrate
> how the principles of scientific research can be degraded when they
> are evaluated by bureaucrats who simply "count" the number of papers
> in your CV and your annual reports, but never read those papers, and
> probably can not discern their implications.
> 
> The Insight Journal is after the important goal of recovering honesty
> in the scientific endeavor by enforcing REPRODUCIBILITY.  Papers sent
> to the Insight Journal are expected to be in the format of a technical
> report that makes possible for *any* reader to replicate the results
> described in a paper.

Again, believe me, you're preaching to someone who already drinks
the kool-aid.  Wearing one of my other hats, I could have written
almost exactly the same thing you did above.  (I wrote down a total
of three notes from SIGGRAPH this year, one of those is to check out TIJ,
based on Terry Yoo's talk).

("vanity journals" however is kind of a red herring - there are
plenty of CV-padding vanity journals out there.  ISI top-10 journals,
etc, are not part of this list (despite success in getting an article
into them, being worth a bit of vanity). Even bureaucrats are
clever enough to have scoring schemes that de-weight vanity journals
significantly, and up-weight publications that actually contribute to
other people's research and productivity.  My biggest concern is that
TIJ is currently _exactly_ what it's claiming to fight against - a
CV-padding venue that doesn't actually measurably impact anyone's
productivity.  I'd love to be proven wrong.)

However...  The reality of the academic world is that
one lives or dies, depending on whether one's research is seen, and
found to be valuable by one's peers.  The only measure that's applied
to that "found to be valuable" statistic, is re-citation by other
researchers.

I don't like the model.  I don't like having to work
under it.  I don't like the fact that there's a huge amount of social
inerta that keeps the machine working this way.

I am however, pragmatic enough to be worried about putting the results
of a year's worth of an institution's support funds, into a venue that
they may see as valueless.  Give me the ammunition to prove that they're
wrong, and I'll happily make the effort to try to prove it to them.
Then put that ammunition up on your web-page, because I guarantee there
are others with the same concerns that I have.


> When you are joining a revolution based on principles,
> you should not ask
> 
>                 "How many have already joined?",
> 
> instead,
> you should be proud to be among the first ones to join.

Been there.  Done that.  Since 1989 or so.




Will Ray