[CMake] CMake bug tracker discussion

[Alan W. Irwin]

On 2010-12-09 17:06-0500 David Cole wrote:

> Hello CMake users and devs,
>
> (And now for something completely different...)
>
> Controversial questions:
>
> - Should we eliminate the bug tracker entirely and just do all
> discussion and patches on the mailing list? (Why have two sources of
> information...?)
>
> - Or, alternatively, should we eliminate the bulk of mailing list
> traffic, and insist on issues in the bug tracker being the main
> conversational forum for the whole community?
>
> I'd like to have this discussion here publicly, to try to get a good
> sense of varous community members attitudes and feelings.
>
>
> I'll start the ball rolling by saying that, personally, I like the bug
> tracker. I find it much easier to keep a list of issues organized and
> accessible than I can with email filters and folders. But I still see
> a need for both tools.
>

I am glad you brought this subject up because there is a real problem
engulfing CMake's bug tracker as we speak.  It appears from the
resolved category of CMake bugs on the bug tracker, that ~10 bugs were
resolved (not necessarily fixed) in November, but that same month saw
~50 new bugs reported (to the cmake-devel mailing list).  That ratio
of 5 new bugs for every one resolved means the CMake bug tracker is
rapidly being filled with unresolved issues with no hope of ever
resolving them unless some substantial changes are made.

Here are some ideas to help reduce that insanely unsustainable ratio
of five down to a sustainable unity.

1.  Reduce the number of new bug reports.

   a.  My guess is lots of those new bug reports are noise due to
       misunderstanding of how to use CMake (despite what I consider to
       be outstanding documentation). So reduce the numbers by strongly
       suggesting that all potential bugs should be discussed first on
       the mailing list (with [BUG?] in the subject line to draw
       attention to it) to make sure they really are bugs before
       writing up a bug report.

   b.  Avoid using the bug tracker whenever possible, e.g., quick and
       obvious fixes. I have experienced other organizations which
       demanded everything go to the bugtracker where posting to the
       bugtracker became the point rather than actually fixing bugs. In
       other words, bug trackers have been used by some organizations
       as an excuse for inaction on bugs!  This is clearly not the case
       with the CMake developers (I just this morning had one of my
       discussed bugs fixed without going through the bug tracker), but
       I feel strongly about that important pitfall of bugtrackers so I
       brought it up explicitly here. It should be emphasized that bug
       trackers can play an important role for the more difficult fixes
       that take a lot of experimentation and thought to get right, but
       in my view creating a bug report on a bug tracker has an obvious
       cost for the reporter and for the bug triage team afterward
       so should be avoided for the simple stuff.

2. Increase the resources you spend on resolving bugs in the bug
tracker.

   a.  More community involvement, e.g., encourage a community bugfix
       team whose principal job was to do the first-level bug triage
       such as investigating the questions of whether it is a
       well-posed bug report (e.g., is there enough information in the
       bug report), is the issue reproducible, and ultimately is it a
       real bug or not.  And if not, members of that team would have
       the power to close the bug as "not a bug".

   b.  A modest increase in Kitware resources devoted to bug fixing.
       This is obviously a judgement call that is up to Kitware, but if
       I had a say in Kitware's allocation of resources, this is how I
       would argue for these additional resources. Kitware doesn't get
       direct money for their CMake development efforts, but CMake is
       an enormous boost to their reputation which presumably
       translates to more commercial success through their paid-for
       products.  The bottom line is ultimately it hurts Kitware's
       reputation if their CMake bugtracker is filled to the brim with
       unresolved issues so some increase in Kitware resources as well
       as encouraging community involvement is warranted to help deal
       with the current bad situation.

In sum, I agree with the others who have posted on this question that
you need both mailing-list discussion of all bugs and the bugtracker.
My additional ideas beyond that general idea are (1) you should
encourage mailing list discussion to make sure a CMake problem some
user is having is really due to a bug, and (b) make a conscious effort
to make quick fixes when those are warranted, and (c) encourage use of
the bug tracker only for the more difficult issues where there is
obviously no quick fix.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state implementation
for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of
Linux Links project (loll.sf.net); and the Linux Brochure Project
(lbproject.sf.net).
__________________________

Linux-powered Science
__________________________