[vtk-developers] cdash/gerrit emails about failing tests...

[David Gobbi]

Hi Bill,

I'm just saying that the situation isn't quite as bad as the dashboard
makes it look.  And, to be honest, I remember plenty of times in years
past where the dashboard was much worse for extended periods of time.

As far as the dashboard is concerned, the number of things that have
to be fixed is small and quite manageable:

1) The coverage machine needs to be more stable, you can't be doing
coverage on a bleeding-edge system.

2) The 25 tests that fail on all machines must be fixed.  This is a pretty
small number.  Heck, in the past I've fixed that number of failing tests
by myself in a week during my spare time.  Unfortunately I don't have
as much spare time as I used to.  But I can take 5 of the 25.

3) Valgrind tests.  Most developers ignore this part of the dashboard
completely.  This is not good.

There would be a #4, compiler warnings, but the dashboard is
remarkably clean in this regard, so warnings are a low priority at the
moment.

Now the overall issue of developer participation in the code quality
process... that's a much bigger issue than the dashboard alone.
Is it a mentorship issue, i.e. are new developers not being taught
the "ways of the source"?  Are there too many developers, i.e. too
many cats to herd?  Does gerrit make it too time-consuming to
submit follow-up fixes when people break the dashboard?  (I myself
have found that some developers do not respond when I ask for
a review... and I feel guilty about going to the "reliable" reviewers
over and over again).

 - David





On Wed, Jan 30, 2013 at 3:10 PM, Bill Lorensen <bill.lorensen at gmail.com> wrote:
> David,
>
> In years past, I gave many talks bragging about the high quality of
> our toolkits. I would often give a live demo and point to the nightly
> dashboard. We and others used software quality as a selling point of
> our commitment to open source processes. I know for certain that we
> won at least two large government grants because of our committment to
> quality.
>
> We also gave many GE internal talks, taunting our process and I
> believe many GE businesses to improve their software processes.
>
> I suspect that you, as our first outside developer, also promoted the
> quality of VTK.
>
> Bill
>
> On Wed, Jan 30, 2013 at 4:57 PM, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>> I'm saying that the machine that reports coverage and the machine that
>> runs valgrind tests less than 1/2 the code.
>>
>> I agree that there are so many failing tests that we have no idea
>> about the quality of vtk.
>>
>> In the past, we bragged about our process. We cannot do that anymore.
>>
>> Bill
>>
>> On Wed, Jan 30, 2013 at 4:51 PM, David Gobbi <david.gobbi at gmail.com> wrote:
>>> On Wed, Jan 30, 2013 at 1:25 PM, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>>>
>>>> Coverage is down to 44%. This means we test less than 1/2 of vtk's code.
>>>> Why? Because over 900 tests are failing on the coverage machine:
>>>> http://open.cdash.org/viewTest.php?onlyfailed&buildid=2789553
>>>
>>> Your statement that we test less than 1/2 of the code is false.  There are
>>> some dashboard machines (e.g. hythloth) cover much more.  I know that
>>> I'm being picky with semantics here, but the truth is, we have so many
>>> failing tests that the dashboard isn't even able to produce accurate code
>>> quality metrics.
>>>
>>>  - David
>>
>>
>>
>> --
>> Unpaid intern in BillsBasement at noware dot com
>
>
>
> --
> Unpaid intern in BillsBasement at noware dot com