[vtk-developers] Valgrind defects are zero!

[David Cole]

On Thu, Dec 8, 2011 at 2:33 PM, David Doria <daviddoria at gmail.com> wrote:
>> We support too many platforms and options to be able to cover all possibilities.
>
> Of course we can't cover ALL possible configurations. However, the
> only defects we know about (the ones you fixed) are the platforms we
> test for. Therefore it seems to make sense to only allow new code that
> does not introduce problems on platforms we support/test, right?
>
>> Also, ITK has 3 robot builds that are kicked off for each gerrit patch.
>
> Right, but the *assumption* is that the human reviewers will look at
> this, which may or may not happen in practice. What I am suggesting is
> that these robot builds report back to gerrit (as "robot reviewers")
> and flag the patch as "not approved" if anything fails/has defects.
>
>> This certainly helps but does not eliminate the introduction of defects.
>
> My proposal *eliminates* the introduction of defects on platforms we
> test, which is really the best we can hope to do.
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://www.vtk.org/mailman/listinfo/vtk-developers
>

All fine suggestions in theory, but running a valgrind dashboard on
every single gerrit topic will be quite time consuming. We will need
multiple cdash at home clients running dashboards in order to keep up
with the volume of commits if we expect a valgrind dashboard from each
and every gerrit patch.

And Bill's point remains valid: there will still be untested
combinations (in fact, I'm sure there are untested combinations right
now) that will still allow defect introduction.

We can definitely do better, but the only way to guarantee not adding
any defects is to stop development work. :-)