[vtk-developers] C++ unit tests

Tue Jul 28 14:32:44 EDT 2015

Sounds like we "just" need a wiki page / blog post / Source article /
white-papery-thingy about C++ VTK testing.

(And then a follow-on one for python.)

Having a way to add a C++ unit test of a newly added VTK class will be
great! (Adding them for all the existing classes will certainly be a
chore, but it is finite, and will increase code coverage, although
perhaps not as easily as any of us would like...)

On Tue, Jul 28, 2015 at 2:27 PM, Shawn Waldon <shawn.waldon at kitware.com> wrote:
> +1 to the test recipe.
>
> On Tue, Jul 28, 2015 at 2:22 PM, David E DeMarle <dave.demarle at kitware.com>
> wrote:
>>
>> We've got years of layers of great but poorly described regression test
>> infrastructure in the code today that makes it harder than it could be to
>> make new tests. If this is yet another layer, I'm not in favor.
>>
>> But if this makes adding tests tractable for newbies and we put together
>> and post an easy but comprehensive recipe for adding tests along with this,
>> then I'm very much in favor.
>>
>>
>>
>>
>> David E DeMarle
>> Kitware, Inc.
>> R&D Engineer
>> 21 Corporate Drive
>> Clifton Park, NY 12065-8662
>> Phone: 518-881-4909
>>
>> On Tue, Jul 28, 2015 at 2:10 PM, Bill Lorensen <bill.lorensen at gmail.com>
>> wrote:
>>>
>>> I have been adding unit tests lately using old school vtk testing.
>>> I'll be willing to kick the tires on David's framework.
>>>
>>> My goal in these unit tests to get as close as 100% coverage as
>>> possible, verifying results when possible. This includes exercising
>>> Error and Warning code and checking the messages themselves.
>>>
>>> I'll admit that I'm skeptical, but hopeful that adding a new test
>>> framework will increase code coverage.
>>>
>>> On Tue, Jul 28, 2015 at 1:34 PM, David Thompson
>>> <david.thompson at kitware.com> wrote:
>>> > Hi David(s),
>>> >
>>> > I really like the single-header idea, especially the macros for testing
>>> > whether values match to within a tolerance.
>>> >
>>> > My concerns are
>>> >
>>> > 1. The C++ Testing/Rendering/vtkTesting.h class which provides (or now
>>> > that we have ExternalData, provided?) access to baseline images doesn't seem
>>> > to fit into the FIXTURE macro. Is there a way to integrate it? Or a way to
>>> > extend the vtkTesting class to be more usable by including methods similar
>>> > to your macros?
>>> >
>>> > 2. One thing I dislike about most/all of the C++ test frameworks is
>>> > that the error messages are assumed to be fixed and the methods often do not
>>> > present a way to include the incorrect result as part of the failure
>>> > message. It is nice that your macros include that. However, given that tests
>>> > often fail on machines I don't have access to, it is really nice to
>>> > encourage people to write useful messages. Macros like CHECK_CLOSE would be
>>> > better if they accepted a message as a final argument, especially if the
>>> > argument was passed to a std::ostream object the way
>>> > vtkGenericWarningMacro() does so that people can (are forced to) provide
>>> > more context about where the two numbers being compared come from.
>>> >
>>> >         David
>>> >
>>> >> On Jul 28, 2015, at 10:43 AM, David Gobbi <david.gobbi at gmail.com>
>>> >> wrote:
>>> >>
>>> >> Hi David (and David),
>>> >>
>>> >> The only advantage of my unit test header is that it is a single
>>> >> header
>>> >> (a short one, too) with zero dependencies.  So it's easy to add to VTK
>>> >> and it's trivial to maintain.
>>> >>
>>> >> If either of you has a better solution, then go ahead and make a merge
>>> >> request.  I'll review it for you ;)
>>> >>
>>> >>  - David
>>> >>
>>> >>
>>> >> On Tue, Jul 28, 2015 at 8:27 AM, David Thompson
>>> >> <david.thompson at kitware.com> wrote:
>>> >> Another project I work on has been using Bandit[1] to good effect.
>>> >>
>>> >>         David
>>> >>
>>> >> [1]:http://banditcpp.org/reference.html
>>> >>
>>> >> On Jul 28, 2015, at 10:01 AM, David Cole via vtk-developers
>>> >> <vtk-developers at vtk.org> wrote:
>>> >>
>>> >> > A single header file unit testing solution sounds like a good thing.
>>> >> >
>>> >> > ... but ... seems like we could also take advantage of a
>>> >> > well-established generally used unit testing framework (like gtest
>>> >> > or
>>> >> > CppUnit) just as easily, and not clutter the VTK source with
>>> >> > yet-another-custom-thing-to-learn-about-VTKs-infrastructure.
>>> >> >
>>> >> > Would it be just as easy to add unit testing using an external unit
>>> >> > testing package? Or are there things about that scenario that are
>>> >> > overly complicated that having our own makes simpler...?
>>> >> >
>>> >> >
>>> >> > David C.
>>> >> >
>>> >> >
>>> >> >
>>> >> > On Mon, Jul 27, 2015 at 5:25 PM, David Gobbi <david.gobbi at gmail.com>
>>> >> > wrote:
>>> >> >> Hi All,
>>> >> >>
>>> >> >> I've always enjoyed using the python unit testing framework in
>>> >> >> "vtk.test".
>>> >> >> It's been brought up on the list before, but it would be nice if
>>> >> >> VTK
>>> >> >> included a unit test framework for C++, as well.
>>> >> >>
>>> >> >> For my work projects, I've been using a unit test header that I
>>> >> >> wrote a year
>>> >> >> and a half ago:
>>> >> >> https://github.com/dgobbi/UnitTest/blob/master/UnitTest.h
>>> >> >>
>>> >> >> It's a single header file that provides a collection of macros for
>>> >> >> building
>>> >> >> unit tests, it's meant to be innocuous and easy to use.  It could
>>> >> >> go into
>>> >> >> VTK as Testing/Core/vtkUnitTest.h and people could use it at their
>>> >> >> leisure.
>>> >> >>
>>> >> >> Any thoughts?
>>> >> >>
>>> >> >> - David
>>> >>
>>> >>
>>> >
>>> > _______________________________________________
>>> > Powered by www.kitware.com
>>> >
>>> > Visit other Kitware open-source projects at
>>> > http://www.kitware.com/opensource/opensource.html
>>> >
>>> > Search the list archives at:
>>> > http://markmail.org/search/?q=vtk-developers
>>> >
>>> > Follow this link to subscribe/unsubscribe:
>>> > http://public.kitware.com/mailman/listinfo/vtk-developers
>>> >
>>>
>>>
>>>
>>> --
>>> Unpaid intern in BillsBasement at noware dot com
>>> _______________________________________________
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>>
>>> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>>
>>
>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>
>>
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/vtk-developers
>
>