[vtk-developers] Coverage

[David Gobbi]

Yeah, I suspect that a few of the tests aren't in the right module
yet, but from skimming through the list, it looks like most of them
are already in the right location.

I'm not sure why circular dependencies would be a problem, since the
TEST_DEPENDS are listed separately from the main dependencies?
Unless the tests always have to be built at the same time as the module
itself, but I can't imagine that being the case since the tests are in
a separate directory.

At the very worst, we could make a module called Testing/Legacy and
use it as a catch-all for tests that have complex dependency issues.

 - David

On Mon, Mar 4, 2013 at 6:49 AM, Nikhil Shetty <nikhil.shetty at kitware.com> wrote:
> Here is my working list which I used to categorize all the tests. It gives a
> little more information about the include files the test depends on which
> were not modularized at that time.
>
>
> https://docs.google.com/a/kitware.com/spreadsheet/ccc?key=0AkuX7fTJrmgydFdRVTUtckJCVjl1amtzNGtUUUhhLUE#gid=0
>
> So it just may be that many test are not in simply because at the time some
> of the files they were dependent on were not modularized.
>
> From what I remember some others are disabled because they were creating a
> circular dependencies among modules.
>
>
>
> On Mon, Mar 4, 2013 at 8:32 AM, Nikhil Shetty <nikhil.shetty at kitware.com>
> wrote:
>>
>> I wrote one in Common Lisp. I used it to categorize tests. Basically it
>> first makes a map of all the files to modules. Then for each test finds the
>> headers and candidate modules it can belong to. I can send it to you if you
>> like.
>>
>> Here is the output.
>>
>>
>> https://docs.google.com/a/kitware.com/spreadsheet/ccc?key=0AkuX7fTJrmgydE5LaXZvbXdjT2Q5bDRyUUVFaWU0VXc#gid=0
>>
>> On Mon, Mar 4, 2013 at 1:32 AM, David Gobbi <david.gobbi at gmail.com> wrote:
>>>
>>> Is there a script that can automatically list what modules a
>>> particular .cxx file depends on?
>>>
>>> On Sat, Mar 2, 2013 at 4:49 PM, Bill Lorensen <bill.lorensen at gmail.com>
>>> wrote:
>>> > I suspect that many of these were left out during modularization.
>>> > Perhaps
>>> > because they caused dependencies on other modules.
>>> >
>>> >
>>> > On Saturday, March 2, 2013, David Gobbi wrote:
>>> >>
>>> >> I just sent a list of files to vtk-devel that shows what is
>>> >> newly-disabled in master as compared to release.
>>> >>
>>> >> And I've attached another file that includes the full path
>>> >> to each disabled test.
>>> >>
>>> >>  - David
>>> >>
>>> >> On Sat, Mar 2, 2013 at 4:10 PM, Berk Geveci <berk.geveci at kitware.com>
>>> >> wrote:
>>> >> > Can you generate this as a list of files? Then one or more of us has
>>> >> > to go through these and either remove them or make them work again.
>>> >> >
>>> >> > -berk
>>> >> >
>>> >> > On Sat, Mar 2, 2013 at 5:21 PM, David Gobbi <david.gobbi at gmail.com>
>>> >> > wrote:
>>> >> >> There are also a number of tests that aren't being run.  I wrote a
>>> >> >> python script (attached) that looks through all the Testing
>>> >> >> directories and identifies test programs that don't appear in a
>>> >> >> CMakeLists.txt file.  It reports:
>>> >> >>
>>> >> >>  166 out of 1770 tests are not enabled
>>> >> >>  44 are commented out, the rest are unlisted in cmake
>>> >> >>
>>> >> >> Compare this to VTK 5.10:
>>> >> >>
>>> >> >>  54 out of 1213 tests are not enabled
>>> >> >>  21 are commented out, the rest are unlisted in cmake
>>> >> >>
>>> >> >> It's probably more informative to list the tests by language:
>>> >> >>
>>> >> >> Cxx: 132 not enabled (39 in cmake but commented out)
>>> >> >> Tcl: 19 not enabled (2 in cmake but commented out)
>>> >> >> Python: 15 not enabled (3 in cmake but commented out)
>>> >> >>
>>> >> >> Recently Bill and others have done an awesome job of reactivating
>>> >> >> tcl
>>> >> >> and python tests, but a large number of C++ tests are still
>>> >> >> missing/disabled.
>>> >> >>
>>> >> >>  - David
>>> >> >>
>>> >> >>
>>> >> >> On Sat, Mar 2, 2013 at 1:46 PM, Berk Geveci
>>> >> >> <berk.geveci at kitware.com>
>>> >> >> wrote:
>>> >> >>> Yup. One other thing that we would like to try in the near future
>>> >> >>> is
>>> >> >>> to look at combined VTK + ParaView coverage of VTK. ParaView tests
>>> >> >>> some of VTK more thoroughly and that should increase coverage
>>> >> >>> further.
>>> >> >>> I am hoping for 70%. Beyond that, we will have to write more
>>> >> >>> tests.
>>> >> >>>
>>> >> >>> -berk
>>> >> >>>
>>> >> >>> On Sat, Mar 2, 2013 at 10:08 AM, Bill Lorensen
>>> >> >>> <bill.lorensen at gmail.com> wrote:
>>> >> >>>> Berk,
>>> >> >>>>
>>> >> >>>> Looks like tcl and python were added to ubuntu coverage. Now it
>>> >> >>>> is
>>> >> >>>> showing
>>> >> >>>> 66%.
>>> >> >>>>
>>> >> >>>> Bill
>>> >
>>> >
>>> >
>>> > --
>>> > Unpaid intern in BillsBasement at noware dot com
>>> >
>>> _______________________________________________
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://www.vtk.org/mailman/listinfo/vtk-developers
>>>
>>
>>
>>
>> --
>> Regards
>> Nikhil
>
>
>
>
> --
> Regards
> Nikhil