[vtk-developers] cdash API and VTK's most wanted list

[David E DeMarle]

Howdy y'all,

I've been working on cleaning the VTK dashboard lately and recently
learned something that is really useful - that CDash's Web API is
really powerful.

See: http://public.kitware.com/Wiki/CDash:API

Using it I wrote the attached python script which scrapes last night's
VTK test results and presents them in a test centric view, rather than
cdash's native submitter centric view.

The test centric view is more useful I think because it helps you to
prioritize what _code_ or project feature is worst and should be fixed
soonest, as opposed to what machine is performing worst. It also gives
you a more realistic view of the state of the project, because it
won't make you falsely believe, for instance, that there are at most
the same number of problems as the worst dashboard shows. (20 or so vs
70 today)

Also, since it is a script it takes less than a second to generate the
report, whereas if you build up the same list on your own manually by
clicking on cdash's web pages it will take you about an hour on a
project (and project state) like VTK today.

If you run the script you will see that the worst tests are:

vtkIOExportCxx-TestGL2PSExporterVolumeRaster-VerifyRasterizedPNG , 6 ,
['Mac10.7.3-clang-release-x86_64-nightly_master', 'Win32-vs9-Shared',
'Win64-VS10', 'Mac10.7.3-gcc-release-x86_64-nightly_master',
'Win32-vs9-Static', 'Ubuntu-Valgrind']
vtkInteractionStyleTcl-TestInteractorStyleTerrain , 7 , [...]
vtkRenderingOpenGLCxx-TestGaussianBlurPass , 7 , [...]
vtkCommonCore-TestXMLFileOutputWindow , 7 , [...]
vtkRenderingOpenGLCxx-TestTranslucentLUTDepthPeelingPass , 7 , [...]
! vtkRenderingVolumeTcl-gaussian , 10 , [...]
! vtkIOGeometryTcl-TestXYZMol , 10 , [...]
vtkIOSQLCxx-SQLiteTableReadWrite , 10 , [...]
vtkRenderingOpenGLCxx-TestOffAxisStereo , 14 , [...]
! vtkDomainsChemistryCxx-TestProteinRibbon , 19 , [...]

Where the ones I've marked with ! have been fixed and thus should be
off the most listed list tonight.

David E DeMarle
Kitware, Inc.
R&D Engineer
21 Corporate Drive
Clifton Park, NY 12065-8662
Phone: 518-881-4909
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tfails.py
Type: application/octet-stream
Size: 985 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20121005/5e46e6bc/attachment-0001.obj>