[vtk-developers] Failing dashboard tests

[Burlen Loring]

> There are 28 failing tests. Marcus agrees that we can add a new 
> baseline for the TestLinePlotColors test. What about the other tests? 
> Are they failing because of graphics card issues? Should they be 
> suppressed? Or are there changes in VTK code required? 
you need to look at each failing test case by case. I think it's worth 
looking for bugs in the failed tests. You may find VTK bugs even if it's 
a driver issue. It's a time consuming exercise and many of these are 
going to be ATI driver issues. There are a couple of other ATI 
dashboards that you could use for comparison, the two that I'm aware of 
are tarvalon and rocky.

The way to skip a rendering test for specific hardware such as an ATI 
device is to use the classes IsSupported method from the ctest. If there 
isn't one implemented then you should add it. When implementing this 
method you should check for GetIgnoreDriverBugs extension manager flag 
so that when updating drivers it's easy to verify if driver issue has 
been fixed or not. In your ctest call IsSupported, if it returns false 
the practice is to let the test pass. It would be far better if the test 
could return a "not run" code for the dashboard but this isn't supported 
by ctest.

Here's an example of how to remove support because of an ATI driver bug, 
taken from the projected tetrahedra volume rendering algorithm. Note, 
that Apple's OpenGL drivers for ATI devices are much better than ATI's 
so be careful not disable things unnecessarily on the Mac.:-)

   // exclude ATI Radeon HD, except on Apple, because there seems to
   // be a bug in ATI's legacy fixed function texturing support in recent
   // drivers. The Radeon HD cards are identified here by the OpenGL version
   // because the renderer string is inconsistent across platforms.
   // win7 RHD5800, win7 RHD6750M, Linux RHD7870 all exhibit the bug.
   bool driverSupport
   #if defined(__APPLE__)
     = true;
   #else
     = !(extensions->DriverIsATI()
     && (extensions->GetDriverGLVersionMajor() >= 3))
     || extensions->GetIgnoreDriverBugs("ATI texturing bug");
   #endif


On 11/22/2013 2:16 PM, Sankhesh Jhaveri wrote:
> Hello all,
>
> I am in the process of setting up a new dashboard on a machine with an 
> ATI graphics card. The results of a nightly dashboard submission are 
> here: http://open.cdash.org/viewTest.php?onlyfailed&buildid=3109741 . 
> There are 28 failing tests. Marcus agrees that we can add a new 
> baseline for the TestLinePlotColors test. What about the other tests? 
> Are they failing because of graphics card issues? Should they be 
> suppressed? Or are there changes in VTK code required?
>
> Almost half of the tests are failing because the vtkMPIController is 
> leaking. With George's help, we were able to ascertain that MPI is 
> indeed starting two different test processes(thats why two output 
> messages) but does not account for the two processes and exits with an 
> error. The question is, whether it is valid to assume that there is an 
> issue with the VTK MPI test code or is there a flag/setting in CMake 
> that needs to be changed?
>
> List of tests failing on the aurora dashboard:
>
> Tests failing because of image differences:
>
>   * Paralleliso-image
>   * vtkChartsCorePython-TestLinePlotColors
>   * vtkFiltersModelingPython-TestSpherePuzzleArrows
>   * vtkIOParallelLSDynaCxx-MPI-PLSDynaReader
>   * vtkRenderingOpenGLCxx-TestLabelPlacer
>   * vtkRenderingOpenGLCxx-TestShadowMapPass
>   * vtkRenderingOpenGLCxx-TestTilingCxx
>   * vtkRenderingVolumeCxx-TestGPURayCastDataTypesMIP
>   * vtkRenderingVolumeCxx-TestHAVSVolumeMapper
>   * vtkRenderingVolumePython-TestFixedPointRayCasterLinearCropped
>   * vtkRenderingVolumePython-volTM3DCompressedCropRegions
>   * vtkViewsInfovisCxx-TestRenderView
>
>
> Tests failing due to a leaking MPIController issue:
>
>   * vtkFiltersParallelCxx-MPI-DistributedData
>   * vtkFiltersParallelCxx-MPI-DistributedDataRenderPass
>   * vtkFiltersParallelCxx-MPI-TransmitImageData
>   * vtkFiltersParallelCxx-MPI-TransmitImageDataRenderPass
>   * vtkFiltersParallelCxx-MPI-TransmitRectilinearGrid
>   * vtkFiltersParallelCxx-MPI-TransmitStructuredGrid
>   * vtkFiltersParallelFlowPathsCxx-TestPParticleTracers
>   * vtkFiltersParallelFlowPathsCxx-TestPStreamAMR
>   * vtkFiltersParallelFlowPathsCxx-TestPStreamGeometry
>   * vtkParallelMPICxx-MPI-GenericCommunicator
>   * vtkParallelMPICxx-MPI-TestDistributedDataShadowMapPass
>   * vtkParallelMPICxx-MPI-TestNonBlockingCommunication
>   * vtkParallelMPICxx-MPI-TestProcess
>   * vtkRenderingParallelCxx-MPI-TestDistributedDataCompositeZPass
>   * vtkRenderingParallelCxx-MPI-TestPCompositeZPass
>   * vtkRenderingParallelCxx-MPI-TestPShadowMapPass
>
>
> Thanks.
> -- 
> Sankhesh Jhaveri
> Research & Development Engineer
> Kitware, Inc.
> 28 Corporate Drive
> Clifton Park, NY 12065-8662
> Phone: 518-881-4417
>
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://www.vtk.org/mailman/listinfo/vtk-developers
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20131123/ab5eb02b/attachment.html>