VTK/SoftwareQuality/CodeCoverage/Winter2014: Difference between revisions

From KitwarePublic
< VTK
Jump to navigationJump to search
(Added named components back in)
 
(11 intermediate revisions by 3 users not shown)
Line 16: Line 16:


==Measure==
==Measure==
As of January 2014 the vtk tests covered 67% of vtk code.
As of January 2014 the vtk tests covered 68% of vtk code.


[http://open.cdash.org/viewCoverage.php?buildid=3191187 Here is the current coverage]
[http://open.cdash.org/viewCoverage.php?buildid=3191187 Here is the current coverage]
Line 23: Line 23:


==Analyze==
==Analyze==
===Modules not shown on Karego-at's coverage ===
* AcceleratorsDax - TOFIX
* AcceleratorsPiston - TOFIX:
* CommonColor - optional, turned on
* CommonComputationalGeometry - OK, didn't show since coverage is high enough
* FiltersParallelFlowPaths - optional, turned on
* FiltersStatisticsGNUR - optional - TOFIX
* GUISupportMFC - windows only, need a windows coverage machine to get it
* GUISupportQtOpenGL - optional, turned on
* GUISupportQtSQL - optional, turned on
* GUISUpportQtWebkit - optional, turned on
* InfovisBoost - optional, turned on
* InfovisParallel - optional, turned on
* IOSQL - OK, didn't show since coverage is high enough
* IOVPIC - optional, turned on - but no tests - TOFIX
* ViewsQt - optional, and only one test - TOFIX
* WebCore - optional, turned on
* WebWebGLExporter - optional, turned on
* WebInstall, WebJavaScript, WebPython, WebApplications - optional, primarily not C++
* (+ * TestingCore, Thirdparty*, Utilities* and Wrapping* which are expected)
===Coverage by Module===
===Coverage by Module===
Modules with '''< 70% are bold'''.
* ChartsCore 70.06%
* ChartsCore 70.06%
* CommonCore 67.47%
* CommonCore '''67.47%'''
* CommonDataModel 71.67%
* CommonDataModel 71.67%
* CommonExecutionModel 75.99%
* CommonExecutionModel 75.99%
* CommonMath 77.97%
* CommonMath 77.97%
* CommonMisc 59.87%
* CommonMisc '''59.87%'''
* CommonSystem 40.27%
* CommonSystem '''40.27%'''
* CommonTransforms 74.19%
* CommonTransforms 74.19%
* DomainsChemistry 80.75%
* DomainsChemistry 80.75%
* FiltersAMR 70.15%
* FiltersAMR 70.15%
* FiltersCore 73.09%
* FiltersCore 73.09%
* FiltersExtraction 66.13%
* FiltersExtraction '''66.13%'''
* FiltersFlowPaths 67.21%
* FiltersFlowPaths '''67.21%'''
* FiltersGeneral 71.94%
* FiltersGeneral 71.94%
* FiltersGeneric 73.87%
* FiltersGeneric 73.87%
Line 42: Line 64:
* FiltersHybrid 72.44%
* FiltersHybrid 72.44%
* FiltersHyperTree 82.47%
* FiltersHyperTree 82.47%
* FiltersImaging 23.58%
* FiltersImaging '''23.58%'''
* FiltersModeling 85.38%
* FiltersModeling 85.38%
* FiltersParallel 86.72%
* FiltersParallel 86.72%
* FiltersParallelGeometry 96.70%
* FiltersParallelGeometry 96.70%
* FiltersParallelImaging 44.54%
* FiltersParallelImaging '''44.54%'''
* FiltersParallelMPI 91.98%
* FiltersParallelMPI 91.98%
* FiltersParallelStatistics 78.11%
* FiltersParallelStatistics 78.11%
* FiltersProgrammable 83.43%
* FiltersProgrammable 83.43%
* FiltersReebGraph 64.69%
* FiltersReebGraph '''64.69%'''
* FiltersSelection 67.15%
* FiltersSelection '''67.15%'''
* FiltersSMP 66.96%
* FiltersSMP '''66.96%'''
* FiltersSources 85.73%
* FiltersSources 85.73%
* FiltersStatistics 78.92%
* FiltersStatistics 78.92%
* FiltersTexture 86.99%
* FiltersTexture 86.99%
* FiltersVerdict 32.91%
* FiltersVerdict '''32.91%'''
* GeovisCore 51.15%
* GeovisCore '''51.15%'''
* GUISupportQt 20.59%
* GUISupportQt '''20.59%'''
* ImagingColor 78.00%
* ImagingColor 78.00%
* ImagingCore 81.98%
* ImagingCore 81.98%
Line 70: Line 92:
* ImagingStencil 88.96%
* ImagingStencil 88.96%
* InfovisBoostGraphAlgorithms 75.62%
* InfovisBoostGraphAlgorithms 75.62%
* InfovisCore 63.98%
* InfovisCore '''63.98%'''
* InfovisLayout 63.25%
* InfovisLayout '''63.25%'''
* InteractionImage 50.36%
* InteractionImage '''50.36%'''
* InteractionStyle 59.08%
* InteractionStyle '''59.08%'''
* InteractionWidgets 63.80%
* InteractionWidgets '''63.80%'''
* IOAMR 42.79%
* IOAMR '''42.79%'''
* IOCore 70.01%
* IOCore 70.01%
* IOEnSight 56.67%
* IOEnSight '''56.67%'''
* IOExodus 39.86% 5315
* IOExodus '''39.86%'''
* IOExport 64.75%
* IOExport '''64.75%'''
* IOFFMPEG 72.49%
* IOFFMPEG 72.49%
* IOGDAL 83.33%
* IOGDAL 83.33%
* IOGeoJSON 83.47%
* IOGeoJSON 83.47%
* IOGeometry 41.97%
* IOGeometry '''41.97%'''
* IOImage 59.92%
* IOImage '''59.92%'''
* IOImport 64.60%
* IOImport '''64.60%'''
* IOInfovis 79.99%
* IOInfovis 79.99%
* IOLegacy 55.64%
* IOLegacy '''55.64%'''
* IOLSDyna 8.55%
* IOLSDyna '''8.55%'''
* IOMINC 76.37%
* IOMINC 76.37%
* IOMovie 72.13%
* IOMovie 72.13%
* IOMPIImage 49.34%
* IOMPIImage '''49.34%'''
* IOMPIParallel 9.09%
* IOMPIParallel '''9.09%'''
* IOMySQL 18.02%
* IOMySQL '''18.02%'''
* IONetCDF 38.85%
* IONetCDF '''38.85%'''
* IOODBC 11.24%
* IOODBC '''11.24%'''
* IOParallel 35.47%
* IOParallel '''35.47%'''
* IOParallelExodus 14.80%
* IOParallelExodus '''14.80%'''
* vtkIOParallelLSDyna 38.20%
* IOParallelLSDyna '''38.20%'''
* IOParallelNetCDF 19.03%
* IOParallelNetCDF '''19.03%'''
* IOPLY 59.97%
* IOPLY '''59.97%'''
* IOPostgreSQL 16.58%
* IOPostgreSQL '''16.58%'''
* IOVideo 26.82%
* IOVideo '''26.82%'''
* IOXdmf2 54.18%
* IOXdmf2 '''54.18%'''
* IOXML 72.98%
* IOXML 72.98%
* IOXMLParser 57.08%
* IOXMLParser '''57.08%'''
* ParallelCore 59.51%
* ParallelCore '''59.51%'''
* ParallelMPI 64.58%
* ParallelMPI '''64.58%'''
* RenderingAnnotation 81.53%
* RenderingAnnotation 81.53%
* RenderingContext2D 65.87%
* RenderingContext2D '''65.87%'''
* RenderingCore 75.11%
* RenderingCore 75.11%
* RenderingFreeType 62.73%
* RenderingFreeType '''62.73%'''
* RenderingFreeTypeFontConfig 70.48%
* RenderingFreeTypeFontConfig 70.48%
* RenderingFreeTypeOpenGL 82.92%
* RenderingFreeTypeOpenGL 82.92%
* RenderingGL2PS 65.16%
* RenderingGL2PS '''65.16%'''
* RenderingImage 85.59%
* RenderingImage 85.59%
* RenderingLabel 59.59%
* RenderingLabel '''59.59%'''
* RenderingLIC 81.59%
* RenderingLIC 81.59%
* RenderingLOD 74.25%
* RenderingLOD 74.25%
* RenderingMatplotlib 80.76%
* RenderingMatplotlib 80.76%
* RenderingOpenGL 56.65%
* RenderingOpenGL '''56.65%'''
* RenderingParallel 61.49%
* RenderingParallel '''61.49%'''
* RenderingParallelLIC 7.89%
* RenderingParallelLIC '''7.89%'''
* RenderingQt 18.77%
* RenderingQt '''18.77%'''
* RenderingTk 41.89%
* RenderingTk '''41.89%'''
* RenderingVolume 85.46%
* RenderingVolume 85.46%
* RenderingVolumeAMR 38.83%
* RenderingVolumeAMR '''38.83%'''
* RenderingVolumeOpenGL 68.88%
* RenderingVolumeOpenGL '''68.88%'''
* TestingGenericBridge 67.45%
* TestingGenericBridge '''67.45%'''
* TestingIOSQL 90.91%
* TestingIOSQL 90.91%
* TestingRendering 63.31%
* TestingRendering '''63.31%'''
* ViewsContext2D 81.30%
* ViewsContext2D 81.30%
* ViewsCore 59.15%
* ViewsCore '''59.15%'''
* ViewsGeovis 64.41%
* ViewsGeovis '''64.41%'''
* ViewsInfovis 64.06%
* ViewsInfovis '''64.06%'''
 
===Low hanging Fruit===
===Low hanging Fruit===
Tests that were lost after modularization
Tests that were lost after modularization
Line 157: Line 180:
* ./Common/Core/Testing/Cxx/CMakeLists.txt:  #otherLookupTable.cxx
* ./Common/Core/Testing/Cxx/CMakeLists.txt:  #otherLookupTable.cxx
* ./Common/Core/Testing/Cxx/CMakeLists.txt:  #otherLookupTableWithEnabling.cxx
* ./Common/Core/Testing/Cxx/CMakeLists.txt:  #otherLookupTableWithEnabling.cxx
* ./Common/DataModel/Testing/Cxx/CellLocator.cxx
* ./Common/DataModel/Testing/Cxx/PointLocator.cxx
* ./Common/DataModel/Testing/Cxx/TestGraph2.cxx
* ./Common/DataModel/Testing/Cxx/TestGraphAttributes.cxx
* ./Common/DataModel/Testing/Cxx/TestMeanValueCoordinatesInterpolation1.cxx
* ./Common/DataModel/Testing/Cxx/TestMeanValueCoordinatesInterpolation2.cxx
* ./Common/DataModel/Testing/Cxx/TestSmoothErrorMetric.cxx
* ./Common/DataModel/Testing/Cxx/TestTable.cxx
* ./Common/Transforms/Testing/Python/SliceOrder.py
* ./Common/Transforms/Testing/Python/SliceOrder.py
* ./Domains/Chemistry/Testing/Cxx/TestOpenQubeElectronicData.cxx
* ./Domains/Chemistry/Testing/Cxx/TestOpenQubeElectronicData.cxx
* ./Domains/Chemistry/Testing/Cxx/TestOpenQubeMOPACDensity.cxx
* ./Domains/Chemistry/Testing/Cxx/TestOpenQubeMOPACDensity.cxx
* ./Domains/Chemistry/Testing/Cxx/TestOpenQubeMOPACOrbital.cxx
* ./Domains/Chemistry/Testing/Cxx/TestOpenQubeMOPACOrbital.cxx
* ./Filters/Core/Testing/Cxx/TestNamedComponents.cxx
* ./Filters/General/Testing/Cxx/ArrayExtractFactoredArray.cxx
* ./Filters/General/Testing/Cxx/ArrayMatricizeArray.cxx
* ./Filters/General/Testing/Cxx/ArrayNormalizeMatrixVectors.cxx
* ./Filters/General/Testing/Cxx/CellTreeLocator.cxx
* ./Filters/General/Testing/Cxx/TestPassArrays.cxx
* ./Filters/General/Testing/Cxx/TestPassThrough.cxx
* ./Filters/General/Testing/Cxx/TestTessellator.cxx
* ./Filters/General/Testing/Cxx/expCos.cxx
* ./Filters/Geometry/Testing/Cxx/TestRectilinearGridPartitioner.cxx
* ./Filters/Geometry/Testing/Cxx/TestRectilinearGridPartitioner.cxx
* ./Filters/Geometry/Testing/Cxx/TestStructuredGridPartitioner.cxx
* ./Filters/Geometry/Testing/Cxx/TestStructuredGridPartitioner.cxx
Line 201: Line 205:
* ./GUISupport/Qt/Testing/Python/TestTimeUtility.py
* ./GUISupport/Qt/Testing/Python/TestTimeUtility.py
* ./GUISupport/Qt/Testing/Python/TestvtkQtTableView.py
* ./GUISupport/Qt/Testing/Python/TestvtkQtTableView.py
* ./GUISupport/QtSQL/Testing/Cxx/TestQtSQLDatabase.cxx
* ./IO/AMR/Testing/Cxx/TestAMRReadWrite.cxx
* ./IO/AMR/Testing/Cxx/TestAMRReadWrite.cxx
* ./IO/AMR/Testing/Cxx/TestFlashReader.cxx
* ./IO/AMR/Testing/Cxx/TestFlashReader.cxx
Line 225: Line 228:
* ./IO/XML/Testing/Tcl/TestXMLUnstructuredGridIO.tcl
* ./IO/XML/Testing/Tcl/TestXMLUnstructuredGridIO.tcl
* ./Imaging/Core/Testing/Python/WindowLevelInterface.py
* ./Imaging/Core/Testing/Python/WindowLevelInterface.py
* ./Imaging/Core/Testing/Cxx/CMakeLists.txt:  ImageAutoRange.cxx
* ./Imaging/Core/Testing/Cxx/CMakeLists.txt:  ImageHistogram.cxx
* ./Imaging/Core/Testing/Cxx/CMakeLists.txt:  ImageHistogramStatistics.cxx,NO_VALID
* ./Imaging/Core/Testing/Cxx/CMakeLists.txt:  ImageResize.cxx
* ./Imaging/Core/Testing/Cxx/CMakeLists.txt:  ImageResize3D.cxx
* ./Imaging/Core/Testing/Cxx/CMakeLists.txt:  ImageResizeCropping.cxx
* ./Imaging/Core/Testing/Cxx/CMakeLists.txt:  ImportExport.cxx,NO_VALID # todo (unsatistfied deps)
* ./Imaging/Core/Testing/Cxx/ImageAutoRange.cxx
* ./Imaging/Core/Testing/Cxx/ImageHistogram.cxx
* ./Imaging/Core/Testing/Cxx/ImageHistogramStatistics.cxx
* ./Imaging/Core/Testing/Cxx/ImageResize.cxx
* ./Imaging/Core/Testing/Cxx/ImageResize3D.cxx
* ./Imaging/Core/Testing/Cxx/ImageResizeCropping.cxx
* ./Imaging/Core/Testing/Cxx/ImportExport.cxx
** ./Imaging/Core/Testing/Python/CMakeLists.txt:# vtk_add_test_python(WindowLevelInterface.py) # TCL test not being made as well !!!
** ./Imaging/Core/Testing/Python/CMakeLists.txt:# vtk_add_test_python(WindowLevelInterface.py) # TCL test not being made as well !!!
* ./Infovis/BoostGraphAlgorithms/Testing/Cxx/BoostArrayLogWeighting.cxx
* ./Infovis/BoostGraphAlgorithms/Testing/Cxx/BoostArrayLogWeighting.cxx
Line 292: Line 281:
* ./Rendering/Annotation/Testing/Cxx/TestAxisActor3D.cxx
* ./Rendering/Annotation/Testing/Cxx/TestAxisActor3D.cxx
** ./Rendering/Annotation/Testing/Cxx/CMakeLists.txt:  TestAxisActor3D.cxx
** ./Rendering/Annotation/Testing/Cxx/CMakeLists.txt:  TestAxisActor3D.cxx
* ./Rendering/Core/Testing/Cxx/FrustumClip.cxx
* ./Rendering/Core/Testing/Cxx/Mace.cxx
* ./Rendering/Core/Testing/Cxx/RGrid.cxx
* ./Rendering/Core/Testing/Cxx/TestActor2DTextures.cxx
* ./Rendering/Core/Testing/Cxx/TestBareScalarsToColors.cxx
* ./Rendering/Core/Testing/Cxx/TestHierarchicalBoxPipeline.cxx
* ./Rendering/Core/Testing/Cxx/TestHierarchicalBoxPipeline.cxx
** ./Rendering/Core/Testing/Cxx/CMakeLists.txt:  # TestHierarchicalBoxPipeline.cxx  # legacy code
** ./Rendering/Core/Testing/Cxx/CMakeLists.txt:  # TestHierarchicalBoxPipeline.cxx  # legacy code
* ./Rendering/Core/Testing/Cxx/TestMapVectorsAsRGBColors.cxx
* ./Rendering/Core/Testing/Cxx/TestMapVectorsToColors.cxx
* ./Rendering/Core/Testing/Cxx/TestTDxGeo.cxx
* ./Rendering/Core/Testing/Cxx/TestTDxGeo.cxx
* ./Rendering/Core/Testing/Cxx/TestViewDependentErrorMetric.cxx
* ./Rendering/Core/Testing/Cxx/TestViewDependentErrorMetric.cxx
* ./Rendering/Core/Testing/Cxx/otherLookupTable.cxx
** ./Rendering/Core/Testing/Cxx/CMakeLists.txt:  # otherLookupTable.cxx            # fixme
* ./Rendering/Core/Testing/Cxx/otherLookupTableWithEnabling.cxx
** ./Rendering/Core/Testing/Cxx/CMakeLists.txt:  # otherLookupTableWithEnabling.cxx # fixme
* ./Rendering/Core/Testing/Python/TestParallelCoordinates.py
* ./Rendering/Core/Testing/Python/TestParallelCoordinates.py
** ./Rendering/Core/Testing/Python/CMakeLists.txt:# vtk_add_test_python(TestParallelCoordinates.py) # disabled in TCL too
** ./Rendering/Core/Testing/Python/CMakeLists.txt:# vtk_add_test_python(TestParallelCoordinates.py) # disabled in TCL too
Line 323: Line 301:


==Improve==
==Improve==
We should document these much better.
* Cookie cutter approach to add a test is …
* Useful tools to call from your test are ...


==Control==
==Control==


This is a [http://vtk.org/gitweb?p=VTK.git;a=blob_plain;f=Utilities/Maintenance/computeCodeCoverageLocallyForOneTest.sh;hb=HEAD useful script for computing code coverage]. Currently only works on linux.
This is a [http://vtk.org/gitweb?p=VTK.git;a=blob_plain;f=Utilities/Maintenance/computeCodeCoverageLocallyForOneTest.sh;hb=HEAD useful script for computing code coverage]. Currently only works on linux.

Latest revision as of 21:23, 28 January 2014

Testing is critical to high quality software. There are a number of aspects to the VTK's software quality. One important aspect is code coverage that measures how many lines of code are covered by the test suite.

This experiment seeks to increase VTK's code coverage.

This experiment uses the DMAIC methodology of the Six Sigma management process to "Define", "Measure", "Analyze", "Improve" and "Control" test code coverage in VTK.

The basic methodology (from Wikipedia) consists of the following five steps:

  • Define process goals that are consistent with customer demands and VTK's strategy.
  • Measure key aspects of the current process and collect relevant data.
  • Analyze the data to verify cause-and-effect relationships. Determine what the relationships are, and attempt to ensure that all factors have been considered.
  • Improve or optimize the process.
  • Control to ensure that any deviations from target are corrected before they result in defects. Set up pilot runs to establish software quality, move on to production, set up control mechanisms and continuously monitor the process.

Define

Increase the code coverage of vtk classes.

Measure

As of January 2014 the vtk tests covered 68% of vtk code.

Here is the current coverage

A summary of coverages by module

Analyze

Modules not shown on Karego-at's coverage

  • AcceleratorsDax - TOFIX
  • AcceleratorsPiston - TOFIX:
  • CommonColor - optional, turned on
  • CommonComputationalGeometry - OK, didn't show since coverage is high enough
  • FiltersParallelFlowPaths - optional, turned on
  • FiltersStatisticsGNUR - optional - TOFIX
  • GUISupportMFC - windows only, need a windows coverage machine to get it
  • GUISupportQtOpenGL - optional, turned on
  • GUISupportQtSQL - optional, turned on
  • GUISUpportQtWebkit - optional, turned on
  • InfovisBoost - optional, turned on
  • InfovisParallel - optional, turned on
  • IOSQL - OK, didn't show since coverage is high enough
  • IOVPIC - optional, turned on - but no tests - TOFIX
  • ViewsQt - optional, and only one test - TOFIX
  • WebCore - optional, turned on
  • WebWebGLExporter - optional, turned on
  • WebInstall, WebJavaScript, WebPython, WebApplications - optional, primarily not C++
  • (+ * TestingCore, Thirdparty*, Utilities* and Wrapping* which are expected)

Coverage by Module

Modules with < 70% are bold.

  • ChartsCore 70.06%
  • CommonCore 67.47%
  • CommonDataModel 71.67%
  • CommonExecutionModel 75.99%
  • CommonMath 77.97%
  • CommonMisc 59.87%
  • CommonSystem 40.27%
  • CommonTransforms 74.19%
  • DomainsChemistry 80.75%
  • FiltersAMR 70.15%
  • FiltersCore 73.09%
  • FiltersExtraction 66.13%
  • FiltersFlowPaths 67.21%
  • FiltersGeneral 71.94%
  • FiltersGeneric 73.87%
  • FiltersGeometry 78.51%
  • FiltersHybrid 72.44%
  • FiltersHyperTree 82.47%
  • FiltersImaging 23.58%
  • FiltersModeling 85.38%
  • FiltersParallel 86.72%
  • FiltersParallelGeometry 96.70%
  • FiltersParallelImaging 44.54%
  • FiltersParallelMPI 91.98%
  • FiltersParallelStatistics 78.11%
  • FiltersProgrammable 83.43%
  • FiltersReebGraph 64.69%
  • FiltersSelection 67.15%
  • FiltersSMP 66.96%
  • FiltersSources 85.73%
  • FiltersStatistics 78.92%
  • FiltersTexture 86.99%
  • FiltersVerdict 32.91%
  • GeovisCore 51.15%
  • GUISupportQt 20.59%
  • ImagingColor 78.00%
  • ImagingCore 81.98%
  • ImagingFourier 87.23%
  • ImagingGeneral 88.53%
  • ImagingHybrid 85.30%
  • ImagingMath 83.88%
  • ImagingMorphological 92.12%
  • ImagingSources 81.12%
  • ImagingStatistics 89.08%
  • ImagingStencil 88.96%
  • InfovisBoostGraphAlgorithms 75.62%
  • InfovisCore 63.98%
  • InfovisLayout 63.25%
  • InteractionImage 50.36%
  • InteractionStyle 59.08%
  • InteractionWidgets 63.80%
  • IOAMR 42.79%
  • IOCore 70.01%
  • IOEnSight 56.67%
  • IOExodus 39.86%
  • IOExport 64.75%
  • IOFFMPEG 72.49%
  • IOGDAL 83.33%
  • IOGeoJSON 83.47%
  • IOGeometry 41.97%
  • IOImage 59.92%
  • IOImport 64.60%
  • IOInfovis 79.99%
  • IOLegacy 55.64%
  • IOLSDyna 8.55%
  • IOMINC 76.37%
  • IOMovie 72.13%
  • IOMPIImage 49.34%
  • IOMPIParallel 9.09%
  • IOMySQL 18.02%
  • IONetCDF 38.85%
  • IOODBC 11.24%
  • IOParallel 35.47%
  • IOParallelExodus 14.80%
  • IOParallelLSDyna 38.20%
  • IOParallelNetCDF 19.03%
  • IOPLY 59.97%
  • IOPostgreSQL 16.58%
  • IOVideo 26.82%
  • IOXdmf2 54.18%
  • IOXML 72.98%
  • IOXMLParser 57.08%
  • ParallelCore 59.51%
  • ParallelMPI 64.58%
  • RenderingAnnotation 81.53%
  • RenderingContext2D 65.87%
  • RenderingCore 75.11%
  • RenderingFreeType 62.73%
  • RenderingFreeTypeFontConfig 70.48%
  • RenderingFreeTypeOpenGL 82.92%
  • RenderingGL2PS 65.16%
  • RenderingImage 85.59%
  • RenderingLabel 59.59%
  • RenderingLIC 81.59%
  • RenderingLOD 74.25%
  • RenderingMatplotlib 80.76%
  • RenderingOpenGL 56.65%
  • RenderingParallel 61.49%
  • RenderingParallelLIC 7.89%
  • RenderingQt 18.77%
  • RenderingTk 41.89%
  • RenderingVolume 85.46%
  • RenderingVolumeAMR 38.83%
  • RenderingVolumeOpenGL 68.88%
  • TestingGenericBridge 67.45%
  • TestingIOSQL 90.91%
  • TestingRendering 63.31%
  • ViewsContext2D 81.30%
  • ViewsCore 59.15%
  • ViewsGeovis 64.41%
  • ViewsInfovis 64.06%

Low hanging Fruit

Tests that were lost after modularization

From: David Gobbi <david.gobbi@gmail.com>
Date: Sat, Mar 2, 2013 at 4:10 PM
Subject: Re: [vtk-developers] Coverage
To: VTK Developers <vtk-developers@vtk.org>
Here's a list of tests that are disabled in the master branch,
that were active in VTK 5.10.1.  It should give an idea of the
amount of coverage that has been lost since the last release.
  • ./Accelerators/Piston/Testing/Python/PistonTestCommon.py
  • ./Charts/Core/Testing/Cxx/TestGLSL.cxx
  • ./Common/Core/Testing/Cxx/TestArrayCasting.cxx
    • ./Common/Core/Testing/Cxx/CMakeLists.txt: # TestArrayCasting.cxx # Uses Boost in its own separate test.
  • ./Common/Core/Testing/Cxx/TestCxxFeatures.cxx
    • ./Common/Core/Testing/Cxx/CMakeLists.txt: # TestCxxFeatures.cxx # This is in its own exe too.
  • ./Common/Core/Testing/Cxx/TestInstantiator.cxx
    • ./Common/Core/Testing/Cxx/CMakeLists.txt: # TestInstantiator.cxx # Have not enabled instantiators.
  • ./Common/Core/Testing/Cxx/TestVariantArray.cxx
  • ./Common/Core/Testing/Cxx/CMakeLists.txt: #otherLookupTable.cxx
  • ./Common/Core/Testing/Cxx/CMakeLists.txt: #otherLookupTableWithEnabling.cxx
  • ./Domains/Chemistry/Testing/Cxx/TestOpenQubeElectronicData.cxx
  • ./Domains/Chemistry/Testing/Cxx/TestOpenQubeMOPACDensity.cxx
  • ./Domains/Chemistry/Testing/Cxx/TestOpenQubeMOPACOrbital.cxx
  • ./Filters/Geometry/Testing/Cxx/TestRectilinearGridPartitioner.cxx
  • ./Filters/Geometry/Testing/Cxx/TestStructuredGridPartitioner.cxx
  • ./Filters/Hybrid/Testing/Cxx/TestBSplineWarp.cxx
    • ./Filters/Hybrid/Testing/Cxx/CMakeLists.txt: # TestBSplineWarp.cxx # Fixme after vtkImageViewer deps
  • ./Filters/Matlab/Testing/Cxx/TestMatlabEngineFilter.cxx
  • ./Filters/Matlab/Testing/Cxx/TestMatlabEngineInterface.cxx
  • ./Filters/ParallelStatistics/Testing/Cxx/TestRealDataPDescriptiveStatisticsMPI.cxx
  • ./Filters/Sources/Testing/Cxx/ArrayAPIDenseCoordinates.cxx
  • ./Filters/Sources/Testing/Cxx/ArrayDiagonalMatrixSource.cxx
  • ./Filters/Sources/Testing/Tcl/Disk.tcl
  • ./Filters/Sources/Testing/Tcl/OSCone.tcl
  • ./Filters/Sources/Testing/Tcl/TestButtonSource.tcl
  • ./Filters/Sources/Testing/Tcl/TestPlatonicSolids.tcl
  • ./Filters/Sources/Testing/Tcl/TestRegularPolygonSource.tcl
  • ./Filters/Sources/Testing/Tcl/coneResolution.tcl
  • ./Filters/Sources/Testing/Tcl/glyph2D.tcl
  • ./GUISupport/Qt/Testing/Python/TestConnection.py
  • ./GUISupport/Qt/Testing/Python/TestQVTKRenderWindowInteractor.py
  • ./GUISupport/Qt/Testing/Python/TestQVTKWidget.py
  • ./GUISupport/Qt/Testing/Python/TestQVTKWidget2.py
  • ./GUISupport/Qt/Testing/Python/TestTimeUtility.py
  • ./GUISupport/Qt/Testing/Python/TestvtkQtTableView.py
  • ./IO/AMR/Testing/Cxx/TestAMRReadWrite.cxx
  • ./IO/AMR/Testing/Cxx/TestFlashReader.cxx
    • ./IO/AMR/Testing/Cxx/CMakeLists.txt: # TestFlashReader.cxx there is not data for this test
  • ./IO/Geometry/Testing/Cxx/TestAMRReadWrite.cxx
    • ./IO/Geometry/Testing/Cxx/CMakeLists.txt: TestAMRReadWrite.cxx,NO_VALID
  • ./IO/Geometry/Testing/Cxx/TestDataObjectIO.cxx
  • ./IO/Geometry/Testing/Cxx/TestIncrementalOctreePointLocator.cxx
  • ./IO/Geometry/Testing/Cxx/UnstructuredGridCellGradients.cxx
  • ./IO/Geometry/Testing/Cxx/UnstructuredGridFastGradients.cxx
  • ./IO/Geometry/Testing/Cxx/UnstructuredGridGradients.cxx
  • ./IO/Geometry/Testing/Python/TestPolygonWriters.py
    • ./IO/Geometry/Testing/Python/CMakeLists.txt:# vtk_add_test_python(TestPolygonWriters.py) # needs vtkCGMWriter (not built)
  • ./IO/Image/Testing/Cxx/TestImageReader2Factory.cxx
    • ./IO/Image/Testing/Cxx/CMakeLists.txt: # TestImageReader2Factory.cxx # fixme (deps not satisfied)
  • ./IO/Infovis/Testing/Cxx/TestDelimitedTextReader.cxx
  • ./IO/Infovis/Testing/Cxx/TestTulipReader.cxx
  • ./IO/Infovis/Testing/Cxx/TestTulipReaderClusters.cxx
  • ./IO/Movie/Testing/Cxx/TestMovieWriter.cxx
    • ./IO/Movie/Testing/Cxx/CMakeLists.txt: # TestMovieWriter.cxx # fixme (deps not satisfied)
  • ./IO/SQL/Testing/Cxx/TestSQLGraphReader.cxx
  • ./IO/XML/Testing/Tcl/TestXMLPolyDataIO.tcl
  • ./IO/XML/Testing/Tcl/TestXMLUnstructuredGridIO.tcl
  • ./Imaging/Core/Testing/Python/WindowLevelInterface.py
    • ./Imaging/Core/Testing/Python/CMakeLists.txt:# vtk_add_test_python(WindowLevelInterface.py) # TCL test not being made as well !!!
  • ./Infovis/BoostGraphAlgorithms/Testing/Cxx/BoostArrayLogWeighting.cxx
    • ./Infovis/BoostGraphAlgorithms/Testing/Cxx/CMakeLists.txt: # BoostArrayLogWeighting.cxx # todo (unsatisfied deps)
  • ./Infovis/BoostGraphAlgorithms/Testing/Cxx/TestBoostBrandesCentrality.cxx
    • ./Infovis/BoostGraphAlgorithms/Testing/Cxx/CMakeLists.txt: # TestBoostBrandesCentrality.cxx # todo (unsatisfied deps)
  • ./Infovis/Core/Testing/Cxx/ArrayDotProductSimilarity.cxx
  • ./Infovis/Core/Testing/Cxx/TestArrayNorm.cxx
    • ./Infovis/Core/Testing/Cxx/CMakeLists.txt: # TestArrayNorm.cxx # todo (unsatisfied deps)
  • ./Infovis/Core/Testing/Cxx/TestConvexHull2D.cxx
    • ./Infovis/Core/Testing/Cxx/CMakeLists.txt: # TestConvexHull2D.cxx # todo (unsatisfied deps)
  • ./Infovis/Core/Testing/Cxx/TestKCoreDecomposition.cxx
  • ./Infovis/Core/Testing/Cxx/TestPruneTreeFilter.cxx
    • ./Infovis/Core/Testing/Cxx/CMakeLists.txt: TestPruneTreeFilter.cxx
  • ./Infovis/Core/Testing/Cxx/TestStringToCategory.cxx
  • ./Infovis/Core/Testing/Cxx/TestTableToGraph.cxx
  • ./Infovis/Core/Testing/Cxx/TestTimePoint.cxx
    • ./Infovis/Core/Testing/Cxx/CMakeLists.txt: # TestTimePoint.cxx # deprecated dep
  • ./Infovis/Layout/Testing/Cxx/TestArcEdges.cxx
  • ./Infovis/Layout/Testing/Cxx/TestAssignCoordinates.cxx
    • ./Infovis/Layout/Testing/Cxx/CMakeLists.txt: # TestAssignCoordinates.cxx # todo (unsatisfied deps)
  • ./Infovis/Layout/Testing/Cxx/TestChacoGraphReader.cxx
    • ./Infovis/Layout/Testing/Cxx/CMakeLists.txt: TestChacoGraphReader.cxx
  • ./Infovis/Layout/Testing/Cxx/TestCosmicTreeLayoutStrategy.cxx
    • ./Infovis/Layout/Testing/Cxx/CMakeLists.txt: # TestCosmicTreeLayoutStrategy.cxx # todo (unsatisfied deps)
  • ./Infovis/Layout/Testing/Cxx/TestGroupLeafVertices.cxx
  • ./Infovis/Layout/Testing/Cxx/TestKCoreLayout.cxx
  • ./Infovis/Layout/Testing/Cxx/TestKdTreeBoxSelection.cxx
  • ./Infovis/Layout/Testing/Cxx/TestPruneTreeFilter.cxx
  • ./Infovis/Layout/Testing/Cxx/TestSimple3DCirclesStrategy.cxx
  • ./Interaction/Style/Testing/Python/TestStyleBase.py
  • ./Interaction/Widgets/Testing/Cxx/ImagePlaneWidget.cxx
    • ./Interaction/Widgets/Testing/Cxx/CMakeLists.txt: # ImagePlaneWidget.cxx # todo (unsatisfied deps)
  • ./Interaction/Widgets/Testing/Cxx/TestBrokenLineWidget.cxx
    • ./Interaction/Widgets/Testing/Cxx/CMakeLists.txt: # TestBrokenLineWidget.cxx # todo (unsatisfied deps)
  • ./Interaction/Widgets/Testing/Cxx/TestButtonWidget.cxx
    • ./Interaction/Widgets/Testing/Cxx/CMakeLists.txt: # TestButtonWidget.cxx # todo (unsatistfied deps)
  • ./Interaction/Widgets/Testing/Cxx/TestImageActorContourWidget.cxx
    • ./Interaction/Widgets/Testing/Cxx/CMakeLists.txt: # TestImageActorContourWidget.cxx # todo(unsatistfied deps)
  • ./Interaction/Widgets/Testing/Cxx/TestOrthoPlanes.cxx
    • ./Interaction/Widgets/Testing/Cxx/CMakeLists.txt: # TestOrthoPlanes.cxx # todo (unsatistfied deps)
  • ./Interaction/Widgets/Testing/Cxx/TestResliceCursorWidget2.cxx
    • ./Interaction/Widgets/Testing/Cxx/CMakeLists.txt: # TestResliceCursorWidget2.cxx # todo (unsatistfied deps)
  • ./Interaction/Widgets/Testing/Cxx/TestResliceCursorWidget3.cxx
    • ./Interaction/Widgets/Testing/Cxx/CMakeLists.txt: # TestResliceCursorWidget3.cxx # todo (unsatistfied deps)
  • ./Interaction/Widgets/Testing/Cxx/TestSplineWidget.cxx
    • ./Interaction/Widgets/Testing/Cxx/CMakeLists.txt: # TestSplineWidget.cxx # todo (unsatistfied deps)
  • ./Interaction/Widgets/Testing/Cxx/vtkImplicitPlaneWidget2Test1.cxx
  • ./Parallel/Core/Testing/Python/DummyController.cxx
  • ./Parallel/Core/Testing/Python/SocketClient.cxx
  • ./Parallel/Core/Testing/Python/SocketServer.cxx
  • ./Parallel/Core/Testing/Python/TestSockets.py
  • ./Rendering/Annotation/Testing/Cxx/TestAxisActor3D.cxx
    • ./Rendering/Annotation/Testing/Cxx/CMakeLists.txt: TestAxisActor3D.cxx
  • ./Rendering/Core/Testing/Cxx/TestHierarchicalBoxPipeline.cxx
    • ./Rendering/Core/Testing/Cxx/CMakeLists.txt: # TestHierarchicalBoxPipeline.cxx # legacy code
  • ./Rendering/Core/Testing/Cxx/TestTDxGeo.cxx
  • ./Rendering/Core/Testing/Cxx/TestViewDependentErrorMetric.cxx
  • ./Rendering/Core/Testing/Python/TestParallelCoordinates.py
    • ./Rendering/Core/Testing/Python/CMakeLists.txt:# vtk_add_test_python(TestParallelCoordinates.py) # disabled in TCL too
  • ./Rendering/Core/Testing/Tcl/TestCgShader.tcl
  • ./Rendering/Core/Testing/Tcl/TestGLSLShader.tcl
  • ./Rendering/Core/Testing/Tcl/TestGenericVertexAttributesCg.tcl
  • ./Rendering/Core/Testing/Tcl/TestGenericVertexAttributesGLSL.tcl
  • ./Rendering/Core/Testing/Tcl/TestParallelCoordinates.tcl
  • ./Rendering/Qt/Testing/Cxx/TestQtInitialization.cxx
  • ./Rendering/Volume/Testing/Python/cursor3D.py
  • ./Rendering/Volume/Testing/Python/CMakeLists.txt:vtk_add_test_python(cursor3D.py)
  • ./Views/Geovis/Testing/Cxx/TestGeoView.cxx
  • ./Views/Geovis/Testing/Cxx/TestGeoView2D.cxx
  • ./Views/Infovis/Testing/Cxx/TestQtTreeRingLabeler.cxx
  • ./Views/Qt/Testing/Cxx/TestVtkQtTableView.cxx

Improve

We should document these much better.

  • Cookie cutter approach to add a test is …
  • Useful tools to call from your test are ...

Control

This is a useful script for computing code coverage. Currently only works on linux.