[Insight-developers] Seeking advice about long running tests

[Bradley Lowekamp]

David,

I have looked at the FastMarching just a little and the filter needs some work, and the test could be paired down.

Also this filter makes heavy use of stl object, and I don't think its done in the best way. That is combined with the "feature" MSVC adding a lot of useful debug checking in the STL library make it so slow in debug mode. For example the stl iterator method  has an expected cost of O(1.5N) [1].

I think it should be easy enough to pair these tests down some! It would also be good to profile this filter to know where to start looking. 

https://github.com/Kitware/ITK/blob/master/Modules/Core/Common/include/itkThreadedIteratorRangePartitioner.hxx#L53

Brad

On Aug 2, 2013, at 6:17 AM, David Cole <dlrdave at aol.com> wrote:

> Hi there.
>  
> First of all: kudos to all of you on dashboard progress, and fixing issues with Windows Debug builds. I have now submitted my first ever entirely green dashboard row using the Microsoft compiler in a Debug build for ITK!
>  
> However...... 😊
>  
> I recently started submitting ITK dashboards from “mug.neocisinc” and observe the following today:
>  
> - 20 longest running ITK tests (today):
> itkFastMarchingImageFilterTest_wm_multipleSeeds_StrictTopo 2h 37m 54s 70ms
> itkFastMarchingImageFilterTest_wm_multipleSeeds_NoHandlesTopo 2h 37m 28s 440ms
> itkFastMarchingImageFilterTest_wm_multipleSeeds_NoTopo 2h 27m 34s 760ms
> itkBSplineExponentialImageRegistrationTest 22m 9s 470ms
> itkExponentialImageRegistrationTest 19m 55s 840ms
> itkFastMarchingImageFilterTest_torus_multipleSeeds_NoHandlesTopo 13m 37s 950ms
> itkFastMarchingImageFilterTest_torus_multipleSeeds_StrictTopo 13m 34s 700ms
> itkSimpleImageRegistrationTest 12m 16s 690ms
> itkSimpleImageRegistrationTest2 11m 32s 560ms
> itkFastMarchingImageFilterTest_torus_multipleSeeds_NoTopo 11m 4s 70ms
> itkTimeVaryingVelocityFieldImageRegistrationTest 8m 12s 240ms
> SpatialObjectToImage2 7m 4s 710ms
> ImageRegistration15Test 5m 6s 410ms
> ImageRegistration14Test 3m 56s
> itkParameterizationQuadEdgeMeshFilterTestb0_c3_s0 3m 47s 950ms
> itkParameterizationQuadEdgeMeshFilterTestb1_c3_s0 3m 47s 150ms
> DeformableRegistration13Test1 3m 3s 480ms
> itkTimeVaryingBSplineVelocityFieldImageRegistrationTest 2m 54s 310ms
> itkSyNImageRegistrationTest 2m 38s 180ms
> DeformableRegistration12Test1 2m 27s 350ms
> 
>  
> (from this Debug build using the Microsoft Visual Studio 2012 compiler: http://open.cdash.org/buildSummary.php?buildid=2985530 )
>  
> Does anybody in this community have any advice about how I can get the Debug dashboard test suite to run in a more reasonable amount of time? (Other than simply excluding these 20 tests or so by name?)
>  
> It seems completely ridiculous to me that I have to set a test timeout value of greater than 10 minutes (let alone 2.5 HOURS) for an individual test just for the 10 longest running ITK tests here...... especially when the ENTIRE TEST SUITE finishes in less than 6 minutes for the Release build.
>  
> Can these tests be pared down to use smaller data, or can they be marked as requiring seriously long timeout values using the test TIMEOUT property, or can they be optionally added by people willing to test such long running tests, or perhaps optionally avoided by people like me who are reluctant to run such long running tests .... ?? There’s got to be something we can do.
>  
>  
> Thanks,
> David C.
>  
> _______________________________________________
> Powered by www.kitware.com
> 
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
> 
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.php
> 
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
> 
> Follow this link to subscribe/unsubscribe:
> http://www.itk.org/mailman/listinfo/insight-developers

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/pipermail/insight-developers/attachments/20130802/38217b07/attachment.htm>