[Insight-developers] Re: Seeking advice about long running tests

[David Cole]

> I think it is OK to drop the iterations in the Debug tests. We are still exercising the code. If we

> need alternate baselines, we will still catch exceptions. This will also allow more valgrind tests

> to run to completion.




Ha ha. Ironic. I just noticed the slowest 3 tests have the word “Fast” in the test name... Perhaps we should at the very least consider a rename. 😊


If it’s ok to drop the iterations in the Debug tests, then why not drop the iterations in the Release tests, too? There is something to be said for the “same” code being run for the test in both Debug and Release. If it’s not the same, then I should definitely be able to tell just by glancing at the source code for a test that there are Debug/non-Debug differences...


I would say each individual test should be able to run in *seconds*, not minutes or hours. Ideally, less than a second, but I realize that’s overly ambitious with some ITK code.


However, for the main purpose I have in mind here, namely reducing the time burden on my volunteer machine, I would be absolutely *ecstatic* if all of the 20 tests I listed in the original email were able to run in under 4 or 5 minutes each.


Let me know if I can help out in some additional way (beyond just continuing to submit the build to CDash...)



Thanks for the discussion, I appreciate it

D
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/pipermail/insight-developers/attachments/20130802/1fe1c9c5/attachment.htm>