[Insight-developers] Cancelling Tag for today due to UnacceptableDashboard Status

Matt McCormick matt.mccormick at kitware.com
Tue Jan 31 20:46:24 EST 2012 

On Tue, Jan 31, 2012 at 8:01 PM, brian avants <stnava at gmail.com> wrote:
> Relatedly, there is a lack of testing for performance on difficult problems,
> at least in the registration part of the toolkit.  However, these tests
> might take longer than usual and involve larger datasets.  It would be great
> to have some such tests .... but they might create some backlash due to the
> the time penalty.   What is the current recommended solution?  Is there a
> place for "long" tests?
>

Adding tests for larger datasets or longer-running tests is a good
idea.  Perhaps we should start using a label to identify long tests::

  http://www.cmake.org/cmake/help/cmake-2-8-docs.html#prop_test:LABELS

That can be easily excluded or included when running ctest?

  http://www.cmake.org/cmake/help/ctest-2-8-docs.html#command:ctest_test
  http://www.cmake.org/cmake/help/ctest-2-8-docs.html#opt:-LEregex--label-excluderegex

> On Jan 31, 2012 7:51 PM, "brian avants" <stnava at gmail.com> wrote:
>>
>> I think the issue is more of the "bug fix" nature because most of the code
>> in question was already in itk.  Sometimes bugs become apparent with use.
>> Eg testing on large 3d datasets.
>>
>> On Jan 31, 2012 7:43 PM, "Luis Ibanez" <luis.ibanez at kitware.com> wrote:
>>>
>>>
>>> Brian,
>>>
>>> Just to second Matt: We are making regular tags.
>>>
>>> There is no need to rush to inserting immature
>>> code into the code base.
>>>
>>> Please take the time to raise the code quality to ITK
>>> standards, and make sure that the code is passing
>>> tests in all platforms, have no Valgrind issues and
>>> respect coding style.
>>>
>>> It is a lot more expensive and disruptive to attempt
>>> to improve the code after it has been merged. The
>>> role of the new software process is to eliminate the
>>> large majority of code defects before the code is
>>> merged into the main Git repository.
>>>
>>> If you see an urgency for sharing the code publicly,
>>> (e.g. as a complement to a paper in a conference),
>>> you are welcome to post it to the Insight Journal,
>>> with the proper warning indicating this is code that
>>> can change in the short term.
>>>
>>> Many thanks for helping us maintain the quality of
>>> the code in the repository to the high standards
>>> that are expected from ITK.
>>>
>>>
>>>        Luis
>>>
>>>
>>>
>>> -----------------------------------------------------------
>>> On Tue, Jan 31, 2012 at 6:15 PM, Matt McCormick
>>> <matt.mccormick at kitware.com> wrote:
>>>>
>>>> Hi Brian,
>>>>
>>>> Please work out the kinks the rest of this week, and we can make
>>>> another RC tag next week.  There is no need to rush in features.  We
>>>> will have multiple RC's, and point releases will not be that far
>>>> apart.  Please focus on cleaning up the dashboard immediately before
>>>> releases.
>>>>
>>>> Thanks,
>>>> Matt
>>>>
>>>> On Tue, Jan 31, 2012 at 6:00 PM, brian avants <stnava at gmail.com> wrote:
>>>> > fyi --- nick and i have been working out a couple algorithm details
>>>> > and i think these are just about done.
>>>> >
>>>> > the two patches are :
>>>> >
>>>> > large deformation registration
>>>> >
>>>> > http://review.source.kitware.com/#/c/3752/
>>>> >
>>>> > and  syn
>>>> >
>>>> > http://review.source.kitware.com/#/c/3753/
>>>> >
>>>> > both are diffeomorphic methods
>>>> >
>>>> > this 2nd one is ready to go, i think, as there was only one small
>>>> > issue with a test raising a false alarm.   the first is probably good
>>>> > to go too.
>>>> >
>>>> > as we are so close to having these right, it would be great if they
>>>> > could enter the tag.
>>>> >
>>>> > brian
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On Tue, Jan 31, 2012 at 5:54 PM, M Stauffer -V- <mstauff at verizon.net>
>>>> > wrote:
>>>> >> With the last patch I put up today
>>>> >> (http://review.source.kitware.com/#/c/3786/), all the
>>>> >> registrationv4-related
>>>> >> errors and warnings below should be handled. Those not handled
>>>> >> in this patch
>>>> >> have already been merged.
>>>> >>
>>>> >> -M
>>>> >>
>>>> >> ________________________________
>>>> >> From: insight-developers-bounces at itk.org
>>>> >> [mailto:insight-developers-bounces at itk.org] On Behalf Of Luis Ibanez
>>>> >> Sent: Monday, January 30, 2012 2:14 PM
>>>> >> To: Insight Developers
>>>> >> Cc: Terry Yoo
>>>> >> Subject: [Insight-developers] Cancelling Tag for today due to
>>>> >> UnacceptableDashboard Status
>>>> >>
>>>> >>
>>>> >> The sad current status of the Dashboard forces us to��
>>>> >> cancel��the tagging that we had scheduled for today.
>>>> >>
>>>> >> From our conversation at the tcon on Friday, it was
>>>> >> expected that the parties responsible for failing tests,
>>>> >> warnings and Valgrind error were going to take active
>>>> >> steps towards fixing them.
>>>> >>
>>>> >> We will hold on tagging until we can get a Dashboard
>>>> >> that is representative of the level of quality that is��
>>>> >> expected from ITK.
>>>> >>
>>>> >>
>>>> >>
>>>> >> Here is the list of significant problems:
>>>> >>
>>>> >>
>>>> >>
>>>> >> http://www.cdash.org/CDash/testSummary.php?project=2&name=itkJointHistogramMutualInformationImageToImageMetricv4Test&date=2012-01-30��
>>>> >>
>>>> >>
>>>> >> http://www.cdash.org/CDash/testSummary.php?project=2&name=itkTimeVaryingBSplineVelocityFieldImageRegistrationTest&date=2012-01-30
>>>> >>
>>>> >>
>>>> >> http://www.cdash.org/CDash/testSummary.php?project=2&name=itkSimpleImageRegistrationTest&date=2012-01-30
>>>> >>
>>>> >>
>>>> >> http://www.cdash.org/CDash/testSummary.php?project=2&name=itkOtsuThresholdImageFilterTestShort&date=2012-01-30
>>>> >>
>>>> >>
>>>> >> http://www.cdash.org/CDash/testSummary.php?project=2&name=ResampleImageFilter9Test&date=2012-01-30
>>>> >>
>>>> >>
>>>> >> http://www.cdash.org/CDash/testSummary.php?project=2&name=itkSyNImageRegistrationTest&date=2012-01-30
>>>> >>
>>>> >>
>>>> >>
>>>> >> http://www.cdash.org/CDash/testSummary.php?project=2&name=OpenCVImageBridgeTest&date=2012-01-30
>>>> >>
>>>> >>
>>>> >> http://www.cdash.org/CDash/testSummary.php?project=2&name=OpenCVVideoCaptureTest&date=2012-01-30
>>>> >>
>>>> >>
>>>> >> http://www.cdash.org/CDash/testSummary.php?project=2&name=OpenCVVideoIOFactoryTest&date=2012-01-30
>>>> >>
>>>> >>
>>>> >> http://www.cdash.org/CDash/testSummary.php?project=2&name=OpenCVVideoIOTest&date=2012-01-30
>>>> >>
>>>> >>
>>>> >>
>>>> >> Warnings:
>>>> >>
>>>> >> These are lack of local attention to quality.
>>>> >> Please remember that we have a zero warnings policy.
>>>> >>
>>>> >> Warning Set 1:
>>>> >> http://www.cdash.org/CDash/viewBuildError.php?type=1&buildid=1960751
>>>> >>
>>>> >> Warning Set 2:
>>>> >> http://www.cdash.org/CDash/viewBuildError.php?type=1&buildid=1961365
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> Valgrind Errors
>>>> >>
>>>> >> http://www.cdash.org/CDash/viewDynamicAnalysis.php?buildid=1961416
>>>> >>
>>>> >>
>>>> >> itkANTSNeighborhoodCorrelationImageToImageRegistrationTest
>>>> >> http://www.cdash.org/CDash/viewDynamicAnalysisFile.php?id=2702653
>>>> >>
>>>> >> itkQuasiNewtonOptimizerv4RegistrationTest3
>>>> >> http://www.cdash.org/CDash/viewDynamicAnalysisFile.php?id=2702654
>>>> >>
>>>> >> itkQuasiNewtonOptimizerv4RegistrationTest1
>>>> >> http://www.cdash.org/CDash/viewDynamicAnalysisFile.php?id=2702655
>>>> >>
>>>> >> itkQuasiNewtonOptimizerv4RegistrationTest2
>>>> >> http://www.cdash.org/CDash/viewDynamicAnalysisFile.php?id=2702656
>>>> >>
>>>> >> itkJointHistogramMutualInformationImageToImageRegistrationTest2
>>>> >> http://www.cdash.org/CDash/viewDynamicAnalysisFile.php?id=2702657
>>>> >>
>>>> >> itkCorrelationImageToImageMetricv4Test
>>>> >> http://www.cdash.org/CDash/viewDynamicAnalysisFile.php?id=2702658
>>>> >>
>>>> >>
>>>> >> A helper script for debugging Valgrind problems is available at:
>>>> >>
>>>> >> �� �� �� �� �� ITK/Utilities/Maintenance/runValgrind.sh
>>>> >>
>>>> >>
>>>> >>
>>>> >> �� ��Please address these software quality problems so that
>>>> >> �� ��we can get back on track with the release schedule.
>>>> >>
>>>> >>
>>>> >> �� �� �� ��Thanks
>>>> >>
>>>> >>
>>>> >> �� �� �� �� �� �� Luis
>>>> >>
>>>> >>
>>>> >> _______________________________________________
>>>> >> Powered by www.kitware.com
>>>> >>
>>>> >> Visit other Kitware open-source projects at
>>>> >> http://www.kitware.com/opensource/opensource.html
>>>> >>
>>>> >> Kitware offers ITK Training Courses, for more information visit:
>>>> >> http://kitware.com/products/protraining.html
>>>> >>
>>>> >> Please keep messages on-topic and check the ITK FAQ at:
>>>> >> http://www.itk.org/Wiki/ITK_FAQ
>>>> >>
>>>> >> Follow this link to subscribe/unsubscribe:
>>>> >> http://www.itk.org/mailman/listinfo/insight-developers
>>>> >>
>>>> > _______________________________________________
>>>> > Powered by www.kitware.com
>>>> >
>>>> > Visit other Kitware open-source projects at
>>>> > http://www.kitware.com/opensource/opensource.html
>>>> >
>>>> > Kitware offers ITK Training Courses, for more information visit:
>>>> > http://kitware.com/products/protraining.html
>>>> >
>>>> > Please keep messages on-topic and check the ITK FAQ at:
>>>> > http://www.itk.org/Wiki/ITK_FAQ
>>>> >
>>>> > Follow this link to subscribe/unsubscribe:
>>>> > http://www.itk.org/mailman/listinfo/insight-developers
>>>
>>>
>

More information about the Insight-developers mailing list