[Insight-developers] itkSimpleImageRegistrationTest failing (gcc 4.4 change in floating point co

[M Stauffer -V-]

Hi,

Yes, itkSimpleImageRegistrationTest has been failing on these platforms
for quite some weeks. First we ruled out a bug that we fixed. I believe
the failing builds are caused by accumulated floating-point errors, but
can't say why.

Brad's link to the Stackoverflow topic is interesting, but I don't think
it'd be a compile-time computation in this case since that would mean
hardcoded values. The other suggestion in that topic was that older gcc
was using double under the hood whereas the new one doesn't by default,
so there's possible a step somewhere in the algorithm that's not getting
promoted to double in gcc 4.4.

The test runs a three-level registration, first with an affine
transform, then with a displacement field transform. Looking at the
debug output of the test, the difference between passing and failing
tests get progressively larger with each level of the registration.
Levels 1 and 2 have a smoothing sigma of 2 and 1, respectively, and the
third level is has a smoothing sigma of zero. The results start to
differ much faster at level three, so I created
itkSimpleImageRegistrationTest3 which is the same except that it has a
smoothing sigma of 1 for level 3. The results here differ less greatly.
So presumably the smoothing is covering up a lot of the difference, not
surprisingly. Note that Test3 always passes, it's not running the
comparison of the result images. Test3 outputs debug info with fewer
characters, so that the results will be less truncated b/c of the 1024
char limit on test output in passing tests.

Test3 on a build that passes for main test:
http://open.cdash.org/testDetails.php?test=136903622&build=2046665

Test3 on a build that fails for main test:
http://open.cdash.org/testDetails.php?test=136904738&build=2046706

Note that itkSimpleImageRegistrationTest2 is the same as the main test,
just with 1 thread instead of 2. This also produces different results,
even on the same platform. Presumably b/c of precision rounding
differences like we've identified before.

Note that other multi-level registration tests
(itkSyNImageRegistrationTest,
itkTimeVaryingBSplineVelocityFieldImageRegistrationTest,
itkTimeVaryingVelocityFieldImageRegistrationTest) don't currently run a
comparison against a baseline. We could add that and see if the same
issue happens.

-M

>-----Original Message-----
>From: insight-developers-bounces at itk.org 
>[mailto:insight-developers-bounces at itk.org] On Behalf Of Bill Lorensen
>Sent: Tuesday, February 28, 2012 8:14 AM
>To: Bradley Lowekamp
>Cc: ITK Developers
>Subject: Re: [Insight-developers] 
>itkSimpleImageRegistrationTest failing (gcc 4.4 change in 
>floating point computation)
>
>Brad,
>
>It looks like there are already 3 baselines. And, according to the
>test output, the second baseline is pretty close to the results of the
>2 failing platforms. Seems that the displayed error images are from
>the first baseline.
>
>Perhaps we just need a higher tolerance. Something greater than 31.
>
>Also, we may have a bug in the regression testing that causes the
>wrong difference image to be used.
>
>Bill
>
>On Mon, Feb 27, 2012 at 10:51 PM, Bradley Lowekamp
><blowekamp at mail.nih.gov> wrote:
>> Hello,
>>
>> The itkSimpleImageRegistrationTest  is failing on a couple 
>systems. Perhaps those with newer versions of gcc (gcc4.4+ )
>>
>> 
>http://open.cdash.org/testSummary.php?project=2&name=itkSimpleI
mageRegistrationTest&date=2012-02-26
>>
>> They appear to be consistently failing in the same way each 
>night and across the platforms. Why do I think it's failing? I 
>hope someone has more insight, but it appears that gcc has 
>changed something in how it does/optimized floating point 
>computation so that results are slightly different. My first 
>potentially relavent google hit:
>>
>> 
>http://stackoverflow.com/questions/8802831/gcc-rounding-differe
>nce-between-versions
>>
>>
>> It is not clear to me if the output is "close" enough to be 
>considered correct. Specifically, the bottom right appears to 
>be more then a pixel off in the registration. Is it correct enough?
>>
>> If it is I am tempted to add a second baseline image to the 
>test ( as opposed to adding tolerances  to the comparison ).
>>
>> Thought on how to clean up the dashboard further?
>>
>> Brad
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Kitware offers ITK Training Courses, for more information visit:
>> http://kitware.com/products/protraining.html
>>
>> Please keep messages on-topic and check the ITK FAQ at:
>> http://www.itk.org/Wiki/ITK_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.itk.org/mailman/listinfo/insight-developers
>
>
>
>-- 
>Unpaid intern in BillsBasement at noware dot com
>_______________________________________________
>Powered by www.kitware.com
>
>Visit other Kitware open-source projects at
>http://www.kitware.com/opensource/opensource.html
>
>Kitware offers ITK Training Courses, for more information visit:
>http://kitware.com/products/protraining.html
>
>Please keep messages on-topic and check the ITK FAQ at:
>http://www.itk.org/Wiki/ITK_FAQ
>
>Follow this link to subscribe/unsubscribe:
>http://www.itk.org/mailman/listinfo/insight-developers