[Insight-developers] ITK test failures in Debug only
Mathieu Coursolle
mcoursolle at rogue-research.com
Thu Sep 13 09:46:57 EDT 2007
Hi Luis,
I just tried a clean build, downloaded all source code from cvs
(2007/09/11) in
an empty folder, than create 2 empty build folders, one for Debug and one
for Release. I build both of them using the downloaded source code, default
parameters (SDK 10.4u, arch i386, examples on, testing on).
The only difference is that one is Debug and the other Release.
I built the Release one first, ran ctest and got 4 test failures. I expected
those as they've been one our dashboard for a while:
<http://www.itk.org/Testing/Sites/RogueResearch2/MacOSXTiger-univ-
release/20070912-0100-Nightly/Test.html>
I than built the Debug one, ran ctest and got 42 test failures. Most of
them being
Illegal and segmentation fault. I ran them in gdb and realized I have to
run them multiple
times to have a failure. The tests which fail are failing about 10% of
the time.
I don't know if it is the same, but that other dashboard seems to
experience similar problems:
<http://www.itk.org/Testing/Sites/SINTEF.LabMAC.OSX/
ITKcvs_CMake246_gcc401/20070912-0100-Nightly/Test.html>
I than looked at the backtrace and it looks like it is always the same
pattern, always coming
from itk::MultiThreader::....
Here are a few examples of the backtraces I've got:
-------------------------------------------------------------------
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x36e47410
[Switching to process 1959 thread 0x1f0b]
0x8fe0fd47 in __dyld__ZN16ImageLoaderMachO19instantiateSegmentsEPKh ()
(gdb) bt
#0 0x8fe0fd47 in __dyld__ZN16ImageLoaderMachO19instantiateSegmentsEPKh ()
#1 0x8fe12ea5 in __dyld__ZN16ImageLo
aderMachOC1EPKciPKhyyRK4statRKN11ImageLoader11LinkContextE ()
#2 0x0037a000 in itk::NeighborhoodIn
nerProduct<itk::OrientedImage<float, 2u>, double, double>::operator()
(this=0xb0182d7e, it=@0xb0182c7c, op=@0x336e474) at /Volumes/Extra/ITK/
20070911/Insight/Code/Common/itkNeighborhoodInnerProduct.h:71
#3 0x0037d33e in itk::NeighborhoodOp
eratorImageFilter<itk::OrientedImage<float, 2u>, itk::Image<float, 2u>,
double>::ThreadedGenerateData (this=0x336e3e0,
outputRegionForThread=@0xb0182ecc, threadId=3) at /Volumes/Extra/ITK/
20070911/Insight/Code/BasicFilters/itkNeighborhoodOperatorImageFilter.txx:130
#4 0x002fc69c in itk::ImageSource<itk::Image<float, 2u>
>::ThreaderCallback (arg=0x381f29c) at /Volumes/Extra/ITK/20070911/
Insight/Code/Common/itkImageSource.txx:273
>#5 0x001e7ef8 in itk::MultiThreader::SingleMethodProxy (arg=0x381f29c)
at /Volumes/Extra/ITK/20070911/Insight/Code/Common/itkMultiThreader.cxx:754
#6 0x900245c7 in _pthread_body ()
-------------------------------------------------------------------
-------------------------------------------------------------------
0 <<00000000>> 0x903aab20 0 + -1875203296
1 itkIOTests 0x003a4045 itk::Neighborhood<double*, (unsigned)
6, itk::NeighborhoodAllocator<double*> >::SetRadius(itk::Size<(unsigned)
6> const&) + 181 (itkNeighborhood.txx:103)
2 itkIOTests 0x003a7c71 itk
::ConstNeighborhoodIterator<itk::Image<double, (unsigned)6>,
itk::ZeroFluxNeumannBoundaryCondition<itk::Image<double, (unsigned)6> >
>::Initialize(itk::Size<(unsigned)6> const&, itk::Image<double,
(unsigned)6> const*, itk::ImageRegion<(unsigned)6> const&) + 125
(itkConstNeighborhoodIterator.txx:355)
3 itkIOTests 0x003a7fc0 itk
::ConstNeighborhoodIterator<itk::Image<double, (unsigned)6>,
itk::ZeroFluxNeumannBoundaryCondition<itk::Image<double, (unsigned)6> >
>::ConstNeighborhoodIterator[in-charge](itk::Size<(unsigned)6> const&,
itk::Image<double, (unsigned)6> const*, itk::ImageRegion<(unsigned)6>
const&) + 120 (itkConstNeighborhoodIterator.h:116)
4 itkIOTests 0x003a9c09 itk
::DifferenceImageFilter<itk::Image<double, (unsigned)6>,
itk::Image<double, (unsigned)6> >::ThreadedGenerateData(itk::ImageRegion<
(unsigned)6> const&, int) + 553 (itkDifferenceImageFilter.txx:154)
5 itkIOTests 0x003911ec itk::ImageSource<itk::Image<double,
(unsigned)6> >::ThreaderCallback(void*) + 144 (itkImageSource.txx:282)
>6 itkIOTests 0x0028b40c itk::MultiThreader::SingleMethodProxy
(void*) + 26 (itkMultiThreader.cxx:756)
7 libSystem.B.dylib 0x900245c7 _pthread_body + 84
-------------------------------------------------------------------
-------------------------------------------------------------------
0 libSystem.B.dylib 0x90024cdf semaphore_wait_trap + 7
>1 SegmentationExamples 0x001e31c1
itk::MultiThreader::WaitForSingleMethodThread(_opaque_pthread_t*) + 35
(itkMultiThreader.cxx:800)
2 SegmentationExamples 0x001e3826
itk::MultiThreader::SingleMethodExecute() + 1160 (itkMultiThreader.cxx:353)
3 SegmentationExamples 0x003003a7 itk::ImageSource<itk::Image<double,
(unsigned)6> >::GenerateData() + 203 (itkImageSource.txx:233)
4 SegmentationExamples 0x001e9fc5 itk::ProcessObject::UpdateOutputData
(itk::DataObject*) + 1085 (itkProcessObject.cxx:1013)
5 SegmentationExamples 0x001e1175 itk::DataObject::UpdateOutputData()
+ 141 (itkDataObject.cxx:423)
6 SegmentationExamples 0x001e0ec6 itk::DataObject::Update() + 60
(itkDataObject.cxx:345)
7 SegmentationExamples 0x001ea8ee
itk::ProcessObject::UpdateLargestPossibleRegion() + 118
(itkProcessObject.cxx:1133)
8 SegmentationExamples 0x00003a9c RegressionTestImage(char const*,
char const*, int, double, unsigned, unsigned) + 1426 (itkTestMain.h:296)
9 SegmentationExamples 0x00005054 main + 1394 (itkTestMain.h:174)
10 SegmentationExamples 0x000025c2 _start + 216
11 SegmentationExamples 0x000024e9 start + 41
-------------------------------------------------------------------
-------------------------------------------------------------------
0 dyld 0x8fe12fbe stub_binding_helper_interface + 60
1 FilteringExamples8 0x002bf89c itk::ImageSource<itk::Image<unsigned
char, (unsigned)2> >::ThreaderCallback(void*) + 144 (itkImageSource.txx:282)
>2 FilteringExamples8 0x001e2450 itk::MultiThreader::SingleMethodProxy
(void*) + 26 (itkMultiThreader.cxx:756)
3 libSystem.B.dylib 0x900245c7 _pthread_body + 84
-------------------------------------------------------------------
Thanks!
Mathieu
--
____________________________________________________________
Mathieu Coursolle mcoursolle at rogue-research.com
Rogue Research www.rogue-research.com
Montréal, Québec, Canada
>
>Hi Mathieu,
>
> Is the Debug build a "clean" build ?
>
> I mean: Is it starting from scratch every time ?
>
>
>The messages I see in the failing tests are mostly of the
>two following types:
>
>
> a) *** Exception executing: Segmentation fault
>
> b) *** Exception executing: Illegal instruction
>
>
>
>The typical suspects therefore are:
>
> 1) You may have CVS updated some headers but
> not recompiled the corresponding libraries
>
> 2) Uninitialized variables (in particular pointers)
>
> 3) Byte swapping misinterpretation errors
>
> 4) Outdated shared libraries getting in the way.
> (do you have ITK shared libraries in your path ?)
>
>
>Here are some patterns in the failing tests:
>
>
> 3 are image registration tests
> 20 are IO tests (**)
> 5 are image resampling tests
> 2 are level sets
> 4 are various filtering tests
>
>
>(**) This lead to suspect of:
>
> a) a byte swapping misinterpretation of your architecture
>
> b) some relationship with a non rebuild *ITKIO* library
> (note that the ImageIOBase object has changed recently)
> If for some reason your ITKIO library didn't rebuild
> then there is a mismatch on the ImageIO objects virtual
> tables.
>
>
>A first sanity check will be to do a clean build.
>
>If the error reappear, then we still have the shared
>outdated shared library suspect, uninitialized memory
>suspect and the byte swapping suspect (this last one
>has the alibi of the Release build working...)
>
>
>The *good news* is that your build is a *Debug* build.
>
>
>Could you please launch one of these failing test
>inside gdb and let us know in what file and line
>is the segmentation fault happening ?
>
>
>
> Please let us know what you find,
>
>
> Thanks
>
>
> Luis
>
>
>-------------------------
>Mathieu Coursolle wrote:
>> Hi ITK-developpers,
>>
>> We currently have 2 dashboard submissions for ITK, one is
>> release:
>> <http://www.itk.org/Testing/Sites/RogueResearch2/MacOSXTiger-univ-
>> release/20070912-0100-Nightly/Notes.html>
>>
>> and another is debug:
>> <http://www.itk.org/Testing/Sites/RogueResearch2/MacOSXTiger-univ-debug/
>> 20070912-0740-Experimental/Notes.html>
>>
>> We usually have a few (2 to 8) tests failures in release which seem to
>> be thread related.
>> However, the debug build has around 40 test failures. The current debug
>> submission has MINC2 support,
>> but this should not be the cause of all those failures.
>>
>> I was able to reproduce on another computer as well (i386, OSX 10.4, 2
>> dual core), downloaded the
>> source and build a Debug and a Release build using the default settings
>> (No minc2, no patented, no preview, etc).
>> I always got around 5 tests failures in release, but always more than 35
>> in Debug.
>>
>> Any idea on why this is happening?
>>
>> Thanks!
>>
>>
>
More information about the Insight-developers
mailing list