[Insight-developers] Issue with types (unsigned) long and Visual Studio x64

[maquette]

Hi All,

I have a question about some Issue that was discussed on insight-users 
back in 2007:

http://www.itk.org/pipermail/insight-users/2007-February/021069.html

The thread faded without a real solution to an "Allocate > 4GB on 
Windows x64" Problem. Luis told us patience and to wait for fixes in ITK 
3.xx. Now I have still problems as soon as an image chunk with > 2GB is 
used in the output of the ResamplingFilter. To review:
- Problems back in 2007 were suspected to stem from using "long" and 
"unsigned long" in stead of size_t and ptrdiff_t for pixel amounts, 
pixel offsets, etc. Some user tried to replace (unsigned) long in the 
basic files itkImage.h -ImageBase.h -Index.h -Size.h -Offset.h but got 
into trouble elsewhere, so stopped "fixing" this crosscutting issue.

-Today I'm encountering some strange issues as well, but not while an 
Image is Allocated, but rather when using the Resample Filter 
multithreaded (16 threads, no problem with 1 thread, why??). The outIt 
iterator in LinearThreadedGenerateData(...) happens to fall into way too 
low memory, when it is expected to work on one of the "upper" portions 
of the large Output Image (in my case the Image was 1541^3 x 2Byte). So 
allocation works fine, no exception raised, just some pointer 
miscalculation somewhere. Looks like an overrun of a 4byte "unsigned 
long". Visual Studio uses 4Byte for long even in 64bit mode! So theres 
no problem on GCC I suppose.

-As the Iterators Regions are using 4byte it cannot work like that. => 
tried to do the replacements in itkImage.h -ImageBase.h -Index.h -Size.h 
-Offset.h and some itk...Iterator.h files (offset table) as well ... and 
running into trouble elsewhere. The files all use "long" and "unsigned 
long" even in 3.18 so there has been no fix so far?

=> I think this is worth beeing resolved, as memory is cheap, Win is 
widespread (would rather use unix though, but the boss says...) 3D 
Images are Big, and interesting Algorithms are not local ;)

=> In my subjective opinion "long" and "unsigned long" should not be 
used for Pixel Indexes, which can get really big, if images are big and 
not nicely cube-shaped. E.g. if one pixel has 1Byte, and the Image has 
only 1 dimension one would need the whole range of size_t and ptrdiff_t 
to navigate a [1,1,5000000000] Image.

Do you think, that ITK 3.xx can be fixed by:
1) replacing all integers which can amount to memory chunk size by 
either size_t (unsigned) or ptrdiff_t (signed)? The symptom now is IMHO 
that I can allocate an Image without failure/exception which causes the 
pipeline to fail without a noticable exception. But is it good enough to 
look for anything related to a pixel position/difference or memory 
position/difference and change the types? This is rather a design 
decision than a lokalized bug fix.

2) starting in Code/Common first? Of course this will lead to many 
failures in other ITK classes which follow the policy "memorysize is 
long" BUT would at least remove the flaw from Core, and make the rest 
fixable by the more specialized developers (outside Common).

3) telling you where the crash happens exactly, and why its only in 
multithreaded? Well, I think it can not work on a Platform where 
sizeof(long) < sizeof(ptrdiff_t).

I did my best to search for some successor of the old thread, because I 
was almost sure that this issue is treated somewhere, perhaps in ITK 4. 
So please don't be upset that I put it here, but teach me to do better 
(only a user so far...)