[Insight-developers] [Insight-users] Image offset is giving bad pointer for large datasets (7Gb)
Luis Ibanez
luis.ibanez at kitware.com
Fri Jul 10 12:30:12 EDT 2009
Hi Kana,
Thanks a lot for looking into this and sharing your findings.
I just confirmed that in Windows 64bits, the "long" type is
only 4 bytes.
Here is the program I used:
#include <iostream>
#include "itkOffset.h"
#include "itkNumericTraits.h"
int main()
{
unsigned long tt;
std::cout << "size = " << sizeof(tt) << std::endl;
tt = -1;
std::cout << "tt = " << tt << std::endl;
typedef itk::Offset<3> OffsetType;
typedef OffsetType::OffsetValueType OffsetValueType;
OffsetValueType offsetValue;
std::cout << "sizeof(offsetValue) = " << sizeof( offsetValue ) <<
std::endl;
offsetValue = itk::NumericTraits< OffsetValueType >::max();
std::cout << "OffsetValueType max() = " << offsetValue << std::endl;
return EXIT_SUCCESS;
}
with this CMakeLists.txt file
CMAKE_MINIMUM_REQUIRED(VERSION 2.4)
IF(COMMAND CMAKE_POLICY)
CMAKE_POLICY(SET CMP0003 NEW)
ENDIF(COMMAND CMAKE_POLICY)
PROJECT(64bitsTest)
FIND_PACKAGE(ITK REQUIRED)
INCLUDE(${ITK_USE_FILE})
ADD_EXECUTABLE(typesTest typesTest.cxx )
TARGET_LINK_LIBRARIES(typesTest ITKCommon)
---------
So, it seems that we have to introduce a new ITK type,
that will be the longest int available in the platform.
We currently have the following declarations in the file:
Insight/Code/Common/itkIntTypes.h
/** Convenient and more descriptive integer types. */
typedef char ITK_INT8;
typedef int ITK_INT32;
#ifndef _WIN32
typedef long long ITK_INT64;
#endif
#ifdef _WIN32
typedef long ITK_INT64;
#endif
typedef unsigned char ITK_UINT8;
typedef unsigned short ITK_UINT16;
typedef unsigned ITK_UINT32;
#ifndef _WIN32
typedef unsigned long long ITK_UINT64;
#endif
#ifdef _WIN32
typedef unsigned long ITK_UINT64;
#endif
typedef int ITK_INTPTR;
typedef unsigned ITK_UINTPTR;
#ifdef __cplusplus
}
#endif
One option that comes to mind is that we
should simply use
size_t
or std::size_t
Which is the type expected by the "mem"
methods, {memcpy,memdup...}, and the
allocation "new" method.
I have verified that "size_t" will have 8 bits
on Windows 64.
std::cout << "sizeof( size_t ) = " << sizeof( size_t ) << std::endl;
Unless there are any objections, I'll suggest
that we replace the type of all integer variables
related to image offsets and image size, with
the type "size_t".
BTW: note that there is also "size_type"
It seems that "size_type" will be the type used by STL
containers.
Any comments ?
Thanks
Luis
------------------------------------------------------------------------------------------------------------
On Thu, Jul 9, 2009 at 2:53 AM, Arunachalam Kana <
Kana.Arunachalam at fh-wels.at> wrote:
> Hi Luis,
>
> Thank you for your response. I ran the test program and the given the
> results are given below along with detailed
> System information and what option i used to compile itk.
>
> System Information
> ------------------
> Time of this report: 7/8/2009, 17:16:29
> Machine name: CT-DELL
> Operating System: Windows XP Professional x64 Edition (5.2, Build
> 3790) Service Pack 2 (3790.srv03_sp2_gdr.090319-1204)
> Language: English (Regional Setting: German)
> System Manufacturer: Dell Inc.
> System Model: Precision WorkStation T7400
> BIOS: Default System BIOS
> Processor: Intel(R) Pentium(R) III Xeon-Prozessor (8 CPUs),
> ~3.2GHz
> Memory: 65534MB RAM
> Page File: 717MB used, 65267MB available
> Windows Dir: C:\WINDOWS
> DirectX Version: DirectX 9.0c (4.09.0000.0904)
> DX Setup Parameters: Not found
> DxDiag Version: 5.03.3790.3959 32bit Unicode
>
>
> Itk compilation
> ---------------------
> Itk compiled using Microsoft visual studio 2009 x64 (option).
>
> Microsoft visual studio 2008
> ---------------------
> Configuration manager details:
> Active Solution Configuration: Debug
> Active Solution Platform: x64
>
> Test run details:
> 1. Program:
> unsigned long tt;
> std::cout << "size = " << sizeof(tt) << std::endl;
> tt = -1;
> std::cout << "tt = " << tt << std::endl;
> output: size = 4; tt = 4294967295
>
> 2. Program:
> unsigned long long tt;
> std::cout << "size = " << sizeof(tt) << std::endl;
> tt = -1;
> std::cout << "tt = " << tt << std::endl;
> output: size = 8; tt = 18446744073709551615
>
> I changed the configuration details:
> --------------------------------------
> Active Solution Configuration: Debug
> Active Solution Platform: Win32
>
> Test run details:
> 1. Program:
> unsigned long tt;
> std::cout << "size = " << sizeof(tt) << std::endl;
> tt = -1;
> std::cout << "tt = " << tt << std::endl;
> output: size = 4; tt = 4294967295
>
> 2. Program:
> unsigned long long tt;
> std::cout << "size = " << sizeof(tt) << std::endl;
> tt = -1;
> std::cout << "tt = " << tt << std::endl;
> output: size = 8; tt = 18446744073709551615
>
> For both Win32 and x64 i get the same result.
> Unsigned long is 4 byte and unsigned long long is 8 byte.
>
> I was a little confused after the result, so i searched for data type
> ranges in msdn. Link below:
>
> http://msdn.microsoft.com/en-us/library/s3f49ktz.aspx
>
> From the information given in above link, i understand that for 32 bit
> and 64 bit compiler unsigned long is 4 byte and unsigned long long is 8
> byte.
>
> I checked in the wiki for 64 bit machines (link below)
>
> http://en.wikipedia.org/wiki/64-bit
>
> In the link above, under topic "scientific data model": It is given that
> Microsoft win64 uses the LLP data model. From the above links, I
> understand the following:
>
> 1. A 64bit machine with Microsoft Windows x64 Operating system has
> 4 byte for long and 8 byte for long long.
> 2. Microsoft visual studio 2009 has long as 4 byte and long long as
> 8 bytes for both win32 and x64 compiler option.
>
> What's next?
>
> Thank you,
> Regards,
> Kana
>
> -----Original Message-----
> From: Luis Ibanez [mailto:luis.ibanez at kitware.com]
> Sent: 08 July 2009 16:10
> To: Arunachalam Kana
> Cc: insight-users at itk.org
> Subject: Re: [Insight-users] Image offset is giving bad pointer for
> large datasets (7Gb)
>
>
>
> Hi Arunachalam,
>
>
> Thanks for your detailed post.
>
>
> Let's discuss first ITK by itself,
>
>
> (before we go into the integration with VTK and VTKEdge)
>
>
>
> As you pointed out, the type
>
> OffsetValueType
>
> is currently defined as "long" in ITK.
>
>
>
> However, your assumptiong that "long" types in 64 bits
> platforms, are storing values up to 2^32 is incorrect.
>
>
> An unsigned long type in a 64bits platforms will have
> 8 bytes, and therefore can store values up to 2^64.
> which is:
>
> 18446744073709551615 L
>
>
> You can verify this with the following code:
>
>
> #include <iostream>
> int main()
> {
> unsigned long tt;
> std::cout << "size = " << sizeof(tt) << std::endl;
> tt = -1;
> std::cout << "tt = " << tt << std::endl;
> }
>
>
> The output of this program, in a 64bits machine will be:
>
>
> size = 8
> tt = 18446744073709551615
>
>
> The same code, when compiled in a 32-bits machine
> will produce as output:
>
>
> size = 4
> tt = 4294967295
>
>
> Therefore, if your OffsetValueType is getting saturated,
> it is likely that you are *not* compiling for 64 bits.
>
>
> Are you sure that when you configured ITK with CMake,
> you selected the "64 bits" version of your Visual Studio
> compiler.
>
>
> A simple way to double-check is to compile the code
> above and run it.
>
>
>
> Please do so, and let us konw what you find.
>
>
> Thanks
>
>
>
> Luis
>
>
>
> ----------------------------------
> Arunachalam Kana wrote:
> > Hi,
> >
> >
> >
> > Machine details: Windows XP, visual studio 2008, 64 bit platform,
> 64Gb Ram
> >
> > Image size: 1442 x 1566 x 1657 ( approx 7Gb )
> >
> > Other libraries: Along with ITK, VTK and VTK Edge is also used. VTK
> Edge
> > is used for changing vtk image to itk image and vice versa.
> >
> >
> >
> > Problem:
> >
> > In ITK the OffsetValueType is long. Range of long datatype:
> > -2,147,483,648 to 2,147,483,647
> >
> > But the image m_OffsetTable values of type OffsetValueType are : 1;
> > 1442; 2258172; 3,741,791,004
> >
> > In the above, it is clearly seen that the image offset value exceeds
> the
> > OffsetValueType range.
> >
> >
> >
> > Correction attempts:
> >
> > 1. I changed the OffsetValueType to unsigned long to increase
> the
> > range. Unsigned long range : 0 to 4,294,967,295. But
> >
> > this gives problems on conversion back to vtk. The vtkdataarray
> created
> > has a variable maxID which is of VtkIDType.
> >
> > VtkIDType is of long long. The maxID contains the value =
> -553,176,292.
> >
> >
> >
> > Correlation between 3,741,791,004 and -553,176,292.
> >
> > Decimal Hexa
> >
> > 3,741,791,003 DF07331B
> >
> > -553,176,292 FFFFFFFF DF07331B
> >
> >
> >
> > So, the program crashes due to bad pointer when it used vtkdata array
> > details to draw histogram.
> >
> > I think the probelm here is assigning of unsigned long to long long. I
>
> > don't want to change the
> >
> > Vtk data type to unsigned long, so in next attempt i change the itk
> > OffsetValueType to long long.
> >
> >
> >
> > 2. I changed the itk OffsetValueType to long long, but this
> leads
> > to change of lot of several other
> >
> > variable type in different file. Specially some of the basic image
> > container variables like TElementIdentifier, TElement have to be
> changed.
> >
> > For now I only can give these 2 variable, but i am afraid more
> variable
> > data types have to be changed in the future.
> >
> >
> >
> > I would be glad to have some help to solve this problem.
> >
> >
> >
> > Regards,
> >
> > Kana
> >
> >
> >
> ------------------------------------------------------------------------
> >
> > _____________________________________
> > Powered by www.kitware.com
> >
> > Visit other Kitware open-source projects at
> > http://www.kitware.com/opensource/opensource.html
> >
> > Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
> >
> > Follow this link to subscribe/unsubscribe:
> > http://www.itk.org/mailman/listinfo/insight-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/mailman/private/insight-developers/attachments/20090710/fc89e45a/attachment.htm>
More information about the Insight-developers
mailing list