[Insight-users] Image offset is giving bad pointer for large datasets (7Gb)
Luis Ibanez
luis.ibanez at kitware.com
Fri Jul 10 17:47:32 EDT 2009
Hi Michael,
We have the following "large image" tests in
the directory
Insight/Testing/Code/IO
The filename is:
itkLargeImageWriteReadTest.cxx
and the CMakeLists.txt file runs in with
two sizes:
IF( ${ITK_COMPUTER_MEMORY_SIZE} GREATER 5 )
# Just below 2 Gigabytes
ADD_TEST(itkLargeImageWriteReadTest1 ${IO_TESTS}
itkLargeImageWriteReadTest
${ITK_TEST_OUTPUT_DIR}/LargeImage01.mhd 30000L )
# Just above 2 Gigabytes
ADD_TEST(itkLargeImageWriteReadTest2 ${IO_TESTS}
itkLargeImageWriteReadTest
${ITK_TEST_OUTPUT_DIR}/LargeImage02.mhd 40000L )
ENDIF( ${ITK_COMPUTER_MEMORY_SIZE} GREATER 5 )
As you can see, this is conditioned to the CMake variable
ITK_COMPUTER_MEMORY_SIZE.
Could you please use CMake to set this variable to a value == 6,
(since your machine has 6Gb), and run the tests ?
We run them normally on a 16Gb machine.
Please let us know what you find.
I'm going through the changes for using "size_t", and
will add a third "large image" test for checking images
above 8Gb in size.
We will see how it goes...
Luis
---------------------------------------------------------------------
On Fri, Jul 10, 2009 at 1:21 PM, Michael Jackson <
mike.jackson at bluequartz.net> wrote:
> "contributing" was being nice ;-). Sorry about that, had a long week.
> Actually you just need to create an image that is 2^32 + 1 in size. I tried
> to get something like this done in VTK but it ended up being a true API
> change which never happened.
>
> Let me know how the test goes. If you need another machine I have a 6GB
> Mac Pro that I can setup to run a few tests if needed.
>
> Thanks for the quick feedback. I think these changes, while possibly being
> a bit disruptive _now_ will save lots of bugs/issues in the future.
>
> Cheers.
> _________________________________________________________
> Mike Jackson mike.jackson at bluequartz.net
>
>
> On Jul 10, 2009, at 1:01 PM, Luis Ibanez wrote:
>
>
>> Hi Michael,
>>
>>
>> Thanks for contributing to this topic.
>>
>>
>> I'm running an experimental build by using "size_t" in
>> replacement for "long". (on a Windows 64-bits machine).
>>
>> This, most likely will trigger a cascade of changes
>> in other places where replacing "long" with "size_t"
>> will be required to provide consistency.
>>
>>
>> To be completely sure, we should add a test that is
>> equivalent to what promted Kana to post his question
>> to the list. E.g. creating an image of 8Gb or larger.
>>
>>
>>
>> Luis
>>
>>
>>
>> -----------------------
>> Michael Jackson wrote:
>>
>>> <rant>
>>> I am not a C++ expert by any stretch but with my limited 4 years of
>>> experience I have learned one thing. "Long" is the "root of all evil" ;-)
>>> IMNSHO, ITK (and VTK by extension) should absolutely BAN the use of
>>> "long" in their projects. Period. Too many avoidable bugs come up from its
>>> use. If you want a 32 bit integer use int or the standard ansi int type, if
>>> you want a 64 bit integer then use "long long int" or the standard ansi 64
>>> bit integer type.
>>> "long int" is just a mess waiting to happen. Maybe a rule should be put
>>> into the KWStyle project to look for and flag the use of 'long'?
>>> </rant>
>>> A few years back I spent the better part of 2 days tracking down an
>>> issue with VTK that turned out to be the same root problem. The use of
>>> "long" when "long long" was really needed. Every time I see something
>>> declared 'long' I then have to rethink the ramifications of using those
>>> classes or variables as I flip-flop between 32 and 64 bit compiles
>>> regularly.
>>> Just my thoughts on this friday. Take 'em or leave 'em.
>>> I vote for Luis Solution below. I _seems_ like it would solve the
>>> problem ..
>>> _________________________________________________________
>>> Mike Jackson mike.jackson at bluequartz.net
>>> On Jul 10, 2009, at 12:30 PM, Luis Ibanez wrote:
>>>
>>>> Hi Kana,
>>>>
>>>> Thanks a lot for looking into this and sharing your findings.
>>>>
>>>> I just confirmed that in Windows 64bits, the "long" type is
>>>> only 4 bytes.
>>>>
>>>> Here is the program I used:
>>>>
>>>> #include <iostream>
>>>> #include "itkOffset.h"
>>>> #include "itkNumericTraits.h"
>>>>
>>>> int main()
>>>> {
>>>> unsigned long tt;
>>>> std::cout << "size = " << sizeof(tt) << std::endl;
>>>> tt = -1;
>>>> std::cout << "tt = " << tt << std::endl;
>>>>
>>>> typedef itk::Offset<3> OffsetType;
>>>> typedef OffsetType::OffsetValueType OffsetValueType;
>>>>
>>>> OffsetValueType offsetValue;
>>>>
>>>> std::cout << "sizeof(offsetValue) = " << sizeof( offsetValue ) <<
>>>> std::endl;
>>>>
>>>> offsetValue = itk::NumericTraits< OffsetValueType >::max();
>>>>
>>>> std::cout << "OffsetValueType max() = " << offsetValue << std::endl;
>>>>
>>>> return EXIT_SUCCESS;
>>>> }
>>>>
>>>>
>>>> with this CMakeLists.txt file
>>>>
>>>>
>>>> CMAKE_MINIMUM_REQUIRED(VERSION 2.4)
>>>> IF(COMMAND CMAKE_POLICY)
>>>> CMAKE_POLICY(SET CMP0003 NEW)
>>>> ENDIF(COMMAND CMAKE_POLICY)
>>>>
>>>>
>>>> PROJECT(64bitsTest)
>>>>
>>>> FIND_PACKAGE(ITK REQUIRED)
>>>> INCLUDE(${ITK_USE_FILE})
>>>>
>>>> ADD_EXECUTABLE(typesTest typesTest.cxx )
>>>>
>>>> TARGET_LINK_LIBRARIES(typesTest ITKCommon)
>>>>
>>>>
>>>>
>>>> ---------
>>>>
>>>>
>>>> So, it seems that we have to introduce a new ITK type,
>>>> that will be the longest int available in the platform.
>>>>
>>>>
>>>> We currently have the following declarations in the file:
>>>>
>>>> Insight/Code/Common/itkIntTypes.h
>>>>
>>>>
>>>> /** Convenient and more descriptive integer types. */
>>>> typedef char ITK_INT8;
>>>> typedef int ITK_INT32;
>>>>
>>>> #ifndef _WIN32
>>>> typedef long long ITK_INT64;
>>>> #endif
>>>>
>>>> #ifdef _WIN32
>>>> typedef long ITK_INT64;
>>>> #endif
>>>>
>>>> typedef unsigned char ITK_UINT8;
>>>> typedef unsigned short ITK_UINT16;
>>>> typedef unsigned ITK_UINT32;
>>>>
>>>> #ifndef _WIN32
>>>> typedef unsigned long long ITK_UINT64;
>>>> #endif
>>>>
>>>> #ifdef _WIN32
>>>> typedef unsigned long ITK_UINT64;
>>>> #endif
>>>>
>>>> typedef int ITK_INTPTR;
>>>> typedef unsigned ITK_UINTPTR;
>>>>
>>>> #ifdef __cplusplus
>>>> }
>>>> #endif
>>>>
>>>>
>>>>
>>>> One option that comes to mind is that we
>>>> should simply use
>>>>
>>>>
>>>> size_t
>>>> or std::size_t
>>>>
>>>> Which is the type expected by the "mem"
>>>> methods, {memcpy,memdup...}, and the
>>>> allocation "new" method.
>>>>
>>>> I have verified that "size_t" will have 8 bits
>>>> on Windows 64.
>>>>
>>>> std::cout << "sizeof( size_t ) = " << sizeof( size_t ) << std::endl;
>>>>
>>>>
>>>>
>>>> Unless there are any objections, I'll suggest
>>>> that we replace the type of all integer variables
>>>> related to image offsets and image size, with
>>>> the type "size_t".
>>>>
>>>>
>>>> BTW: note that there is also "size_type"
>>>>
>>>> It seems that "size_type" will be the type used by STL
>>>> containers.
>>>>
>>>>
>>>> Any comments ?
>>>>
>>>>
>>>> Thanks
>>>>
>>>>
>>>> Luis
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------------------------------------
>>>> On Thu, Jul 9, 2009 at 2:53 AM, Arunachalam Kana <
>>>> Kana.Arunachalam at fh-wels.at > wrote:
>>>> Hi Luis,
>>>>
>>>> Thank you for your response. I ran the test program and the given the
>>>> results are given below along with detailed
>>>> System information and what option i used to compile itk.
>>>>
>>>> System Information
>>>> ------------------
>>>> Time of this report: 7/8/2009, 17:16:29
>>>> Machine name: CT-DELL
>>>> Operating System: Windows XP Professional x64 Edition (5.2, Build
>>>> 3790) Service Pack 2 (3790.srv03_sp2_gdr.090319-1204)
>>>> Language: English (Regional Setting: German)
>>>> System Manufacturer: Dell Inc.
>>>> System Model: Precision WorkStation T7400
>>>> BIOS: Default System BIOS
>>>> Processor: Intel(R) Pentium(R) III Xeon-Prozessor (8 CPUs),
>>>> ~3.2GHz
>>>> Memory: 65534MB RAM
>>>> Page File: 717MB used, 65267MB available
>>>> Windows Dir: C:\WINDOWS
>>>> DirectX Version: DirectX 9.0c (4.09.0000.0904)
>>>> DX Setup Parameters: Not found
>>>> DxDiag Version: 5.03.3790.3959 32bit Unicode
>>>>
>>>>
>>>> Itk compilation
>>>> ---------------------
>>>> Itk compiled using Microsoft visual studio 2009 x64 (option).
>>>>
>>>> Microsoft visual studio 2008
>>>> ---------------------
>>>> Configuration manager details:
>>>> Active Solution Configuration: Debug
>>>> Active Solution Platform: x64
>>>>
>>>> Test run details:
>>>> 1. Program:
>>>> unsigned long tt;
>>>> std::cout << "size = " << sizeof(tt) << std::endl;
>>>> tt = -1;
>>>> std::cout << "tt = " << tt << std::endl;
>>>> output: size = 4; tt = 4294967295
>>>>
>>>> 2. Program:
>>>> unsigned long long tt;
>>>> std::cout << "size = " << sizeof(tt) << std::endl;
>>>> tt = -1;
>>>> std::cout << "tt = " << tt << std::endl;
>>>> output: size = 8; tt = 18446744073709551615
>>>>
>>>> I changed the configuration details:
>>>> --------------------------------------
>>>> Active Solution Configuration: Debug
>>>> Active Solution Platform: Win32
>>>>
>>>> Test run details:
>>>> 1. Program:
>>>> unsigned long tt;
>>>> std::cout << "size = " << sizeof(tt) << std::endl;
>>>> tt = -1;
>>>> std::cout << "tt = " << tt << std::endl;
>>>> output: size = 4; tt = 4294967295
>>>>
>>>> 2. Program:
>>>> unsigned long long tt;
>>>> std::cout << "size = " << sizeof(tt) << std::endl;
>>>> tt = -1;
>>>> std::cout << "tt = " << tt << std::endl;
>>>> output: size = 8; tt = 18446744073709551615
>>>>
>>>> For both Win32 and x64 i get the same result.
>>>> Unsigned long is 4 byte and unsigned long long is 8 byte.
>>>>
>>>> I was a little confused after the result, so i searched for data type
>>>> ranges in msdn. Link below:
>>>>
>>>> http://msdn.microsoft.com/en-us/library/s3f49ktz.aspx
>>>>
>>>> From the information given in above link, i understand that for 32 bit
>>>> and 64 bit compiler unsigned long is 4 byte and unsigned long long is 8
>>>> byte.
>>>>
>>>> I checked in the wiki for 64 bit machines (link below)
>>>>
>>>> http://en.wikipedia.org/wiki/64-bit
>>>>
>>>> In the link above, under topic "scientific data model": It is given
>>>> that
>>>> Microsoft win64 uses the LLP data model. From the above links, I
>>>> understand the following:
>>>>
>>>> 1. A 64bit machine with Microsoft Windows x64 Operating system has
>>>> 4 byte for long and 8 byte for long long.
>>>> 2. Microsoft visual studio 2009 has long as 4 byte and long long
>>>> as
>>>> 8 bytes for both win32 and x64 compiler option.
>>>>
>>>> What's next?
>>>>
>>>> Thank you,
>>>> Regards,
>>>> Kana
>>>>
>>>>
> _____________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.itk.org/mailman/listinfo/insight-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/pipermail/insight-users/attachments/20090710/e681161d/attachment-0001.htm>
More information about the Insight-users
mailing list