[Insight-users] Memory problem

Melanie Uks meluks2010 at googlemail.com
Mon Apr 18 07:25:56 EDT 2011


Hi,

up to now I could not solve my memory problem. I used the memory probes
collector base and I think the problem is that the GUI (Qt) needs too much
memory. If I run the code without any GUI it does the operation if I run the
code with GUI it does not.
I will try now to use a 64bit system. I have no experience and I will need
some help. I will use Visual Studio 2008 Prof. Do I need a special cmake or
can I use the Windows 32-bit installer???
Thanks!!!
Melanie

2011/4/13 Lodron, Gerald <Gerald.Lodron at joanneum.at>

>  You can use the
>
>
> itkMemoryProbesCollectorBase
>
> to watch your memory. I also use time probe. Define following as global on
> top of your code:
>
> itk::MemoryProbesCollectorBase g_oMemorymeter;
>
> itk::TimeProbesCollectorBase g_oChronometer;
>
> #define itkProbesStart( text ) std::cout << "Begin: " << text << std::endl;
> g_oChronometer.Start( text ); g_oMemorymeter.Start( text )
>
> #define itkProbesStop( text ) std::cout << "End: " << text << std::endl;
> g_oChronometer.Stop( text ); g_oMemorymeter.Stop( text )
>
> #define itkProbesReport( stream ) g_oChronometer.Report( stream );
> g_oMemorymeter.Report( stream )
>
> In your code you can use it like this
>
> itkProbesStart( "Step1")
>
> do something
>
> itkProbesStart( "Step1-1")
>
> do something
>
> itkProbesStop( "Step1-1 )
>
> itkProbesStart( "Step1-2 )
>
> do something
>
> itkProbesStop( "Step1-2 )
>
> itkProbesStop( "Step1 )
>
> itkProbesStart( "Step2")
>
> itkProbesStop( "Step2")
>
> itkProbesReport(std::cout);
>
>
>
>
>  ------------------------------
> *Von:* insight-users-bounces at itk.org [mailto:insight-users-bounces at itk.org]
> *Im Auftrag von *Melanie Uks
> *Gesendet:* Mittwoch, 13. April 2011 16:25
>
> *An:* insight-users at itk.org
> *Betreff:* Re: [Insight-users] Memory problem
>
> Thanks for your replies.
>
> Neil, I splitted the image and repeated the process (see the code at the
> end of my e-mail). The strange thing I totally don't understand is why the
> processing is working when I run just this processing in a separate programm
> and not when I include the processing as a "function" in my acutal
> program.In the separate program it is just working when I split the image in
> 4 parts.
>
> Gerald, I used now for every filter: filter->SetReleaseDataFlag(true);
> There was no difference... unfortunately :-(
>
>
> 2011/4/13 Lodron, Gerald <Gerald.Lodron at joanneum.at>
>
>>  itk uses pipelining which means that there is one image in memory after
>> each filtering. So when you have
>>
>> ->image->filter1->filter2->filter3
>>
>> you get the memory of the image 4 times. If you want to release the memory
>> after each filter step you can use the release data flag of each filter
>> (SetReleaseDataFlagOn). Then you hage the memory only one time
>>
>> If you have an image of 300x300x512x3 with float (4 byte) you have 527 MB
>> memory per image, windows 32 theoretically only allows propgramms 2GB (in my
>> tests this theoretically is 1.2 GB in practice) so you only can have the
>> image stored into two vairables, in the third time the allocation will fail!
>>
>> Another solution is to use the RequestedRegion instead of
>> largestpossibleregion but this is not supported for all filters. here only a
>> part of an image is loaded and processed...I have no experience with this
>>
>> best regards
>>
>>  ------------------------------
>> *Von:* insight-users-bounces at itk.org [mailto:
>> insight-users-bounces at itk.org] *Im Auftrag von *Melanie Uks
>> *Gesendet:* Mittwoch, 13. April 2011 15:34
>> *An:* insight-users at itk.org
>> *Betreff:* Re: [Insight-users] Memory problem
>>
>>   Sorry, I don't have that much expertice: what does "individual chunks"
>> mean and how to implement?
>>
>> 2011/4/13 Neil Panjwani <paniwani at gmail.com>
>>
>>> I had similar memory allocation problems when running several CT slices
>>> in 32 bit. If you can, using 64-bit should solve your memory problems.
>>>
>>> Otherwise, I think you'd have to apply your algorithm on individual
>>> chunks at a time and combine them at the end.
>>>
>>>
>>> On Wed, Apr 13, 2011 at 9:22 AM, Melanie Uks <meluks2010 at googlemail.com>wrote:
>>>
>>>> I am running 32 bit.
>>>>
>>>>
>>>> 2011/4/13 Neil Panjwani <paniwani at gmail.com>
>>>>
>>>>> Are you running in 32 or 64 bit?
>>>>>
>>>>>   On Wed, Apr 13, 2011 at 4:47 AM, Melanie Uks <
>>>>> meluks2010 at googlemail.com> wrote:
>>>>>
>>>>>>  Hi all,
>>>>>>
>>>>>> I tried to filter a 3D image (300x300x512) with the CannyEdgeFilter. I
>>>>>> had the problem that I could not allocate the memory for the filtering.
>>>>>> Anyway I don't like the CannyEdgeFilter Implementation as one has to guess
>>>>>> which values to take for the thresholding before the gradientmagnitude image
>>>>>> exists... Therefore I wanted to write my own edge detection routine.
>>>>>>
>>>>>> The steps:
>>>>>> I generate a image (size: 300x300x512)
>>>>>> I generate a vector image (size: 300x300x512, vector dimension:3)
>>>>>> a) I split the image, process the imagepart with
>>>>>> GradientRecursiveGaussianImageFilter
>>>>>> b) I paste the processed imagepart in the vector image with the
>>>>>> PasteImageFilter
>>>>>> I repeat a) and b) until the whole image is processed
>>>>>>
>>>>>> Now the first question: Why can't I process the complete image. I have
>>>>>> to split the image and get a result... This is the error I get:
>>>>>> *itk::ExceptionObject (0151C728)
>>>>>> Location: "class itk::CovariantVector<float,3> *__thiscall
>>>>>> itk::ImportImageConta
>>>>>> iner<unsigned long,class itk::CovariantVector<float,3>
>>>>>> >::AllocateElements(unsig
>>>>>> ned long) const"
>>>>>> File:
>>>>>> h:\itk\insighttoolkit-3.16.0\code\common\itkImportImageContainer.txx
>>>>>> Line: 188
>>>>>> Description: Failed to allocate memory for image.*
>>>>>>
>>>>>> Ok, if I accept that I have to split the image, I have a second
>>>>>> question: I was able to run my example programm (code at the end of this
>>>>>> mail). Then I copied the code into my actual program. It is a programm with
>>>>>> several itk functions and Qt GUI. The only difference is that the image is
>>>>>> saved as global variable. I was not able to run the processing. I always had
>>>>>> the memory error. Why???
>>>>>>
>>>>>> Here now the code for the example processing:
>>>>>>
>>>>>> edgetest.cxx
>>>>>>
>>>>>> #include "conio.h"
>>>>>>
>>>>>> #include "itkImageFileWriter.h"
>>>>>> #include "itkImageFileReader.h"
>>>>>> #include "itkRegionOfInterestImageFilter.h"
>>>>>> #include "itkGradientRecursiveGaussianImageFilter.h"
>>>>>> #include "itkGradientToMagnitudeImageFilter.h"
>>>>>> #include "itkPasteImageFilter.h"
>>>>>> #include "itkImage.h"
>>>>>>
>>>>>> int main(int argc, char* argv[])
>>>>>> {
>>>>>>    // Verify number of parameters in command line
>>>>>>   // if( argc < 3 )
>>>>>>     // {
>>>>>>     // std::cerr << "Usage: " << std::endl;
>>>>>>     // std::cerr << argv[0] << " inputImageFile  outputVectorImageFile
>>>>>> " << std::endl;
>>>>>>     // return EXIT_FAILURE;
>>>>>>     // }
>>>>>>   typedef float   PixelType;
>>>>>>   typedef float   ComponentType;
>>>>>>   static const unsigned int Dimension = 3;
>>>>>>   typedef itk::Image< PixelType, Dimension > ImageType;
>>>>>>   typedef itk::CovariantVector< ComponentType,
>>>>>>                                 Dimension  >      OutputPixelType;
>>>>>>   typedef itk::Image< OutputPixelType, Dimension >    OutputImageType;
>>>>>>
>>>>>>
>>>>>>   ImageType::Pointer image = ImageType::New();
>>>>>>
>>>>>>   ImageType::IndexType start;
>>>>>>   for(int i = 0; i < Dimension; i++)
>>>>>>     start[i]=0;
>>>>>>
>>>>>>   ImageType::SizeType size;
>>>>>>   size[0] = 300;
>>>>>>   size[1] = 300;
>>>>>>   size[2] = 512;
>>>>>>
>>>>>>   ImageType::RegionType region;
>>>>>>   region.SetSize(size);
>>>>>>   region.SetIndex(start);
>>>>>>
>>>>>>   ImageType::SpacingType spacing;
>>>>>>   spacing[0] = 20;
>>>>>>   spacing[1] = 20;
>>>>>>   spacing[2] = 4.00493;
>>>>>>
>>>>>>   image->SetRegions(region);
>>>>>>   image->SetSpacing(spacing);
>>>>>>   image->Allocate();
>>>>>>
>>>>>>   std::cout << region << std::endl;
>>>>>>
>>>>>>   OutputImageType::Pointer vec_image = OutputImageType::New();
>>>>>>   OutputImageType::RegionType vecregion;
>>>>>>
>>>>>>   OutputImageType::SizeType vecsize;        //Size
>>>>>>   vecsize[0] = (image->GetLargestPossibleRegion().GetSize())[0];
>>>>>>   vecsize[1] = (image->GetLargestPossibleRegion().GetSize())[1];
>>>>>>   vecsize[2] = (image->GetLargestPossibleRegion().GetSize())[2];
>>>>>>   std::cout<<"size0: "<< vecsize[0]<< " size1: "<< vecsize[1] << "
>>>>>> size2: " << vecsize[2] <<std::endl;
>>>>>>   vecregion.SetSize( vecsize );
>>>>>>
>>>>>>   OutputImageType::IndexType vecstart;        //Start
>>>>>>   vecstart[0] = (image->GetOrigin())[0];
>>>>>>   vecstart[1] = (image->GetOrigin())[1];
>>>>>>   vecstart[2] = (image->GetOrigin())[2];
>>>>>>   std::cout<<" start0: "<< vecstart[0]<< " start1: "<< vecstart[1] <<
>>>>>> " start2: " << vecstart[2] <<std::endl;
>>>>>>   vecregion.SetIndex( vecstart );
>>>>>>
>>>>>>   vec_image->SetRegions(vecregion);
>>>>>>   vec_image->SetSpacing(image->GetSpacing());
>>>>>>   vec_image->Allocate();
>>>>>>
>>>>>>     // The image buffer is initialized to a particular value
>>>>>>   OutputImageType::PixelType  initialValue;
>>>>>>
>>>>>>   // A vector can initialize all its components to the
>>>>>>   // same value by using the Fill() method.
>>>>>>   initialValue.Fill( 0.0 );
>>>>>>
>>>>>>   // Now the image buffer can be initialized with this
>>>>>>   // vector value.
>>>>>>   vec_image->FillBuffer( initialValue );
>>>>>>   std::cout<< "Allocate" << std::endl;
>>>>>>
>>>>>>   typedef itk::RegionOfInterestImageFilter< ImageType, ImageType >
>>>>>> ROIFilterType;
>>>>>>   ROIFilterType::Pointer roifilter = ROIFilterType::New();
>>>>>>
>>>>>>   // Number of Regions
>>>>>>   int splitcnt_x = 2;
>>>>>>   int splitcnt_y = 2;
>>>>>>   int overlap = 15;
>>>>>>
>>>>>>   int stepcnt_x = (int) (size[0]*1.0/splitcnt_x + 0.5);
>>>>>>   int stepcnt_y = (int) (size[1]*1.0/splitcnt_y + 0.5);
>>>>>>
>>>>>>
>>>>>>   ImageType::IndexType roistart;
>>>>>>   roistart[2]=0;
>>>>>>   ImageType::SizeType roisize;
>>>>>>   roisize[2]=512;
>>>>>>   ImageType::RegionType roiregion;
>>>>>>
>>>>>>
>>>>>>   for (int cnt_x = 0; cnt_x < splitcnt_x; cnt_x++)
>>>>>>   {
>>>>>>     for (int cnt_y = 0; cnt_y < splitcnt_y; cnt_y++)
>>>>>>     {
>>>>>>         roistart[0]= cnt_x*stepcnt_x - overlap;
>>>>>>         roistart[1]= cnt_y*stepcnt_y - overlap;
>>>>>>         if(cnt_x == 0)
>>>>>>         {
>>>>>>             roistart[0] = 0;
>>>>>>             roisize[0] = stepcnt_x + overlap;
>>>>>>         }
>>>>>>         else
>>>>>>         {
>>>>>>             roisize[0] = stepcnt_x + 2*overlap;
>>>>>>         }
>>>>>>         if(roisize[0]+roistart[0] > size[0])
>>>>>>         {
>>>>>>             roisize[0] = size[0]-roistart[0];
>>>>>>         }
>>>>>>         if(cnt_y == 0)
>>>>>>         {
>>>>>>             roistart[1] = 0;
>>>>>>             roisize[1] = stepcnt_y + overlap;
>>>>>>         }
>>>>>>         else
>>>>>>         {
>>>>>>             roisize[1] = stepcnt_y + 2*overlap;
>>>>>>         }
>>>>>>         if(roisize[1]+roistart[1] > size[1])
>>>>>>         {
>>>>>>             roisize[1] = size[1]-roistart[1];
>>>>>>         }
>>>>>>
>>>>>>
>>>>>>         roiregion.SetSize(roisize);
>>>>>>         roiregion.SetIndex(roistart);
>>>>>>
>>>>>>         std::cout << "cnt_x: " << cnt_x << " cnt_y: " << cnt_y <<
>>>>>> std::endl;
>>>>>>
>>>>>>         std::cout << roiregion << std::endl;
>>>>>>
>>>>>>         std::cout << "ROI region inside image region is " <<
>>>>>> region.IsInside(roiregion) << std::endl;
>>>>>>
>>>>>>         roifilter->SetRegionOfInterest(roiregion);
>>>>>>         roifilter->SetInput(image);
>>>>>>
>>>>>>         //Filter class is instantiated
>>>>>>         typedef itk::GradientRecursiveGaussianImageFilter<ImageType,
>>>>>> OutputImageType> GradFilterType;
>>>>>>
>>>>>>         GradFilterType::Pointer gradfilter = GradFilterType::New();
>>>>>>
>>>>>>         //sigma is specified in millimeters
>>>>>>         gradfilter->SetSigma( 1.5 );
>>>>>>
>>>>>>         //  processing pipeline:
>>>>>>         gradfilter->SetInput(roifilter->GetOutput());
>>>>>>
>>>>>>
>>>>>>         typedef itk::PasteImageFilter <OutputImageType,
>>>>>> OutputImageType > PasteImageFilterType;
>>>>>>         // The SetDestinationIndex() method prescribes where in the
>>>>>> first
>>>>>>         // input to start pasting data from the second input.
>>>>>>         // The SetSourceRegion method prescribes the section of the
>>>>>> second
>>>>>>         // image to paste into the first.
>>>>>>
>>>>>>         OutputImageType::IndexType destinationIndex;
>>>>>>         destinationIndex[0] = cnt_x*stepcnt_x;
>>>>>>         destinationIndex[1] = cnt_y*stepcnt_y;
>>>>>>         destinationIndex[2] = 0;
>>>>>>
>>>>>>         PasteImageFilterType::Pointer pasteFilter =
>>>>>> PasteImageFilterType::New ();
>>>>>>         pasteFilter->SetSourceImage(gradfilter->GetOutput());
>>>>>>         pasteFilter->SetDestinationImage(vec_image);
>>>>>>
>>>>>>         OutputImageType::RegionType pasteregion;
>>>>>>         OutputImageType::IndexType pastestart;
>>>>>>         pastestart[2]=0;
>>>>>>         OutputImageType::SizeType pastesize;
>>>>>>         pastesize[2]=512;
>>>>>>
>>>>>>         pastestart[0]= overlap;
>>>>>>         pastestart[1]= overlap;
>>>>>>         pastesize[0] = stepcnt_x;
>>>>>>         pastesize[1] = stepcnt_y;
>>>>>>
>>>>>>         if(cnt_x == 0)
>>>>>>         {
>>>>>>             pastestart[0] = 0;
>>>>>>         }
>>>>>>         if(cnt_y == 0)
>>>>>>         {
>>>>>>             pastestart[1] = 0;
>>>>>>         }
>>>>>>
>>>>>>         pasteregion.SetIndex(pastestart);
>>>>>>         pasteregion.SetSize(pastesize);
>>>>>>
>>>>>>         pasteFilter->SetSourceRegion(pasteregion);
>>>>>>         pasteFilter->SetDestinationIndex(destinationIndex);
>>>>>>
>>>>>>         try
>>>>>>         {
>>>>>>             pasteFilter->Update();
>>>>>>         }
>>>>>>         catch( itk::ExceptionObject & err )
>>>>>>         {
>>>>>>             std::cerr << "ExceptionObject caught !" << std::endl;
>>>>>>             std::cerr << err << std::endl;
>>>>>>             return EXIT_SUCCESS;
>>>>>>         }
>>>>>>     }
>>>>>>     typedef itk::GradientToMagnitudeImageFilter< OutputImageType,
>>>>>> ImageType > MagFilterType;
>>>>>>     MagFilterType::Pointer magfilter = MagFilterType::New();
>>>>>>
>>>>>>     magfilter->SetInput(vec_image);
>>>>>>
>>>>>>     try
>>>>>>     {
>>>>>>         magfilter->Update();
>>>>>>         image = magfilter->GetOutput();
>>>>>>     }
>>>>>>     catch( itk::ExceptionObject & err )
>>>>>>     {
>>>>>>         std::cerr << "ExceptionObject caught !" << std::endl;
>>>>>>         std::cerr << err << std::endl;
>>>>>>         return EXIT_SUCCESS;
>>>>>>     }
>>>>>>
>>>>>>   }
>>>>>>
>>>>>>   getch();
>>>>>>   return EXIT_SUCCESS;
>>>>>> }
>>>>>>
>>>>>> _____________________________________
>>>>>> Powered by www.kitware.com
>>>>>>
>>>>>> Visit other Kitware open-source projects at
>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>
>>>>>> Kitware offers ITK Training Courses, for more information visit:
>>>>>> http://www.kitware.com/products/protraining.html
>>>>>>
>>>>>> Please keep messages on-topic and check the ITK FAQ at:
>>>>>> http://www.itk.org/Wiki/ITK_FAQ
>>>>>>
>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>> http://www.itk.org/mailman/listinfo/insight-users
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> _____________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at
>>>> http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Kitware offers ITK Training Courses, for more information visit:
>>>> http://www.kitware.com/products/protraining.html
>>>>
>>>> Please keep messages on-topic and check the ITK FAQ at:
>>>> http://www.itk.org/Wiki/ITK_FAQ
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> http://www.itk.org/mailman/listinfo/insight-users
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/pipermail/insight-users/attachments/20110418/9dfac141/attachment.htm>


More information about the Insight-users mailing list