[Insight-users] Memory management
Arunachalam Kana
Kana.Arunachalam at fh-wels.at
Tue Mar 29 05:28:50 EDT 2011
Hi Brad,
Thank you for your reply. As recommended by you, I will use the itkHessianImageFilter to calculate the hessian. I see the filter is multithreaded which
Was not available before, so i think it will be faster than before. On the other hand the itkHessianImageFilter calculates an output image, which i think
Will drastically increase the use of RAM compared to the itkDiscreteHessianGaussianImagefunction.h. Am i right?
However, I will have to test for the speed and for the memory usage.
Thank you for the filter.
Regards,
Kana
From: Bradley Lowekamp [mailto:blowekamp at mail.nih.gov]
Sent: 29 March 2011 06:42
To: Arunachalam Kana
Cc: insight-users at itk.org
Subject: Re: [Insight-users] Memory management
Hello Kana,
Sorry for the delay in getting this out there. I have placed what I have been working on github:
https://github.com/blowekamp/itkLocalDiscreteHessian
The itkHessianDiscreteGaussianimageFilter is not yet implemented, and I still need to migrate more testing over to the github module.
I would recommend just coping over itkHessianImageFilter into your project, writing some test to make yourself confident that it's computing the correctly, then keep an eye on the github repository.
Good luck,
Brad
On Mar 21, 2011, at 12:20 PM, Arunachalam Kana wrote:
Hi Brad,
Thank you for the reply.
In brief : My goal is to calculate the eigen vectors using eigenanalysisfunction.
Calculation of hessian with recursivegaussian occupy 12times more memory. discretegaussian is 30 times slower and not multithreaded.
My image is 10Gb and my ram is 64Gb too.
Thank you for the advice and for the information that you are already working on it. I will wait for the code.
Thank you,
Regards,
Kana
From: Bradley Lowekamp [mailto:blowekamp at mail.nih.gov]
Sent: 21 March 2011 17:03
To: Arunachalam Kana
Cc: insight-users at itk.org<mailto:insight-users at itk.org>
Subject: Re: [Insight-users] Memory management
Hello Kana,
I would strongly not recommend not taking this approach. It is rather inefficient, and version 3.20 has a bug related to this method.
I have an alternative implementation with exploits the separability of the convolution by a derivative. Simply, I implemented a discrete Hessian filter where a Gaussian convolution can occur as a prior filter. I hopefully will be able to get this to a sharable state this week.
Brad
On Mar 21, 2011, at 10:35 AM, Arunachalam Kana wrote:
Hi Vera,
Thank you for the information that the recursive Gaussian cannot be streamed.
Hi Masslawi,
I have not tried the release version , i will surely try it.
I decided to use discretehessiancalculation for calculation of hessian matrix. I thought of increasing the speed by multithreading in my own filter.
My filter takes 1 input image and gives 1 output image. So i calculate the hessian for each voxel.
The problem: My program breaks due to the following error.
ERROR MESSAGE:
Windows has triggered a breakpoint in iAnalyseGUI.exe.
This may be due to a corruption of the heap, which indicates a bug in iAnalyseGUI.exe or any of the DLLs it has loaded.
This may also be due to the user pressing F12 while iAnalyseGUI.exe has focus.
The output window may have more diagnostic information.
The breakage happens when the 3rd thread starts, and in DiscreteHessianGaussianImageFunction.txx line 116.
I need some help in solving the heap problem.
And finally the question comes to whether is it possible to multithread the DiscreteHessianGaussianImageFunction.txx ?????
As i have to work with large size data, i really need it, so if i can find the code somewhere please direct me or i would need some help to
Multithread the DiscreteHessianGaussianImageFunction.txx.
//code
template<class TInputImageType, class TOutputImageType>
void MYFilter<TInputImageType, TOutputImageType>
::ThreadedGenerateData(const OutputImageRegionType& outputRegionForThread, int threadId)
{
typename OutputImageType::Pointer output = this->GetOutput();
typename InputImageType::ConstPointer input = this->GetInput();
//set hessian function
m_HessianFunction = HessianFunctionType::New();
m_HessianFunction->SetInputImage(m_GradientMagnitudeImage );
m_HessianFunction->SetSigma( 1 );
m_HessianFunction->SetMaximumError( 0.01 );
m_HessianFunction->SetMaximumKernelWidth( 32 );
m_HessianFunction->SetNormalizeAcrossScale( 1 );
m_HessianFunction->SetUseImageSpacing( false );
m_HessianFunction->Initialize( );
//initiate iamge iterators
InputImageRegionIteratorType inIter(m_InputImage, outputRegionForThread);
OutputImageRegionIteratorType outIter(m_OutputImage, outputRegionForThread);
inIter.GoToBegin();
while ( !inIter.IsAtEnd() )
{
InputIndexType currentindex =inIter.GetIndex();
InputPixelType currentpixel = inIter.Get();
//initialise the variables
HessianFunctionType::TensorType hess_matrix;
hess_matrix = m_HessianFunction->EvaluateAtIndex( currentindex );
++inIter;
}//while
}
Thank you,
Regards,
Kana
From: Dawood Masslawi [mailto:masslawi at gmail.com]
Sent: 19 March 2011 14:48
To: Arunachalam Kana
Cc: insight-users at itk.org<mailto:insight-users at itk.org>
Subject: RE: Memory management
Hi Kana,
Have you tried to build in release mode? It can make a big difference.
Good luck,
Dawood
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>><<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>><<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
> Hi ITK Users,
>
> I need some advice on memory management and speed.
>
> My system details: 64 bit windowsXP system and 4Gb RAM.
>
> My goal: To obtain eigen vectors for 100Mb unsigned short image. Eigen
> vectors not for the whole image but for certain regions (approx occupies 40%
> of image).
>
>
>
> I have to do some preprocessing too.
>
> The pipeline is :
>
> Original image -> discretegaussian -> gradientmagnitude ->
> HessianRecursiveGaussianImageFilter->symmetric eigenvalue analysis ->
> finally 3 vector image
>
> For the above pipeline for a 100Mb image, i am running out of RAM.
>
> After reading itkmails, i decided to use streamfilter to stream part of the
> image so that the RAM usage is low. I only tested half way and the RAM was
> overloaded.
>
>
>
> I am using streamimagefilter to reduce the memory usage. After calculation
> of gradientmagnitude i have used 1.08Gb RAM. After this i need 1.2Gb RAM
>
> For hessianimageoutput, so by using streamfilter i thought i would be
> reaching 2.5Gm RAM but i reach approx 6Gb(virtual memory is used).
>
> 1. Am i doing something wrong with the stream filter ? (code given
> below)
>
> 2. I tried another option: As i do not need hessian for the whole
> image, I used discretehessianfunction for hessian calculation but it is 30
> times slower that the recursivehessian. Is discretegaussianimagefunction
> multithreaded?
>
> 3. Is there any other way to achieve speed vs RAM compromised
> solution?
>
> 4. The 100Mb data set is test data. The real data is 10Gb for which
> i will use 64 bit linux system (opensuse 11.2) . will the pipeline be
> executed in linux too?
>
> Below is my code and after application of filter show the RAM usage:
>
>
>
> char *infilename = "StreamTest.mhd";
>
> typedef itk::CovariantVector<float,3> VectorPixelType;
>
> typedef itk::Vector< VectorPixelType, 3 > EV_PixelType;
>
>
>
> //Image type
>
> typedef itk::Image<unsigned short, 3> InputImageType;
>
> typedef itk::Image<float, 3> FloatImageType;
>
> typedef
> itk::Image<EV_PixelType,3>
> EVImageType;
>
> **** RAM usage = 674Mb****
>
> //reader initialisation and reading file
>
> typedef itk::ImageFileReader<InputImageType>
> ReaderType;
>
> ReaderType::Pointer reader = ReaderType::New();
>
> reader->SetFileName( infilename );
>
> reader->Update();
>
> InputImageType::Pointer inImage = reader->GetOutput();
>
>
>
> **** RAM usage = 783Mb****
>
> typedef itk::DiscreteGaussianImageFilter<InputImageType, FloatImageType>
> RGIFType;
>
> RGIFType::Pointer gaussfilter = RGIFType::New();
>
> gaussfilter->SetInput(inImage);
>
> gaussfilter->SetVariance(4.0);
>
> gaussfilter->SetMaximumError(0.01);
>
>
>
> typedef itk::CastImageFilter<FloatImageType,InputImageType> CIFType;
>
> CIFType::Pointer castfilter = CIFType::New();
>
> castfilter->SetInput(gaussfilter->GetOutput());
>
>
>
> typedef itk::GradientMagnitudeImageFilter<InputImageType,FloatImageType>
> GMIFType;
>
> GMIFType::Pointer gmfilter = GMIFType::New();
>
> gmfilter->SetInput(castfilter->GetOutput());
>
> gmfilter->Update();
>
>
>
> FloatImageType::Pointer gmImage = gmfilter->GetOutput();
>
> **** RAM usage = 1290Mb****
>
>
>
> gaussfilter->UnRegister();
>
> castfilter->UnRegister();
>
> **** RAM usage = 1080Mb****
>
>
>
> typedef itk::HessianRecursiveGaussianImageFilter<FloatImageType>
> HessianRecursiveGaussianFilterType;
>
> typedef HessianRecursiveGaussianFilterType::OutputImageType
> HessianImageType;
>
> HessianRecursiveGaussianFilterType::Pointer hessianfilter =
> HessianRecursiveGaussianFilterType::New();
>
> hessianfilter->SetInput(gmImage);
>
> hessianfilter->SetSigma(1);
>
>
>
> typedef itk::StreamingImageFilter<HessianImageType,HessianImageType>
> StreamerType;
>
> StreamerType::Pointer streamer = StreamerType::New();
>
> streamer->SetInput( hessianfilter->GetOutput() );
>
> streamer->SetNumberOfStreamDivisions( 20 );
>
> streamer->Update();
>
> **** RAM usage = 3600Mb**** after few seconds it came to **** RAM usage =
> 6120Mb****
>
>
>
> Thank you in advance.
>
> Regards,
>
> Kana Arunachalam Kannappan
>
> Research Associate
>
> FH OÖ Forschungs & Entwicklungs GmbH
>
> Stelzhamer Strasse 23,
>
> 4600 Wels,
>
> Austria.
>
> Phone: +43 (0)7242 72811 -4420
>
> kana.arunachalam at fh-wels.at<http://www.itk.org/mailman/listinfo/insight-users>
>
> www.fh-ooe.at<http://www.fh-ooe.at>; www.3dct.at<http://www.3dct.at>
<ATT00001..txt>
========================================================
Bradley Lowekamp
Lockheed Martin Contractor for
Office of High Performance Computing and Communications
National Library of Medicine
blowekamp at mail.nih.gov<mailto:blowekamp at mail.nih.gov>
========================================================
Bradley Lowekamp
Lockheed Martin Contractor for
Office of High Performance Computing and Communications
National Library of Medicine
blowekamp at mail.nih.gov<mailto:blowekamp at mail.nih.gov>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/pipermail/insight-users/attachments/20110329/72755b00/attachment.htm>
More information about the Insight-users
mailing list