[Insight-users] itk filters and memory use

Tue Sep 21 09:48:41 EDT 2010

In the pipeline approach, streaming would indeed reduce memory usage,
and that would help performance to the extent that it reduces virtual
memory paging. The same number of reads and writes to RAM will be
issued, however, so if memory wasn't a problem to start with, I
wouldn't expect a performance gain by using streaming.

I think the functor image filters I cited should be compatible with
streaming, but I am not an expert in ITK's streaming facilities.

Cory

On Tue, Sep 21, 2010 at 9:38 AM, Sergio Vera <sergio.vera at alma3d.com> wrote:
> Just an additional question. Does not ITK enable pixel streaming in such
> scenarios, thus saving memory and time?
>
> Thanks
>
> On Tue, Sep 21, 2010 at 3:26 PM, Cory Quammen <cquammen at cs.unc.edu> wrote:
>>
>> For the computation you are trying to perform, chaining together a set
>> of filters will be much slower than iterating. Each filter generates a
>> new intermediate image in memory. The next filter in the pipeline will
>> read the newly generated image, generate a new image, and save it to
>> memory. Reading and writing images to memory in this manner will kill
>> your performance and use a lot of memory. Your experience bears this
>> out.
>>
>> Fortunately, ITK provides a nice mechanism for defining functors
>> (operations carried out on every pixel in one or more input images).
>> See
>>
>> UnaryFunctorImageFilter
>> BinaryFunctorImageFilter
>> TernaryFunctorImageFilter
>> NaryFunctorImageFilter
>>
>> See SubtractImageFilter for an example of how to implement a functor
>> image filter. Instead of subtracting one image from the other, your
>> functor would implement the computation you described.
>>
>> The key advantage to these filters is that the iteration details are
>> taken care of for you. All you need to do is supply the input images
>> and the set of operations to carry out.
>>
>> Hope that helps,
>> Cory
>>
>> On Tue, Sep 21, 2010 at 7:45 AM,  <devieill at irit.fr> wrote:
>> > Hi all, (sorry for duplicates)
>> >
>> > still on benching my progs on a 4g ram 2 core laptop, I face an ugly
>> > issue
>> > : code with iterators (thus not threaded) goes much faster than a code
>> > done with the filters, of course this happens for "large" 3D images
>> > (starting from 380^3 voxels).
>> >
>> > So my questions are :
>> > - are filters designed to save memory ?
>> > - is it better to process data on each filter separately ( thus no chain
>> > with connected input/output) from a memory point of view ?
>> >
>> > To better explain my purpose, I want to compute the gradient of the
>> > Total
>> > Variation in dimension 3 using the "beta" trick to ensure stability.
>> >
>> > basically it look likes, given an image src:
>> > compute
>> >  l2norm_beta = sqrt( dx(src)^2+dy(src)^2+dy(src)^2 +beta)
>> >  grad = dx(dx(src)/l2norm_beta) + dy(dy(src)/l2norm_beta) +
>> > dz(dz(src)/l2norm_beta)
>> > return grad
>> >
>> > As you can imagine lots of filter need to be connect in order to fit
>> > this
>> > computation, and sadly a simple iterator based computation works better
>> > for large image on my 4gig computer.
>> >
>> > In fact it is not clear to me whether this should be valid :
>> >
>> > typedef itk::SquareImageFilter< WorkingImageType, WorkingImageType >
>> >               SquareFilter;
>> > SquareFilter::Pointer  sqrt_op = SquareFilter::New();
>> > sqrt_op->SetInput(src);
>> > src = sqrt_op->GetOutput();
>> > src->Update();
>> >
>> > As far as I understand it, the filters will use iterators, so assuming
>> > the
>> > things are done properly (and not on all possible filters btw) the "Set"
>> > operation comes after the "Get" one, so this should not mess-up the
>> > computation and prevent some useless memory allocation.
>> >
>> > So in anyway, if there is a way to use the connecting itk pipeline
>> > without
>> > overusing the memory (and to avoid swapping...) I would be glad to hear
>> > it.
>> >
>> > Regards,
>> > de Vieilleville François
>> > _____________________________________
>> > Powered by www.kitware.com
>> >
>> > Visit other Kitware open-source projects at
>> > http://www.kitware.com/opensource/opensource.html
>> >
>> > Kitware offers ITK Training Courses, for more information visit:
>> > http://www.kitware.com/products/protraining.html
>> >
>> > Please keep messages on-topic and check the ITK FAQ at:
>> > http://www.itk.org/Wiki/ITK_FAQ
>> >
>> > Follow this link to subscribe/unsubscribe:
>> > http://www.itk.org/mailman/listinfo/insight-users
>> >
>>
>>
>>
>> --
>> Cory Quammen
>> Center for Computer Integrated Systems for Microscopy and Manipulation
>> (CISMM)
>> Department of Computer Science
>> University of North Carolina at Chapel Hill
>> http://www.cs.unc.edu/~cquammen
>> _____________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Kitware offers ITK Training Courses, for more information visit:
>> http://www.kitware.com/products/protraining.html
>>
>> Please keep messages on-topic and check the ITK FAQ at:
>> http://www.itk.org/Wiki/ITK_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.itk.org/mailman/listinfo/insight-users
>
>
>
> --
> Sergio Vera
>
>  Alma IT Systems
>  C/ Vilana, 4B, 4º 1ª
>  08022 Barcelona
>  T. (+34) 932 380 592
>  www.alma3d.com
>

-- 
Cory Quammen
Center for Computer Integrated Systems for Microscopy and Manipulation (CISMM)
Department of Computer Science
University of North Carolina at Chapel Hill
http://www.cs.unc.edu/~cquammen