[Insight-developers] Any appreciable pipeline overhead in SimpleITK

[Matt McCormick]

On Thu, Oct 25, 2012 at 3:00 PM, Bradley Lowekamp
<blowekamp at mail.nih.gov> wrote:
> Jim,
>
> I have not looked into the overhead of the pipeline..before. And this has not been a major concern of mine for SimpleITK.
>
> There part that does concern me is the needless overhead that in introduced by SimpleITK itself. The overhead I am talking about here is the needless reconstruction of the member function factory each time a SimpleITK filter is created [1]. In the filters constructor a new factory is created and all the instantiated functions are registered into the factory. This is really pointless work and should not be done.
>
> Ok, as I am writing this is has peaked my interest! So I just wrote a python script which is attached. I created a 1x1 image and ran the sitk::CastImageFilter. This one was chosen because it has by far the most number of functions registered and is a more complicated I also ran the Square filters. With a 1x1 sized image almost all of the work should be overhead and not image processing. And these are my results:
>
> $  ~/temp/sitkOverhead.py
> With SITK function factory creation we are able to execute the cast filter 2145.70611073 per second.
> With no SITK function factory creation we are able to execute the cast filter 4448.20061903 per second.
> With SITK function factory creation we are able to execute the square filter 4135.92337362 per second.
> With no SITK function factory creation we are able to execute the square filter 4579.52943668 per second.
>
>
> And then for fun
>
> $ export ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS=1
> $  ~/temp/sitkOverhead.py
> With SITK function factory creation we are able to execute the cast filter 3366.01247918 per second.
> With no SITK function factory creation we are able to execute the cast filter 16825.8089362 per second.
> With SITK function factory creation we are able to execute the square filter 13411.8373109 per second.
> With no SITK function factory creation we are able to execute the square filter 15961.5489267 per second.
>
> I ran this on my i7 laptop which defaults to 8 threads.
>
> So to answer your questions:
>
> No, I don't think there is significant overhead by the pipeline's update sequence.
>
> Up to 87% of the overhead execution time of the sitk::Cast is spent creating the function factory and the creation threads.
> Up to 70% of the overhead execution time is spent in the creation of threads.

Interesting analysis!

Add it to the pile of evidence that we could use a thread pool...

Matt


>
> Brad
>
>
> [1] https://github.com/SimpleITK/SimpleITK/blob/master/Code/BasicFilters/src/sitkCastImageFilter.cxx#L36
>
>
> On Oct 25, 2012, at 9:52 AM, "Miller, James V (GE Global Research)" <millerjv at ge.com> wrote:
>
>> Hey Brad,
>>
>> After the meeting on Monday, I starting wondering whether SimpleITK sees any appreciable overhead from the pipeline when Update() is called given each pipeline is only one filter long.
>>
>> Have you measured how much time SimpleITK spends chaining through the interpretative layer verses calling update verses actually running the filter?  Would it make sense to add something to ITK to shortcut some of the pipeline checks?
>>
>> I wouldn't expect much overhead but I thought I would ask whether you had looked at it.
>>
>> Jim
>>
>> Jim Miller
>> Senior Scientist
>> GE Research
>> Interventional and Therapy
>>
>> GE imagination at work
>>
>>
>>
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.php
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.itk.org/mailman/listinfo/insight-developers
>