[Insight-developers] itk performance numbers
Bradley Lowekamp
blowekamp at mail.nih.gov
Wed Jul 25 10:52:51 EDT 2012
Hello Rupert,
I see you are still using Visual Studio 8 2005, I am curious how VS10 would compare. I know that there are a couple special case optimization added to ITKv4 which VS8 is not able to take advantage off.
As others have reported we are begining to address some of these issues.
Thanks for sharing!
Brad
On Jul 25, 2012, at 10:12 AM, Rupert Brooks wrote:
> Hi,
>
> I was commenting the other day that ITK4 seems to have lower performance than ITK3. I put together a little benchmark to test this objectively. Results for a couple of systems are below. The benchmark is neither particularly sophisticated nor exact, but i have tried to make it typical of what i use ITK for. I have not tried it with Hans' patch as that got abandoned. Will follow up separately on that.
>
> In the numbers below, you will see that performance seems to have degraded across the board. One exception is the 2D image test. In this test i was lazy, and i left the image axis aligned with unit spacing. I am suspecting a special case optimization in ITK4. At the moment, this blocks my ability to upgrade my projects to ITK4, as they are performance critical. Any hints and tips to get those milliseconds back will be much appreciated.
>
> The code is at https://github.com/rupertbrooks/itkbench, i'd be curious if others get the same results, and of course, criticism and improvements are welcome.
>
> Note that the output about what processor / cores is on the system is bogus. The windows system is a core i7 with 4 cores, the linux one is a Core2 Quad. I suspect a bug in itksys::SystemInformation, but i will follow up on that separately also.
>
> In all itk builds, ITK_USE_REVIEW is on. In the itk3 builds, USE_OPTIMZIZED_REGISTRATION, and the new statistics framework are turned on.
>
> Cheers,
> Rupert
>
> Data follows.....
>
> itk3 Windows XP 32bit Visual Studio Pro 2005 Build RelWithDebInfo
> System: CAMD5C5PMN1
> Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache
> Serial #:
> Cache: -1
> Clock: 2800
> Cores: 4 cpus x 1 Cores = 4
> OSName: Windows
> Release: XP Professional
> Version: Service Pack 3 (Build 2600)
> Platform: x86
> Operating System is 32 bit
> ITK Version: 3.20.0
> Virtual Memory: Total: 2047 Available: 2021
> Physical Memory: Total:3581 Available: 1457
> Probe Name: Count Min Mean Stdev Max Total
> FilterChain_1_threads 10 0.0345688 0.0361805 0.0381523 0.0375099 0.361805
> FilterChain_2_threads 10 0.0257301 0.0265144 0.0279574 0.0282478 0.265144
> FilterChain_3_threads 10 0.022007 0.0232452 0.0245125 0.0240822 0.232452
> FilterChain_4_threads 10 0.0199661 0.0221756 0.0235205 0.0288773 0.221756
> Image2D 10 0.144909 0.14665 0.154586 0.148136 1.4665
> Image3D 10 0.19125 0.193378 0.203846 0.196495 1.93378
> MeanSquares_1_threads 10 0.612728 0.616159 0.649503 0.627789 6.16159
> MeanSquares_2_threads 10 0.359703 0.368051 0.388106 0.392696 3.68051
> MeanSquares_3_threads 10 0.268024 0.280672 0.296142 0.312557 2.80672
> MeanSquares_4_threads 10 0.199249 0.213209 0.224918 0.2243 2.13209
>
>
> itk4 Windows XP 32bit Visual Studio Pro 2005 Build RelWithDebInfo
>
> System: CAMD5C5PMN1
> Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache
> Serial #:
> Cache: -1
> Clock: 2800
> Cores: 4 cpus x 1 Cores = 4
> OSName: Windows
> Release: XP Professional
> Version: Service Pack 3 (Build 2600)
> Platform: x86
> Operating System is 32 bit
> ITK Version: 4.3.0
> Virtual Memory: Total: 2047 Available: 2008
> Physical Memory: Total:3581 Available: 1574
> Probe Name: Count Min Mean Stdev Max Total
> FilterChain_1_threads 10 0.0797005 0.0808155 0.0851898 0.0820313 0.808155
> FilterChain_2_threads 10 0.0670509 0.0921925 0.0991914 0.109177 0.921925
> FilterChain_3_threads 10 0.0648499 0.113566 0.120975 0.125893 1.13566
> FilterChain_4_threads 10 0.0536919 0.104662 0.114558 0.132114 1.04662
> Image2D 10 0.0953979 0.0965282 0.101753 0.0978088 0.965282
> Image3D 10 0.196468 0.198288 0.209017 0.200562 1.98288
> MeanSquares_1_threads 10 0.982071 0.986089 1.03944 0.996445 9.86089
> MeanSquares_2_threads 10 0.686028 0.728621 0.768821 0.799675 7.28621
> MeanSquares_3_threads 10 0.534885 0.554387 0.5846 0.578133 5.54387
> MeanSquares_4_threads 10 0.441551 0.463463 0.488903 0.502502 4.63463
>
>
> itk3 Linux 64 bit gcc 4.6.1 Build Release
> System: morrigan
> Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz
> Serial #:
> Cache: 2048
> Clock: 2003
> Cores: 4 cpus x 4 Cores = 16
> OSName: Linux
> Release: 3.0.0-21-generic
> Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012
> Platform: x86_64
> Operating System is 64 bit
> ITK Version: 3.20.1
> Virtual Memory: Total: 20795 Available: 20795
> Physical Memory: Total:8001 Available: 3416
> Probe Name: Count Min Mean Stdev
> Max Total
> FilterChain_1_threads 10 0.0309269 0.031097 0.0327796 0.031528 0.31097
> FilterChain_2_threads 10 0.021672 0.0248219 0.0261976 0.026264 0.248219
> FilterChain_3_threads 10 0.0213931 0.0217796 0.0229634 0.0228851 0.217796
> FilterChain_4_threads 10 0.0198419 0.0207421 0.0218725 0.0216701 0.207421
> Image2D 10 0.0263629 0.0264159 0.0278451 0.0267591 0.264159
> Image3D 10 0.0263779 0.0263906 0.0278181 0.026463 0.263906
> MeanSquares_1_threads 10 0.463909 0.465901 0.491113 0.473644 4.65901
> MeanSquares_2_threads 10 0.349138 0.40529 0.429438 0.487785 4.0529
> MeanSquares_3_threads 10 0.331908 0.357685 0.377834 0.400941 3.57685
> MeanSquares_4_threads 10 0.299428 0.336914 0.356878 0.395144 3.36914
>
> itk4 Linux 64-bit gcc 4.6.1 Build Release
> System: morrigan
> Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz
> Serial #:
> Cache: 2048
> Clock: 2003
> Cores: 4 cpus x 4 Cores = 16
> OSName: Linux
> Release: 3.0.0-21-generic
> Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012
> Platform: x86_64
> Operating System is 64 bit
> ITK Version: 4.3.0
> Virtual Memory: Total: 20795 Available: 20795
> Physical Memory: Total:8001 Available: 3400
> Probe Name: Count Min Mean Stdev
> Max Total
> FilterChain_1_threads 10 0.0663331 0.0668247 0.0704416 0.0681288 0.668247
> FilterChain_2_threads 10 0.0353 0.0393441 0.042048 0.0534279 0.393441
> FilterChain_3_threads 10 0.025275 0.0265869 0.0282408 0.0364931 0.265869
> FilterChain_4_threads 10 0.0208249 0.0222288 0.0235333 0.028336 0.222288
> Image2D 10 0.026361 0.026421 0.0278503 0.0266371 0.26421
> Image3D 10 0.0263991 0.0266509 0.0280954 0.0274949 0.266509
> MeanSquares_1_threads 10 0.69005 0.69459 0.732169 0.701676 6.9459
> MeanSquares_2_threads 10 0.590396 0.699826 0.739994 0.766702 6.99826
> MeanSquares_3_threads 10 0.527146 0.538844 0.568047 0.548791 5.38844
> MeanSquares_4_threads 10 0.417008 0.454278 0.480139 0.533593 4.54278
> --------------------------------------------------------------
> Rupert Brooks
> rupert.brooks at gmail.com
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.php
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.itk.org/mailman/listinfo/insight-developers
========================================================
Bradley Lowekamp
Medical Science and Computing for
Office of High Performance Computing and Communications
National Library of Medicine
blowekamp at mail.nih.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/pipermail/insight-developers/attachments/20120725/f38981c9/attachment.htm>
More information about the Insight-developers
mailing list