[Insight-developers] itk performance numbers
Rupert Brooks
rupert.brooks at gmail.com
Wed Jul 25 10:12:55 EDT 2012
Hi,
I was commenting the other day that ITK4 seems to have lower performance
than ITK3. I put together a little benchmark to test this objectively.
Results for a couple of systems are below. The benchmark is neither
particularly sophisticated nor exact, but i have tried to make it typical
of what i use ITK for. I have not tried it with Hans' patch as that got
abandoned. Will follow up separately on that.
In the numbers below, you will see that performance seems to have degraded
across the board. One exception is the 2D image test. In this test i was
lazy, and i left the image axis aligned with unit spacing. I am suspecting
a special case optimization in ITK4. At the moment, this blocks my ability
to upgrade my projects to ITK4, as they are performance critical. Any
hints and tips to get those milliseconds back will be much appreciated.
The code is at https://github.com/rupertbrooks/itkbench, i'd be curious if
others get the same results, and of course, criticism and improvements are
welcome.
Note that the output about what processor / cores is on the system is
bogus. The windows system is a core i7 with 4 cores, the linux one is a
Core2 Quad. I suspect a bug in itksys::SystemInformation, but i will
follow up on that separately also.
In all itk builds, ITK_USE_REVIEW is on. In the itk3 builds,
USE_OPTIMZIZED_REGISTRATION, and the new statistics framework are turned on.
Cheers,
Rupert
Data follows.....
itk3 Windows XP 32bit Visual Studio Pro 2005 Build RelWithDebInfo
System: CAMD5C5PMN1
Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache
Serial #:
Cache: -1
Clock: 2800
Cores: 4 cpus x 1 Cores = 4
OSName: Windows
Release: XP Professional
Version: Service Pack 3 (Build 2600)
Platform: x86
Operating System is 32 bit
ITK Version: 3.20.0
Virtual Memory: Total: 2047 Available: 2021
Physical Memory: Total:3581 Available: 1457
Probe Name: Count Min Mean
Stdev Max Total
FilterChain_1_threads 10 0.0345688 0.0361805
0.0381523 0.0375099 0.361805
FilterChain_2_threads 10 0.0257301 0.0265144
0.0279574 0.0282478 0.265144
FilterChain_3_threads 10 0.022007 0.0232452
0.0245125 0.0240822 0.232452
FilterChain_4_threads 10 0.0199661 0.0221756
0.0235205 0.0288773 0.221756
Image2D 10 0.144909 0.14665
0.154586 0.148136 1.4665
Image3D 10 0.19125 0.193378
0.203846 0.196495 1.93378
MeanSquares_1_threads 10 0.612728 0.616159
0.649503 0.627789 6.16159
MeanSquares_2_threads 10 0.359703 0.368051
0.388106 0.392696 3.68051
MeanSquares_3_threads 10 0.268024 0.280672
0.296142 0.312557 2.80672
MeanSquares_4_threads 10 0.199249 0.213209
0.224918 0.2243 2.13209
itk4 Windows XP 32bit Visual Studio Pro 2005 Build RelWithDebInfo
System: CAMD5C5PMN1
Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache
Serial #:
Cache: -1
Clock: 2800
Cores: 4 cpus x 1 Cores = 4
OSName: Windows
Release: XP Professional
Version: Service Pack 3 (Build 2600)
Platform: x86
Operating System is 32 bit
ITK Version: 4.3.0
Virtual Memory: Total: 2047 Available: 2008
Physical Memory: Total:3581 Available: 1574
Probe Name: Count Min Mean
Stdev Max Total
FilterChain_1_threads 10 0.0797005 0.0808155
0.0851898 0.0820313 0.808155
FilterChain_2_threads 10 0.0670509 0.0921925
0.0991914 0.109177 0.921925
FilterChain_3_threads 10 0.0648499 0.113566
0.120975 0.125893 1.13566
FilterChain_4_threads 10 0.0536919 0.104662
0.114558 0.132114 1.04662
Image2D 10 0.0953979 0.0965282
0.101753 0.0978088 0.965282
Image3D 10 0.196468 0.198288
0.209017 0.200562 1.98288
MeanSquares_1_threads 10 0.982071 0.986089
1.03944 0.996445 9.86089
MeanSquares_2_threads 10 0.686028 0.728621
0.768821 0.799675 7.28621
MeanSquares_3_threads 10 0.534885 0.554387
0.5846 0.578133 5.54387
MeanSquares_4_threads 10 0.441551 0.463463
0.488903 0.502502 4.63463
itk3 Linux 64 bit gcc 4.6.1 Build Release
System: morrigan
Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz
Serial #:
Cache: 2048
Clock: 2003
Cores: 4 cpus x 4 Cores = 16
OSName: Linux
Release: 3.0.0-21-generic
Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012
Platform: x86_64
Operating System is 64 bit
ITK Version: 3.20.1
Virtual Memory: Total: 20795 Available: 20795
Physical Memory: Total:8001 Available: 3416
Probe Name: Count Min Mean
Stdev
Max Total
FilterChain_1_threads 10 0.0309269 0.031097
0.0327796 0.031528 0.31097
FilterChain_2_threads 10 0.021672 0.0248219
0.0261976 0.026264 0.248219
FilterChain_3_threads 10 0.0213931 0.0217796
0.0229634 0.0228851 0.217796
FilterChain_4_threads 10 0.0198419 0.0207421
0.0218725 0.0216701 0.207421
Image2D 10 0.0263629 0.0264159
0.0278451 0.0267591 0.264159
Image3D 10 0.0263779 0.0263906
0.0278181 0.026463 0.263906
MeanSquares_1_threads 10 0.463909 0.465901
0.491113 0.473644 4.65901
MeanSquares_2_threads 10 0.349138 0.40529
0.429438 0.487785 4.0529
MeanSquares_3_threads 10 0.331908 0.357685
0.377834 0.400941 3.57685
MeanSquares_4_threads 10 0.299428 0.336914
0.356878 0.395144 3.36914
itk4 Linux 64-bit gcc 4.6.1 Build Release
System: morrigan
Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz
Serial #:
Cache: 2048
Clock: 2003
Cores: 4 cpus x 4 Cores = 16
OSName: Linux
Release: 3.0.0-21-generic
Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012
Platform: x86_64
Operating System is 64 bit
ITK Version: 4.3.0
Virtual Memory: Total: 20795 Available: 20795
Physical Memory: Total:8001 Available: 3400
Probe Name: Count Min Mean
Stdev
Max Total
FilterChain_1_threads 10 0.0663331 0.0668247
0.0704416 0.0681288 0.668247
FilterChain_2_threads 10 0.0353 0.0393441
0.042048 0.0534279 0.393441
FilterChain_3_threads 10 0.025275 0.0265869
0.0282408 0.0364931 0.265869
FilterChain_4_threads 10 0.0208249 0.0222288
0.0235333 0.028336 0.222288
Image2D 10 0.026361 0.026421
0.0278503 0.0266371 0.26421
Image3D 10 0.0263991 0.0266509
0.0280954 0.0274949 0.266509
MeanSquares_1_threads 10 0.69005 0.69459
0.732169 0.701676 6.9459
MeanSquares_2_threads 10 0.590396 0.699826
0.739994 0.766702 6.99826
MeanSquares_3_threads 10 0.527146 0.538844
0.568047 0.548791 5.38844
MeanSquares_4_threads 10 0.417008 0.454278
0.480139 0.533593 4.54278
--------------------------------------------------------------
Rupert Brooks
rupert.brooks at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/pipermail/insight-developers/attachments/20120725/afdb70fc/attachment.htm>
More information about the Insight-developers
mailing list