VTK/ImageBenchmark

In VTK 7.1, a benchmarking utility called ImageBenchmark was added to the VTK examples. This utility makes it easy to benchmark certain VTK image filters, in order to assist developers who wish to optimize their performance. The source code is in VTK/Examples/ImageProcessing/Cxx/. The benchmarking utility is written in C++ to make it available to as broad a variety of developers as possible, and also to avoid any concerns that wrapping might invalidate the results (though it most certainly would not).

Sample Output

When run with no arguments, ImageBenchmark will give results for a variety of filters, as shown below. Results are reported in 'megavoxels per second', where a higher number indicates faster execution. The formatting is CSV (comma-separated value), which can be read by most plotting packages.

191.353,colormap:components=3
180.957,colormap:components=4
824.309,colormap:components=1:greyscale
799.908,colormap:components=2:greyscale
562.842,colormap:components=3:greyscale
452.914,colormap:components=4:greyscale
417.646,resize:kernelsize=1
151.029,resize:kernelsize=2
82.5898,resize:kernelsize=4
47.8107,resize:kernelsize=6
958.911,reslice:kernel=nearest:rotation=0/0/0/1
550.164,reslice:kernel=nearest:rotation=90/0/0/1
29.5369,reslice:kernel=nearest:rotation=90/0/1/0
101.706,reslice:kernel=nearest:rotation=45/0/0/1
39.1603,reslice:kernel=nearest:rotation=60/0/1/1
25.3316,reslice:kernel=linear:rotation=60/0/1/1
11.5771,reslice:kernel=cubic:rotation=60/0/1/1
8.26904,reslice:kernel=bspline:rotation=60/0/1/1
3.64946,reslice:kernel=sinc:rotation=60/0/1/1
5.42244,reslice:kernel=sinc:rotation=60/0/1/1:stencil
70.2169,gaussian:kernelsize=3
14.3122,convolve:kernelsize=3
7.81935,separable:kernelsize=3
111.109,resize:kernelsize=3
9.00278,median:kernelsize=3
1239.28,histogram
2086.42,histogram:stencil
11.6615,bspline:degree=3

Options

There are several command-line options that can be used to control how the benchmarking is performed.

--runs N

The number of runs to perform, 6 is recommended for accurate results

--threads N

Request a specific number of threads

--threads N-M

Repeat benchmark for each number of threads N through M

Here is the full list of options:

Usage: ImageBenchmark [options]

Options:
  --runs N                      The number of runs to perform
  --threads N (or N-M or N,M,O) Request a certain number of threads
  --split-mode slab|beam|block  Use the specified splitting mode
  --enable-smp on|off           Use vtkSMPTools vs. vtkMultiThreader
  --clear-cache MBytes          Attempt to clear CPU cache between runs
  --bytes-per-piece N           Ask for N bytes per piece [65536]
  --min-piece-size XxYxZ        Minimum dimensions per piece [1x1x1]
  --size XxYxZ                  The image size [256x256x256]
  --type uchar|short|float      The data type for the input [uchar]
  --source <imagegsource>        Set the data source [random voxels]
  --filter <filter>[:options]   Set the filter to benchmark [median]
  --output filename.png         Output middle slice as a png file.
  --units mvps|mvptps|seconds   The output units (see below for details).
  --header                      Print a header line before the results.
  --verbose                     Print verbose output to stdout.
  --version                     Print the VTK version and exit.
  --help                        Print this message.

This program prints benchmark results to stdout in csv format.  The default
units are megavoxels per second, but the --units option can specify units
of seconds, megavoxels per second (mvps), or megavoxels per thread per
second (mvptps).

If more than three runs are done (by use of --runs), then the mean and
standard deviation over all of the runs except the first will be printed
(use --header to get the column headings).

Sources: these are how the initial data set is produced.
  gaussian    A centered 3D gaussian.
  noise       Pseudo-random noise.
  grid        A grid, for checking rotations.
  mandelbrot  The mandelbrot set.

Filters: these are the algorithms that can be benchmarked.
  median:kernelsize=3        Test vtkImageMedian3D.
  reslice:kernel=nearest     Test vtkImageReslice (see below).
  resize:kernelsize=1        Test vtkImageResize.
  convolve:kernelsize=3      Test vtkImageConvolve.
  separable:kernelsize=3     Test vtkImageSeparableConvolution.
  gaussian:kernelsize=3      Test vtkImageGaussianSmooth.
  bspline:degree=3           Test vtkImageBSplineCoefficients.
  fft                        Test vtkImageFFT.
  histogram:stencil          Test vtkImageHistogram.
  colormap:components=3      Test vtkImageMapToColors.

The reslice filter takes the following options:
  stencil                    Spherical stencil (ignore voxels outside).
  kernel=nearest|linear|cubic|sinc|bspline   The interpolator to use.
  kernelsize=4               The kernelsize (sinc, bspline only).
  rotation=0/0/0/0           Rotation angle (degrees) and axis.

The colormap filter takes the following options:
  components=3               Output components (3=RGB, 4=RGBA).
  greyscale                  Rescale but do not apply a vtkLookupTable.