SimpleITK

Simple ITK

Goals

• Provide an easy-access layer to ITK for non-C++ expert developers

Advisory Review Board (ARB)

• The Advisory Review Board is composed of groups and individual who are potential users of the Simple ITK Layer.
• They provide advice to ITK developers of the simple layer regarding the design and implementation of the SimpleITK API.

• Advisory Review Board

Examples of Other Librarires

• CIMG: http://cimg.sourceforge.net

Sandbox

• Dan's: http://github.com/dblezek/SimpleITK (This is currently the "unofficial" SimpleITK repository)
• Gabe's: http://github.com/gabehart/SimpleITK (Experiments with multi-image filters, e.g. AddImage)
• Brad's: http://github.com/blowekamp/SimpleITK (Meta-programming experiment)

Scope

Current open questions

• Internal execution mechanism
  • switch statement, supported by macros
  • Factory mechanism, new image types could be added at runtime
  • Meta-programming
• Datatypes
  • What to support "out of the box"?
  • How easy/difficult to add new datatypes to a build?
• Compile times
  • Templates radically extend CPU and memory requirements
• Library size / scope
  • Custom SWIG files vs. automatically generated SWIG
  • Automation for SimpleITK facade generation from ITK classes?

Suggested Areas to Cover

• IO
• Basic Filters
• Registration
  • Basic Framework
  • Deformable registration
    • BSplines
    • Demons
• Segmentation
  • Region Growing
  • Level Sets
  • Classifiers
• Meshes ? (Not Yet)

Types

• Image pixel types supported
  • int8_t
  • int16_t
  • int32_t
  • float
• Image dimensions supported
  • 3D
  • Is 2D managed as a 3D image of 1 slice ?
    • Performance penalty for neighborhood filters... ?
    • Could be "2D" versions of 3D filters, i.e. RecursiveGaussianFilter2D. This would work on a slice-by-slice basis.

Languages

Wrapped languages will be any that SWIG supports, if someone is willing to write tests / any needed glue code.

• C++ Layer
• Python
• Lua
• Java
• ?

Implementation Details

• Coding Style
  • Copyright Headers & License
• Const correctness (can we avoid using const's completely?)
• Testing
  • C++ testing (Google Test?)
  • Python tests (PyUnit?)
  • Lua tests (?!?)
  • Java tests (JUnit)
  • ? other languages ?
• Managing basic types
  • Arrays
  • Regions
  • Transforms
• Not so basic types (could these be enums passed as ivars?)
  • Interpolators
  • Optimizers
  • Metrics

Plan

• Image classes
• IO

TCons

• 7/15/2010

• 7/16/2010

• 7/22/2010

Tcon topics 7/22/2010

• Should SimpleITK be mixable with VTK?
  • SimpleImage class include casting to vtkImage?
  • Concept: interactive programming languages are really cool if they include data visualization. We need to have a way to visualize images as we process them.

Tcon topics 8/12/2010

Notes from discussions held during the tcon:

• Gabe Hart reported on experiments with wrapping SimpleITK for Java
  • How to manage classpath ?
  • How to organize ITK classes into jar file.
    • Should they follow the distribution of "modular ITK" ?
• Synergy between WrapITK and SimpleITK
  • Could we use WrapITK infrastructure to generate the instantiations of SimpleITK ?
• Brad L. reported on experiments using template metaprogramming for selecting pixel types
  • Example implemented: http://github.com/blowekamp/SimpleITK/network
• Discussion about what pixel types to provide
  • Should they be extendable by a user ?
  • How about "unsigned long" ? (or is it Label Maps what we really want ?)
• Label Maps
  • We agree in that they should be a first class citizen object, at the pair of the image
  • How to map LabelObjects from one image grid coordinates to another ?
    • Do we need a "resample label map" filter ?
• Procedural look and feel
  • Should SimpleITK has a procedural look ? (ala Matlab ?)
  • Or the sequence of "create object, set parameter, set parameter.... execute"
  • Can we have both ?
    • Without one diminishing the other ?
• What to do with basic Arrat types ?
  • Point, Vector, CovariantVector
    • How to hide their dimension ?
  • Introduce a generic "Array" class at SimpleITK level ?
    • maybe with an internal enum indicating that it is a Point, Vector or CovariantVector ?
    • Manage Dimension dynamically (or hide it internally), just like the simple Image class.
• SimpleITK making it look natural in Wrapped languages
  • Provide functionalities for converting ITK classes into classes of the target language
    • In Java: be able to convert a simpleImage into a Java BufferedImage class, and back.
    • In Python: be able to convert an simpleImage into a Python Image or at least a python array.
• SimpleITK & Visualization
  • Should we provide some simple visualization capabilities ?
  • Easy conversion from ITK simpleImage to vtkImageData
  • Basic Viewer example (maybe based on Qt+VTK: or CTK for that matter) that takes origin, spacing and orientation into account.
• What pixel types to include ?
  • Suggested for Microscopy
    • unsigned char (int8)
    • signed short (int16)
    • unsigned short (uint16)
    • signed int (int32)
    • unsigned int (uint32)
    • float (float32)
    • double

Tcon topics 8/13/2010

• How many classes to expose ?
  • Start with the classes exposed in v3d plugins (~80)
  • Grow to about ~200 ~300 classes
  • From there, listen to community feedback regarding new classes to add.
• Support for Large Data
  • SimpleITK must be usable for building real applications.
  • It must support image larger than the available RAM.
• Identify Target Customers for SimpleITK
  • Groups that are not using ITK at this point, because this consider it too complicated
  • Invite them to be early evaluators of the SimpleITK prototypes.
• Reviewing the MetaProgramming layer used by Brad L. to instantiate types
• How to manage multiple component images ?
  • Have an array of simpleITK images ?
• How to manage time ?
  • ND+t ?

Tcon topics 8/17/2010

• Next ITKv4 Meeting - Potentially October 4-6
• Additional Conference Calls held last week
  • SimpleITK
  • Microscopy
    • Summary: What changes to the ITK fundamentals required for large image sizes. There may be issues for video processing as well
      • How to deal with time as a dimension
      • Memory management (Streaming)
    • Possible changes to the image structure - Luis will make proposal to handle these issues and present to the group
      • Video processing may require a ring buffer structure
      • Iterators may need to be updated to avoid access to the buffer pointer
      • Possibility of using image adapters
    • Multi-resolution Images
      • Possibility of support for this at the level of the image reader (JPEG2000: support for 2D images in files)
      • Query image reader for different resolution images
    • Non-regular image spacing
      • Potential different time point for top and bottom of image
      • VTK Rectilinear grid handles this
      • What filters are required to handle non-uniform spacing. Can convolution be applied and then resample to regular spacing
      • Does this cause a data explosion
    • File formats
      • Desire for more file format support: probably would benefit from just 2-3 formats
      • 64 bit TIFF reader, Vendor specific private tags
      • Update to libTiff is required
      • Streaming support exists for Meta, VTK, MRC. ImageSeriesReader will stream slice-by-slice
  • WrapITK
• Sequestered reference applications
• Clean-up Releases
  • Currently running behind
  • Limited coverage for Windows machines on the Dashboard - Need to turn on BUILD_EXAMPLES for these machines
  • Need to updated OpenJPEG (version 2) and libTiff
• DICOM
  • Design for QR of PACS without writing to disk
    • Currently requires to locally save a copy of the file(s) before reading as an ITK image
    • How do you know what you are going to get (e.g. multi-planar scout images)
    • Create a database or data structure to present to the programmer and then create an image from the bulk data
    • What API is required to support this functionality
    • Data structure to hold information (DOM specification - Larry Tarbox from XIP - Stephen Alyward)
    • Potential to pass Metadata Dictionary

Tcon topics 8/26/2010

• Gerrit
  • What sort of work flow should we follow?
  • How do we get this set up correctly with git and github?
• Interoperability with ITK
  • Are we all agreed that this is a top priority?
• Flexibility w.r.t. dimensions
  • Are we all agreed that no more than 2D and 3D is needed?
• Flexibility w.r.t. types
  • Fixed set of types? (Just a thought)
  • Everything cast to doubles? (Again, just a thought)
  • Typing system
    • PixelTypeID
    • Instantiation
    • Runtime choosing of methods
    • Macros
      • Not pretty
      • Fairly intuitive
    • Meta-Programming
      • Hard to understand completely
      • Adds ability for each filter to specify what types it works for
• Dynamic casting
  • Should there be convenience methods to do this?
  • Should the user have to do it themselves?
• User seeing templates
  • Are we all agreed that the user should never see templates?
  • image creation without templates?
• Documentation
  • Doxygen
  • Tutorial?