[vtkusers] Parallel VTK

[Jim Ahrens]

A technical report entitled, "A Parallel Approach for Efficiently
Visualizing Extremely Large Time-Varying Datasets" describing this work
can be found at http://www.acl.lanl.gov/Viz/viz-papers.html. Additional
information and examples will be available over the next several weeks.

--Jim Ahrens

Charles Law wrote:

>
> We have received several questions about how to use new
> multiprocessing features in VTK (i.e. MPI with VTK).  Here is a
> summary of what has been done so far, and what we plan to do in the
> future:
>
> Kitware has been contracted to extend VTK to handle extremely large
> data sets on multiprocessing/distributed systems.  The parallel-VTK
> tri-lab project (LosAlamos, Sandia, Lawence Livermoore) is being
> funded by the ASCI/VIEWS project and is being led by Jim Ahrens from
> LANL.
>
> Controllers and MPI :
> VTK2.4 had new objects to manage VTK in multiple processes.
> Unfortunately, these objects were not operational in the official
> VTK3.1 release.  They are now working and available in the nightly
> releases.
>
> vtkMultiProcessController provides an api for communication.  It
> initially sets up the processes at the start of the program, and
> manages communication between these processes.  It can sends arrays,
> and has methods for sending vtk data objects between processes.  It
> also implements remote method invocations (RMIs).  Currently, there
> are two concrete implementations for the controller object.  The
> default vtkThreadedController just runs multiple threads and takes
> advantage of shared memory to implement the communication methods.
> vtkMPIController uses MPI for its communications.  The MPI controller
> allows VTK to be run on distributed clusters.
>
> In order to use MPI you have to do several things.  The first thing
> you need to have are the MPI libraries an header files.  Argonne Nat.
> Lab. has a free version of MPI at
> http://www-unix.mcs.anl.gov/mpi/mpich/ .  You need to edit you
> user.make to point to the MPI libraries and include directories.  Here
> is an example from my user.make:
>
> MPI_INCLUDE=-I/home/lawcc/mpich/include
> MPI_LIBS=/home/lawcc/mpich/build/LINUX/ch_p4/lib/libmpich.a
> /home/lawcc/mpich/build/LINUX/ch_p4/lib/libpmpich.a
>
> The last step is to use the option "--with-mpi" when you configure
> VTK.  This will make the vtkMPIController the default implementation
> of the vtkMultiProcessController object.  If you want to switch back
> to the threaded controller, you can set an environment variable
> (setenv
> VTK_CONTROLLER Threaded).
>
> .../vtk/contrib/examplesCxx/TestController.cxx has a simple example of
> how controllers are used.
>
> Ports
> vtkInputPort and vtkOutputPort are classes that were initially
> developed and contributed by Jim Ahrens.  They allow you to connect
> pipelines in separate processes.  They use the default controller for
> communication, and manage pipeline protocol necessary for updating the
> pipeline.  Examples are:
> .../vtk/contrib/examplesCxx/TestImagePort.cxx
> .../vtk/contrib/examplesCxx/TestStructuredGridPort.cxx
> .../vtk/contrib/examplesCxx/TestStructuredGridPort.cxx
> .../vtk/contrib/examplesCxx/TestPolyDataPort.cxx
>
>
> Data Parallelism
> To take advantage of parallelism, you can break up processing between
> multiple processes.  The easiest way to do this is to divide up the
> data and have each process handle a piece of it.  The image processing
> pipeline has always had the ability process structured volumes in
> pieces.  This featured has been used to breakup the datasets for
> streaming (pieces processed one after another).  VTK3.1 extended the
> graphics (unstructured) pipeline to also process pieces.  Unstructured
> pieces are specified by "Borg" notation: piece n of m.  It is the
> responsibility of the source to determine which cells belong in what
> piece.
>
> For an example of how unstructured pieces can be used to implement
> data parallelism take a look at:
> .../vtk/contrib/examplesCxx/ParallelIso.cxx
> In this example, the append filter asks for different pieces from its
> inputs.
>
> Although the pipeline can handle unstructured pieces, only a hand full
> of sources have been extended to supply pieces.  Most use the default
> behavior of putting the entire dataset in the first piece, and leaving
> the other pieces empty.  We are currently working on expanding the
> number of sources that can divide their output intelligently.
>
> Boundaries
> The above approach for defining pieces has a problem when filters use
> information from neighboring cells (i.e. smoothing).  The filters
> treat piece boundaries as dataset boundaries.  This will produce seams
> between pieces.  We are working on ways to solve this problem.
> Filters that require neighboring information would at least give the
> user an option to disallow pieces.  Another proposed solution is to
> implement ghost cells.  Boundary cells would be duplicated in pieces
> with a special "ghost" status.  The last solution will use custom
> communication during the execute methods of equivalent filters in
> different processes.
>
>
>
> Charles Law.
>
>
>
>
>
>
>
>
>
> _______________________________________________ This is the private
> VTK discussion list. Please keep messages on-topic. Check the FAQ
> at: vtkusers mailing list vtkusers at public.kitware.com
> http://public.kitware.com/mailman/listinfo/vtkusers