[vtkusers] merging multiple vtkStructuredGrid's?

[tom fogal]

 <a530bfb5050616155867164a47 at mail.gmail.com>Peter Schmitt writes:
<snip>
>As far as performance is concerned, I will eventually be running all
>of this on a large Linux cluster using parallel vtk + MPI  Once I get
>rolling with this project, I'll try & post my results to vtkusers
>sometime in late July/early August.

sounds like fun =). i'll look forward to hearing about it.

>I have already written a small program that converts my data into the
>*.pvts (vtkXMLpStructuredGrid) format using a series of printf()
>statements.  I have verified that my data is okay by loading it into
>Paraview.   Translating from the *.pvts printf code to code that
>imports directly into vtk objects is a little confusing since I do not
>understand how to implement "pieces" of structured grids directly in
>vtk (ala this thread).

neither do I. This is something I've been wrestling with recently too.

Recently I wrote a data convertor that takes an old data format we have
and creates an XMLRectilinearGrid file. Getting all the XML correct and
such looked difficult to me, so I took the easy route -- I create 1
ridiculous sized RectilinearGrid, pass it to an
XMLRectilinearGridWriter, and then tell the Writer that I want it in
<X> pieces. The writer handles the segmenting of the data transparently
(in addition to making sure the XML is correct such that VTK can read
it).
Maybe a similar approach would work for you.

My current problem is that the streaming ("pieces") doesn't seem to
solve the large data problems. With 250 pieces in a 1.6 gig RGrid,
performance is abysmal on a workstation with 2 gigs of memory. I would
very much like to hear if you're getting similar peformance issues.

Currently I'm working on an HDF5 reader that might be 'smarter'; the
XML reader almost seems like its reading in the whole dataset right
from the bat, and then giving back the corresponding piece of that
in-memory dataset when requested.

Anyway I'm starting to ramble, but the point is I'd very much like to
hear how you (or anyone!) solve(s) the large datafile issue.

-tom

<snip>
>On 6/16/05, tom fogal <tfogal at apollo.sr.unh.edu> wrote:
>>  <a530bfb505061614571fbe2fa6 at mail.gmail.com>Peter Schmitt writes:
>> <snip>
>> >Does anyone have suggstions on how to merge the two StructuredGrids
>> >into one larger (40x20x20) structured grid?
>> 
>> vtkMergeFilter sounds like it will do what you want. If you want more
>> control, it looks like vtkMergeDataObjectFilter might give you that.
>> 
>> In the future, you could try a quick grep through:
>> 
>> http://www.vtk.org/doc/nightly/html/classes.html
>> 
>> sometimes just the name of a class is enough to clue you on what you'll
>> need to do.
>> 
>> >(eventually I will need to merge 100's or 1000's of structuredgrids together).
>> 
>> I don't know how well vtkMergeFilter scales, sorry...
>> Intuitively though, merging a thousand 40x20x20 grids sounds like it
>> will take a \emph{lot} of resources.
>> 
>> >In the end, I hope to use vtkXMLPStructuredGridWriter to store one
>> >pvts file with one structuredgrid consiting of multiple pieces.
>> 
>> I would be \emph{very} interested to hear how well that scales for you.
>> 
>> -tom
>>
>