[vtkusers] VTK data files and formats - addition information...

[Terry J. Ligocki]

> There is a third option.  Create a Structured grid data file with one coordinate data and
> then create a whole set of scalar values for different times like so.  Choose an appropriate
> label for each dataset at a new time.  This way you store all the data in one file with only
> one coordinate data set.  Access the different scalardata types using the readers
> SetScalarsName() function.
>
> prabhu

Thanks for the suggestion (Simon J. Rees also suggested it in an e-mail to me).  This would
work well with some of the data sets we have (and I think I may use your idea in those cases!)
but in some cases we (the visualization group I'm a member of) have 7-20 Gbytes of data
(covering 500-1000 time steps).  In this case, creating one file with everything in it and
then reading it into VTK could be problematic (of course, you didn't know this was the
case!).  Even having two copies of the data - the raw data and the VTK file(s) containing the
data is problematic.  Again, thank-you for your help and suggestions, I really appreciate the
prompt, useful feedback.  It what follows I'm going to expand what I originally said to
clarify my thoughts on this - this should NOT be taken as a response to Prabhu's or Simon's
suggestions which were quite reasonable!

In some cases we have different types of data which needs to be combined in different ways to
produce a file in one of the VTK formats.  In all cases that I've had to deal with I've either
put everything in one file (and rebuild this file if the original data is modified) or I've
put different parts in different VTK data files, read them in, and then assembled the data in
VTK.  In both cases, multiple versions of the data was needed (I needed to keep the original
data so it could be available for other analysis and I needed a copy of this data inside the
VTK file) and if the data changed I needed to rebuild the VTK file(s).

Most of this could be avoided if there was some mechanism in the VTK file format to "include"
other files.  Beyond that one could imagine having "offsets", "strides", etc. but just the
ability to reference other files would go a long way.  It could be as simple as a directive
which when read cause the reader to "stop reading the current file, continue by reading the
referenced file, and when that's done continue reading the original file".  This directive
could be added to all the VTK file formats.  If this doesn't exist already, I think it would
be very useful to add and everything would still be backward compatible.  As I said, I'd be
interested in helping to do this with some guidance if it seemed worthwhile.

Thanks for your time, help, and patience...

                Terry J. (Ligocki, tjligocki at lbl.gov, (510)486-6140)
                Visualization Group
                Lawrence Berkeley National Laboratory &
                National Energy Research Scientific Computing Center