[vtkusers] Vtk xml file format questions.

[Moreland, Kenneth]

Greg,

I don't see any answer to your question yet. Probably because you ask hard questions. I'll give it a shot.

1. I believe you can achieve what you want with the Block tag in a vtm file. I've attached an example vtm file that makes a two level assembly.

2. I don't think there is any way to split the topology from the field information in any of the xml formats.

3. That probably depends on what type of file, encoding, and reading you are planning on doing. If you are trying to read a sub-extent of a structured data set with uncompressed raw data, then I would expect the reader to do a reasonable job seeking to the right parts of the file (although I haven't actually checked). If you are trying to do the same thing with an ASCII file or a file with compressed encoding or an unstructured grid, then the reader will probably have to read through the file. I would expect the reader will be able to seek to a particular time step. If you have a lot of nodes in a large parallel job trying to read from the same file, you are probably going to have a lot of contention in the file system in any case.

4. Yes. I just did an experiment where I created a 200^3 Wavelet source in ParaView and wrote it out in the two formats you prescribed. Writing it out as a vti file with binary compressed data (using the ZLIB encoder), I get a 35MB file. Writing out an ASCII file and then compressing with gzip with the highest compression flag, I get a 46MB file. The binary file probably also has higher precision.

5. Probably. It sounds like you've already outgrown the XML files. Speaking from experience in the ParaView project, we've spent a lot of time making sure Exodus files work well with multi block unstructured grids. A lot of folks have had good luck using the XDMF format (over HDF5). I think Kitware is also working on general ADIOS readers/writers, but I don't think they are done with that yet.

-Ken

From: Greg Schussman <greg.schussman at gmail.com<mailto:greg.schussman at gmail.com>>
Reply-To: "greg.schussman at gmail.com<mailto:greg.schussman at gmail.com>" <greg.schussman at gmail.com<mailto:greg.schussman at gmail.com>>
Date: Tuesday, May 6, 2014 6:40 PM
To: "vtkusers at vtk.org<mailto:vtkusers at vtk.org>" <vtkusers at vtk.org<mailto:vtkusers at vtk.org>>
Subject: [EXTERNAL] [vtkusers] Vtk xml file format questions.

Hi.

I've been googling around and reading up on the vtk xml file format, but not having much luck understanding a few things in particular.

First, it seems that there are parts within the file, but there isn't support for grouping those parts into groups so that one can easily toggle visibility of various groups.  Or (hopefully), have I missed something?

Second, there seems to be a strong notion of keeping the data with the mesh.  But for my purposes, there is too much data to fit in the same file as the mesh.  Splitting the big file into multiple smaller files, but having a redundant copy of the same mesh in each file would be overly space-consuming.   I've seen older discussion of this (for timesteps, I think), writing a mesh to one file, and data as vtkDataObjects to another, and then using vtkMergeDataObjectFilter to combine the two.  But that looked like it was using the legacy file format.  Is it possible to do this with the vtk xml file format?  Would the data be written to a .vtu file where the points and cells are just plain missing?  Or would that be an invalid file from the reader's perspective?  (by the way, is there a dtd avaliable for the vtk xml format?  I wasn't able to find one).

Third, is the vtk xml format friendly to only reading parts of a large file, while still performing well?  That is, if the mesh and all associated data were shoveled into one big file (tens to hundreds of gigabytes, with one mesh, and possibly hundreds of attributes), could only the mesh and specific attributes be extracted quickly and easily?

Fourth, for compressed data in the XML file (raw, appended, if I understand right), does the overall file size compare favorably with a purely ASCII file that has been gzipped?

Lastly, am I barking up the wrong tree?  The vtk xml format, as I understand it, looks pretty nice.  But for the kinds of things I'm asking about here, would some other format be better suited?  Or some other approach to using this format?

Thanks.

Greg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.vtk.org/pipermail/vtkusers/attachments/20140512/903d08cc/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Assembly.vtm
Type: application/octet-stream
Size: 637 bytes
Desc: Assembly.vtm
URL: <http://www.vtk.org/pipermail/vtkusers/attachments/20140512/903d08cc/attachment.obj>