[Paraview-developers] VTK xml formats with external geometry references

[Mark Olesen]

Hi Berk,

If we can split up geometry/fields etc with a FileSeries, that sounds 
highly attractive. A json metafile would be a very convenient way of 
gluing everything together without reinventing the wheel.

Cheers,
/mark

On 02/19/18 17:00, Berk Geveci wrote:
> Hi Mark,
> 
> Depending on how much code you want to write and what you want to do 
> with files, this is doable.
> 
> 0. For unstructured grids, the Exodus format supports this out of box.
> 
> 1. The simplest solution past Exodus would probably be to write some 
> simple Python reader that manages arrays from various files. This could 
> easily support any format. You would simple have some files that have 
> geometry, others that have arrays...
> 
> 2. If you want a single file, the .vtx formats actually support time 
> with the option of some arrays not changing over time. At least 
> partially. This was done a long time ago and would have to be update to 
> the new time support. I can provide some guidance if necessary. I am not 
> sure if it would work with the .pvtx
> 
> 3. If you want multiple files, the pvd reader can be updated to support 
> this. Someone had done this years ago but their changes never made into 
> upstream.
> 
> 4. The FileSeries reader now supports a json based metafile. That reader 
> can be update to do this as well. It is probably an easier piece of code 
> than the pvd one and it would work with other formats.
> 
> 
> 
> On Wed, Jan 24, 2018 at 3:38 PM, Mark Olesen <Mark.Olesen at esi-group.com 
> <mailto:Mark.Olesen at esi-group.com>> wrote:
> 
>     I realize this discussion pops up at regular intervals, but I
>     haven't seen it for a while and I have some use cases that would
>     really need it.
> 
>     Seeing that there are multiblock and parallel VTK file formats such
>     as .vtm, .pvtp, .pvtu serving as collections for externally defined
>     data, I was wondering if it would be feasible to use a similar idea
>     to hold the geometry separate from the cell/point data?
>     AFAIK this isn't really something that the normal (eg, vtp, vtu)
>     formats support, but something that would really help me.
> 
>     A fairly common situation is sampling to predefined surface that is
>     reused for different times and for different fields. At the moment I
>     have the choice of writing geometry+data for each sampled field at
>     each time value. This can lead to fairly excessive disk usage.
> 
>     To avoid this duplicate geometry, I would need a complete
>     (non-trivial) rewrite of my sampling. But even after that I would
>     still have geometry duplication if the geometry is not changing in
>     time. I also lose the ability to easily remove unneeded fields a
>     posteriori.
> 
>     The desired meta-format might look like this:
> 
>     <VTKFile type="XXXUnstructuredGrid"...>
>        <XXXUnstructuredGrid>
>          <Piece file="abc.vtu" NumberOfPoints="1000" NumberOfCells="10">
>            <CellData Name="p" format="file" file="abc_p.dvtu"/>
>            <CellData Name="T" format="file" file="abc_T.dvtu"/>
>            <CellData Name="conc0" format="file" file="abc_scalar.dvtu"/>
>            <CellData Name="conc1" format="file" file="abc_scalar.dvtu"/>
>            <CellData Name="conc2" format="file" file="abc_scalar.dvtu"/>
>            <CellData Name="conc3" format="file" file="abc_scalar.dvtu"/>
>          </Piece>
>        </XXXUnstructuredGrid>
>     </VTKFile>
> 
>     This is somewhat like a pvtu file, but with the geometry and data
>     being able to reside in separate files.
> 
>     The "abc.vtu" base geometry is just a regular VTU file with or
>     without an fields.
> 
>     The "abc_T.dvtu" (dvtu for a data vtu) has data but no geometry. For
>     sanity checks it should have the basic sizing information
>     (NumberOfPoints, NumberOfCells). To be thoroughly redundant, I guess
>     a geometry file could also be specifically named in the respective
>     dvtu file.
> 
>     Is there any thing within the current framework that could be easily
>     leveraged to solve such a problem, and is such a way that it
>     eventually be part of the VTK file-format collections. Is there
>     actually a VTK format that already fits the bill and I just haven't
>     found it?
>     I know that the EnSight format covers many of these points, but a
>     pure VTK solution is what I'm looking for since it would be
>     important to have parallel file format output as per
>     PUnstructuredGrid etc.
> 
>     Cheers,
>     /mark
>     _______________________________________________
>     Powered by www.kitware.com <http://www.kitware.com>
> 
>     Visit other Kitware open-source projects at
>     http://www.kitware.com/opensource/opensource.html
>     <http://www.kitware.com/opensource/opensource.html>
> 
>     Search the list archives at:
>     http://markmail.org/search/?q=Paraview-developers
>     <http://markmail.org/search/?q=Paraview-developers>
> 
>     Follow this link to subscribe/unsubscribe:
>     https://paraview.org/mailman/listinfo/paraview-developers
>     <https://paraview.org/mailman/listinfo/paraview-developers>
> 
> 

-- 
Dr Mark OLESEN
Principal Engineer, ESI-OpenCFD
ESI GmbH | Einsteinring 24 | 85609 Munich | GERMANY
Mob. +49 171 9710 149
www.openfoam.com | www.esi-group.com | mark.olesen at esi-group.com