[Paraview] Parallel file formats (again)...

[Renato N. Elias]

Can anyone shed some light above how is the support status for parallel 
file formats in ParaView?

In my lab most of the students still work with Fortran. It seems that 
"the universe nowadays only speaks C++ (and Python for scripting)" which 
force us to do an extensive evaluation for a good and well supported 
parallel file format to invest before struggling with all that mixed 
languages interface/wrapping annoyances (not everybody working with 
programs are programmers, there's still some engineers like civil, 
mechanical, chemical, etc... doing science...).

I could say that our my concerns about choosing a file format to sticky 
with is:

-- Easiness for installation and use (in this sense, Ensight is 
wonderful since we don't need extra libraries. It's insane when we need 
to compile 50 MB of libraries to link with a 2 MB program that uses just 
one routine of such library);
-- Easiness for interfacing (most of the libraries nowadays is written 
in C++ for C++ programmers which discourage its use by C and Fortran 
programs. Ok, we can always spend some time in interfacing it, but, a 
library should offer more functionality and flexibility than annoyances)
-- Portability.

Some time ago there was some interesting posts from Jean Favre and 
Dominic about this, which give us some overview about the subject.

http://www.paraview.org/pipermail/paraview/2008-May/008070.html
http://www.paraview.org/pipermail/paraview/2008-May/008071.html

My 2 cents for the discussion, *from a Fortran perspective*, is:

1). ENSIGHT:
1.1. Quite simple to implement and use (no need for extra libraries and 
all that stuff. Just a few Fortran statements do the job);
1.2. Implicit support for transient data and parallelism;
1.3. Depending on the number of processes we might have a huge number of 
small/medium files since each point and cell data variable is stored in 
one file (sometimes it can be a serious problem);
1.4. Not compressed (too bad);
1.5. Not so well supported *as a parallel format* by ParaView yet. After 
the change to deal (after PV 2.2.1) with multigroup datasets some 
functionalities were lost until reimplementation.
1.6. Supported by ParaView, Visit and Ensight (of course)

2). XML/VTK:
2.1. Almost impossible for a Fortran user to implement, so, we're forced 
to interface with VTK in order to write something;
2.2. Time series support has been introduced in some sense ;o)
2.3. It's a bit complicated to understand. Ok, it's XML and we should 
use it (and believe on it ;o) ) through some library, so, it's not 
supposed to "hand-implementation";
2.4. Encoding/compression is supported (which is really good)
2.5. It should be the most well parallel file format supported by 
ParaView (after EXODUS, maybe)
2.6. Only supported by VTK based softwares (ParaView, Visit, MayaVi)

3). XDMF/HDF5:
3.1. Same as 2.1, 2.2 and 2.3
3.2. The website describing the library is a bit down lately...
3.3. HDF5 seems a very promising file format. It has some development 
concern about its use by other scientific languages besides being 
flexible, compressed, cross platform, etc... .
3.4. From my knowledge, XDMF is supported by Ensight, ParaView and Visit 
also --> not sure about how good is that support.

4). EXODUS II:
4.1. Same as 2.1 --> I already tried more than once to find something 
about Exodus format. There's a good documentation in SANDIA/SEACAS page 
but the library is not open source (it's a license based distribution) 
which turns it a bit complicated to adopt;
4.2. Nothing to say about timea nd compression support since I never 
used it;
4.3. It must be well supported by PV since it's a Sandia's format;

regards

Renato.