[Paraview] Parallel file formats (again)...

[Chris Kees]

You might want to reconsider XDMF or something based on it. I'm not  
sure that XDMF is significantly harder to implement in fortran than  
straight HDF5. It's just a matter  of doing some additional text i/o  
on a relatively simple XML file. XDMF splits the data (with some  
redundancy) into light/meta data stored as simple XML (ascii) file and  
an HDF5 archive of the "heavy" data.  You can read and write the XML  
file directly from fortran without using the XDMF library and then use  
the HDF5 fortran API directly to write the heavy data.   You have the  
option of storing the heavy data in the XML file as text when HDF5  
isn't available (or when debugging/running on small data).   To me it  
looks like the posts you cite are pointing in this  direction though  
they were unhappy with some aspects of XDMF.  It's not clear to me  
whether it's the XDMF xml format, the documentation of that format, or  
the C API that needs work in order to make it more  useful.

Also, it sounds like you've already decided against a mixed language  
approach, but the the book by H. P. Langtangen "Python Scripting for  
Computational Science" advocates a fortran/python pairing to deal with  
some of your  general concerns.

Chris

On Jun 13, 2008, at 7:58 AM, Renato N. Elias wrote:

> Can anyone shed some light above how is the support status for  
> parallel file formats in ParaView?
>
> In my lab most of the students still work with Fortran. It seems  
> that "the universe nowadays only speaks C++ (and Python for  
> scripting)" which force us to do an extensive evaluation for a good  
> and well supported parallel file format to invest before struggling  
> with all that mixed languages interface/wrapping annoyances (not  
> everybody working with programs are programmers, there's still some  
> engineers like civil, mechanical, chemical, etc... doing science...).
>
> I could say that our my concerns about choosing a file format to  
> sticky with is:
>
> -- Easiness for installation and use (in this sense, Ensight is  
> wonderful since we don't need extra libraries. It's insane when we  
> need to compile 50 MB of libraries to link with a 2 MB program that  
> uses just one routine of such library);
> -- Easiness for interfacing (most of the libraries nowadays is  
> written in C++ for C++ programmers which discourage its use by C and  
> Fortran programs. Ok, we can always spend some time in interfacing  
> it, but, a library should offer more functionality and flexibility  
> than annoyances)
> -- Portability.
>
> Some time ago there was some interesting posts from Jean Favre and  
> Dominic about this, which give us some overview about the subject.
>
> http://www.paraview.org/pipermail/paraview/2008-May/008070.html
> http://www.paraview.org/pipermail/paraview/2008-May/008071.html
>
> My 2 cents for the discussion, *from a Fortran perspective*, is:
>
> 1). ENSIGHT:
> 1.1. Quite simple to implement and use (no need for extra libraries  
> and all that stuff. Just a few Fortran statements do the job);
> 1.2. Implicit support for transient data and parallelism;
> 1.3. Depending on the number of processes we might have a huge  
> number of small/medium files since each point and cell data variable  
> is stored in one file (sometimes it can be a serious problem);
> 1.4. Not compressed (too bad);
> 1.5. Not so well supported *as a parallel format* by ParaView yet.  
> After the change to deal (after PV 2.2.1) with multigroup datasets  
> some functionalities were lost until reimplementation.
> 1.6. Supported by ParaView, Visit and Ensight (of course)
>
> 2). XML/VTK:
> 2.1. Almost impossible for a Fortran user to implement, so, we're  
> forced to interface with VTK in order to write something;
> 2.2. Time series support has been introduced in some sense ;o)
> 2.3. It's a bit complicated to understand. Ok, it's XML and we  
> should use it (and believe on it ;o) ) through some library, so,  
> it's not supposed to "hand-implementation";
> 2.4. Encoding/compression is supported (which is really good)
> 2.5. It should be the most well parallel file format supported by  
> ParaView (after EXODUS, maybe)
> 2.6. Only supported by VTK based softwares (ParaView, Visit, MayaVi)
>
> 3). XDMF/HDF5:
> 3.1. Same as 2.1, 2.2 and 2.3
> 3.2. The website describing the library is a bit down lately...
> 3.3. HDF5 seems a very promising file format. It has some  
> development concern about its use by other scientific languages  
> besides being flexible, compressed, cross platform, etc... .
> 3.4. From my knowledge, XDMF is supported by Ensight, ParaView and  
> Visit also --> not sure about how good is that support.
>
> 4). EXODUS II:
> 4.1. Same as 2.1 --> I already tried more than once to find  
> something about Exodus format. There's a good documentation in  
> SANDIA/SEACAS page but the library is not open source (it's a  
> license based distribution) which turns it a bit complicated to adopt;
> 4.2. Nothing to say about timea nd compression support since I never  
> used it;
> 4.3. It must be well supported by PV since it's a Sandia's format;
>
> regards
>
> Renato.
>
>
>
> _______________________________________________
> ParaView mailing list
> ParaView at paraview.org
> http://www.paraview.org/mailman/listinfo/paraview

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20080613/d523f54c/attachment-0001.htm>