[Paraview] Need advice with parallel file format

[Mohammad Mirzadeh]

Hi I am at a critical point in deciding I/O format for my application. So
far my conclusion is to use parallel HDF5 for restart files as they are
quite flexible and portable across systems.

When it comes to visualization, however, i'm not quite sure. Up until now
I've been using pvtu along with vtu files and although they generally work
fine, one easily gets in trouble when running big simulations on large
number of processors as the number of files can easily get out of control
and even simplest utility commands (e.g. ls) takes minutes to finish!

After many thinking I've come to a point to decide between two strategies:

1) Go with a single parallel HDF5 file that includes data for all
time-steps. This makes it all nice and portable except there are two
issues. i) It looks like doing MPI-IO might not be as efficient as separate
POSIX IO, especially on large number of processors. ii) ParaView does not
seem to be able to read HDF5 files in parallel

2) Go with the same pvtu+vtu strategy except take precautions to avoid file
explosions. I can think of two strategies here: i) use nested folders to
separate vtu files from pvtu and also each time step ii) create an IO group
communicator with much less processors that do the actual IO.

My questions are 1) Is the second approach necessarily more efficient than
MPI-IO used in HDF5? and 2) Is there any plan to support parallel IO for
HDF5 files in paraview?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20140502/d0f8dd61/attachment.html>