[Paraview] [EXTERNAL] Re: Need advice with parallel file format

[David E DeMarle]

On Mon, May 5, 2014 at 2:41 PM, Mohammad Mirzadeh <mirzadeh at gmail.com>wrote:

>
>> It is more likely the case that a serial reader will be read by everyone
>> and then ParaView will strip out everything but the local chunk on each
>> node.
>>
>
> but then isn't this still a limiting factor on the size of the file? I
> assume you would still get into trouble reading a file that is too big.
>
>>
>>
I don't think the file size itself is much of a problem these days. Even in
32 bit days you could do it with the right flags. Burlen of course has
valid points for extremely large files that you might run across in HPC
contexts.

The better readers will have each node read only its own portion from the
>> filesystem. Ideally each node's portion is located in its own file because
>> having lots of nodes read from the same big file can be problematic. I
>> don't think we have any parallel-hdf5 readers currently which efficiently
>> read in parallel from a single hdf5 file.
>>
>> Hum... that's what I was afraid of. So I assume that currently every
> processor reads the whole HDF5 file and deletes the unwanted parts?
>

Nope. In the case of XDMF every processor reads the whole XML file, and
then reads only the parts of the hdf5 file (or better yet files) that they
are responsible for.


>
>> David E DeMarle
>> Kitware, Inc.
>> R&D Engineer
>> 21 Corporate Drive
>> Clifton Park, NY 12065-8662
>> Phone: 518-881-4909
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20140505/08ca7f99/attachment.html>