[Paraview] Opening PVD Files Referencing Thousands of Files

[Kent Eschenberg]

If one has a parallel application using 512 processors to generate 100 time 
steps it seems like a good idea to use the pvd file format where each 
processor can write one file per time step. This creates 51,201 files.

When the pvd file is opened it seems that ParaView tries to read all of the 
files and generate output (from vtkPVDReader) that includes the contents of 
every file. As you might guess, this takes a very long time. Is this an 
error or was the pvd format not designed for so many files?

I've altered our version to temporarily add the "restriction" that the 
timestep must be 0 when the reader sees that there are no other 
restrictions. As a result, when the file is first opened, it generates 
output from only timestep 0. Later, as one animates through the time steps, 
the animator adds a restriction for just the needed time.

Does that fix make senses in general? What have been the experience of 
others with transient data stored in thousands of files?

TIA
Kent
Pittsburgh Supercomputing Center