[Paraview] advice on saving outputs

[Hal Canary]

On 05/19/2012 06:30 PM, Mohammad Mirzadeh wrote:
> Hi guys,
>
> I need a little bit of advice on how I'm outputting data. So far I have
> been using a simple legacy VTK format and dumping my grid and all
> sclaar/vectors into that single file. Basically I start with writing the
> grid and then append as many as variables that I like to the file. This
> has the problem that it was ascii and slow for large data sets,
> especially fro a parallel code where the solver was taking only 5 sec
> but the IO was around 20!!
>
> So I want to turn into binary and I have learned how to output binary
> vtu files. The issue is, since the cluster I'm working with has kill
> times (Max 24 hr) I also need to be able to save and resume my
> simulation. So, what I do right now is dump my variables in binary form
> into a single file that does not have any xml tag or anything -- just
> raw binary data. This is a preferred method since when I want to resume,
> I do not need to parse the xml code; I just read the binary file and
> that's it. Besides, I save on the grid information I'm writing to disk.
> For a 23M grid point, the grid information (which basically is a
> quadtree saved as a general unstructured grid with
> cell connectivity information etc) file is about 1GB whereas my actual
> data is only about 150MB. In most cases my grid does not change with
> time, so I really only need to save it once and that's enough.
>
> Now, the problem is, I want to be able to use or refer to these binary
> files using the xml tags. Here are the questions:
>
> 1) Is this supported in .vtu format? How can I refer to a binary file
> instead of dumping all the binary information in the file?
> 2) If this is not supported in .vtu, is it something xdmf can do? If so
> how? Can you please point me to a tutorial?
> 3) Is there any plan or strategy that would be better than what I have
> in mind right now? Please remember that retaining the raw binary format
> of the files is a must for me (well I do not see convincing reasons to
> give up on it, but if there is one, please let me know)
>
> Thanks
> Mohammad



Just this week, I wrote a Python program that uses the numpy.fromfile 
function and ParaView's paraview.vtk.dataset_adapter.numpyTovtkDataArray 
and paraview.vtk.io.vtkXMLPolyDataWriter functions to convert a large 
binary file to .vtp format.  It is reasonably fast.  Maybe you can use 
something like that as a post-processing step.

This separates the tasks of producing data and converting it to VTK format.


-- 
Hal Canary