[Paraview] rendering large data sets

[Berk Geveci]

As a first step, vtp is a serial format. Even though ParaView will
redistribute it after loading, the first process still has to load the
entire dataset. I am guessing that it is pretty large and does not (or
barely) fit in memory? I would start with partitioning the data at
write time into multiple vtp files and then use a vtm file to point to
those files. pvtp would work too but at this point, vtm is probably
the better choice since it does not force piece to be merged within
each process. Once you have a vtm file, you should be able to load it.
I would start with limiting glyph to fewer glyphs and see if you can
get it to work in a reasonable time.

Best,
-berk

On Sat, Feb 15, 2014 at 7:32 PM, Adam Simpson <adambsimpson at gmail.com> wrote:
> Hi,
>    I have ~1 billion 3D points I am trying to render but have so far been unsuccessful. I import poly data from a .vtp file and then apply a 2D vertex glyph filter but have been unable to get any output even after running on 32 cores for over 12 hours. Is there anything obviously wrong with the following batch script? It seems to work ok for small data sets(~10000 points).
>
> vis.py :
> -----------
> from paraview.simple import *
>
> sim_vtp = XMLPolyDataReader( FileName=['sim-1.vtp'] )
>
> Glyph = Glyph( GlyphType="2D Glyph", GlyphTransform="Transform2" )
> Glyph.GlyphType.GlyphType = 'Vertex'
> Glyph.SetScaleFactor = 0.02
> Glyph.MaximumNumberofPoints = 100000000
>
> DataRepresentation = Show()
>
> RenderView = GetRenderView()
> ResetCamera()
> Render()
> WriteImage("sim-1.png")
> ---------
>
> I launch it with something like this:
> -----------
> $ mpirun -n 32 pvbatch vis.py
> -----------
>
> The header of the .vtp data file looks like this:
> -----------
> <?xml version="1.0"?>
> <VTKFile type="PolyData" version="0.1" byte_order="LittleEndian" compressor="vtkZLibDataCompressor">
>  <PolyData>
>    <Piece NumberOfPoints="997335909" NumberOfVerts="0" NumberOfLines="0" NumberOfStrips="0" NumberOfPolys="0">
>      <PointData>
>      </PointData>
>      <CellData>
>      </CellData>
>      <Points>
>        <DataArray type="Float32" Name="Points" NumberOfComponents="3" format="binary" RangeMin="0.25980760565" RangeMax="246.47706935">
> -----------
>
> I have looked into possibly using the D3 filter to better distribute the data but am unsure if this is necessary or helpful. I have also looked at perhaps using the Clean filter to reduce the number of points as well. Any advice on how to render this in a reasonable time or what my render time expectation should be?
>
> Thanks,
> Adam
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://www.paraview.org/mailman/listinfo/paraview