[Paraview] snappy display of inner points of large grid

[Donjan Rodic]

Continuing to battle with this...

Loading a VTK file, the CellCenters filter with "Vertex Cells" displays 
inner points but doesn't keep color information.
The PointDatatoCellData Filter stores the information in Cell Data, but 
averages colors at the region boundaries. Then using CellCenters 
(+Vertex Cells) does some additional averaging, but at least shows 
something.
Having PointDatatoCellData "Pass Point Data" allows me to access this 
original information as input to CellCenters, but that's not useful 
because it gets averaged at this point.
Can't see how to improve in that direction.

I've tried ExtractComponent, AppendAttributes and others to 
overwrite/adjust/... CellCenters' data array with the original color 
data, but I can't manage to do it with the built-in filters or Create 
Custom Filter. It's a size mismatch anyway.

Another option is to manually load the points into one cell.
Unfortunately both Programmable Filter and Programmable Source segfault 
regardless of the Output Data Set Type (ParaView 5.1.2, upgrading is 
currently undesirable due to 5.1 being the version for Ubuntu 16.10 and 
17.04, ideally the fix would be pushed in a patchlevel update).

I've tried to modify the data in pvpython operating on 
servermanager.Fetch(...).GetCellData() with AddArray(...) and 
GetArray(0).SetComponent(...), but this doesn't seem to be the right way 
to go about things.
Next up is trying to run a ProgrammableFilter within a pvpython script.

Any help on how to achieve the subject line with a reasonable amount of 
effort is welcome.


On 03.05.2017 22:38, Donjan Rodic wrote:
> I'm visualising a large grid dataset with different regions == scalar 
> colors with ParaView 5.1.2 on Ubuntu 16.10.
>
> The computation prints the colors (floats in [0,1]) of a 250^3 
> (usually larger) grid into a CSV file:
> x,y,z,color
> 0,0,0,0.5
> 0,0,1,0.5
> 0,0,2,0.3
> ...
>
> I load it with ParaView (Python script), apply TableToPoints, set 
> points.XColumn = 'x', etc. and the Points representation.
> This works really well with opacity mapping and sliding the Mapping 
> Data curve to highlight the different regions inside the grid.
> The main problem is the large size (hundreds of MB) of the CSV files. 
> Since the grid is rectilinear and equidistant, verbose x,y,z columns 
> are wasteful.
>
> Writing into a legacy VTK file with STRUCTURED_POINTS yields a nice 7x 
> size reduction, but then in ParaView the inner points are not displayed.
> I've tried using the Glyph representation, which makes the rendering 
> unusably slow (10+ seconds for a small rotation).
> 2D Glyphs with Glyph Type Vertex are not noticeably faster and lose 
> coloring.
> Applying the Shrink filter (random googled hint) freezes the GUI for 
> more than 5 minutes before I kill it. At smaller system sizes the 
> performance is bad.
>
> For the CSV->TableToPoints described initially, the resulting data 
> structure is shown in the Information tab as a Polygonal Mesh with 1 
> single cell and 15 million points, Memory: 600 MB. After loading it 
> responds and rotates snappily on a recent Core i7 & NVidia GPU.
> The VTK file loads with several million cells as well as 15 million 
> points, and a Image (Uniform Rectilinear Grid) type. Memory: 15 MB ... 
> much lower but irrelevant with 16 GB RAM.
>
> It appears what I want is to have all points in one single cell, and 
> the CSV reader is the only one I've found doing the right thing.
>
> Using RECTILINEAR_GRID and custom CELL_DATA in the VTK file doesn't 
> help: the format apparently doesn't allow to specify less than 1 cell 
> per 8 vertices (or 4 in 2D).
> Next I've tried to use a raw format, but after setting it up, ParaView 
> insists on creating millions of cells. Same experience as with VTK.
> I've also tried ExtractSurface to at least get a Polygonal Mesh from 
> the Rectilinear Grid: still too slow and too many cells.
>
> Something like a "GridToTable" filter with a subsequent TableToPoints 
> would do the job, but I can't find such a function. Or a routine to 
> merge all cells into one.
>
> Creating a custom filter via GUI doesn't work because the loaded 
> data.vtk "does not have any inputs".
>
> How do I achieve the same performance and display of inner points as 
> CSV->TableToPoints with either another file format or from the Grid 
> data structure offered by the VTK file reader?
>
> cheers
> Donjan
>
> PS: googling for
>   paraview grid "to table"
> mainly yields results for a typo in the documentation of 
> TableToStructuredGrid, saying "Converts to table to structured....".
>