[Paraview] Memory explosion and strange behaviour -- Linux -- 'clip' -- 360 MB file needs 60 GB ??

[Brian Corrie]

Hi David,

I guess I am a bit surprised at the significant jump in memory, although 
I suppose I shouldn't be in thinking about it in more detail. Thanks for 
doing the math 8-)

Given the above, it isn't really practical to use the clip filter on 
volume data set sizes of significant size unless you have a mondo big 
memory computer. At the same time, clipping is a very powerful technique 
for exploration. It is one of my favorite filters to use when looking at 
a new dataset.

Utkarsh suggests using "Extract Subset" or "Slice" but both are somewhat 
limited compared to an arbitrary clip plane. Extract subset results in 
clipping only along the data set axes and slice doesn't show internal 
structure around the slice plane.

Are there any other alternatives to getting an arbitrary clip plane on a 
large volume data set without blowing out the memory? Has anyone come up 
with any clever tricks?

For me this is mostly curiosity driven, as I am no longer working on the 
project that sparked my Dec 2012 email, but clearly there are some 
people that are struggling with this issue still (e.g. Robert).

Brian


On 6/10/2014 11:03 AM, David E DeMarle wrote:
> On Tue, Jun 10, 2014 at 12:41 PM, Brian Corrie <bcorrie at sfu.ca
> <mailto:bcorrie at sfu.ca>> wrote:
>
>     Hi All,
>
>     I have one question regarding this conversation, as I have run into
>     this issue before. It intrigues me as to why one can't clip a
>     structured data set efficiently.
>
>     http://www.paraview.org/__pipermail/paraview/2012-__December/026976.html
>     <http://www.paraview.org/pipermail/paraview/2012-December/026976.html>
>
>     The question I have is - does clip REALLY require all that memory -
>     in my case it was a structured grid of 698x693x665 growing to in
>     excess of 20 GB (see the question on the list above). That seems
>     pretty excessive given the original size of the data set... I
>     haven't done the math on the data set size of an unstructured data
>     set, but 320MB to 20GB is a pretty big step.
>
>
> Very likely.
>
> imagedata
> origin + extent + spacing ~= 36 bytes #(x,y,z + ni,nj,nk + sx,sy,sz) *
> 4bytes per word
> (700^3 * 4)/(1024^3) = 1.27 GB #to store one scalar value on each point
>
> unstructured grid
> points array = (700^3 * 3 * 4)/(1024^3) = 3.8 GB #3=x,y,z, 4=4bytes per word
> cellarray = (699^3 * 9 * 4)/*(1024^3) = 11.45GB, #1 numverts + 8 vertex
> index
> celltype = (699^3 * 4)/(1024^3) = 1.27GB
> celllink = (700^3*9*4)/(1024^3) = 11.5GB #1 numcells + 8 using cells per
> vert
> (700^3 * 4)/(1024^3) = 1.27 GB #to store one scalar value on each point
> (Someone check me on all of that please).

T
> But yes, a clip like widget to something like Extract Subset that is
> easy to use AND preserves the data type would be a really nice thing to
> have in ParaView.
>