[Paraview] Memory explosion and strange behaviour -- Linux -- 'clip' -- 360 MB file needs 60 GB ??

[Berk Geveci]

Hi Brian,

It is actually relatively easy to achieve what you are after. You need two
parallel pipelines:

- Surface -> Clip
- Slice

You need to link the plane widget of the Clip and Slice filters such that
they are identical.

Note that although this is visually identical to clipping with a plane,
under the covers the outputs are significantly different. This approach
produces only the outer shell and is memory efficient. Whereas the clip
filter extracts all the cells inside the clip domain and is useful for
further process (calculating volume for example).

It is possible to make Clip for structured data more memory efficient by
converting only a minimal subset of the output to unstructured data and
keeping the rest structured. However, it is decent amount of work to write
such a filter and the above solution should be sufficient for most people.

Does this make sense?

-berk


On Wed, Jun 11, 2014 at 12:05 PM, Brian Corrie <bcorrie at sfu.ca> wrote:

> Hi David,
>
> I guess I am a bit surprised at the significant jump in memory, although I
> suppose I shouldn't be in thinking about it in more detail. Thanks for
> doing the math 8-)
>
> Given the above, it isn't really practical to use the clip filter on
> volume data set sizes of significant size unless you have a mondo big
> memory computer. At the same time, clipping is a very powerful technique
> for exploration. It is one of my favorite filters to use when looking at a
> new dataset.
>
> Utkarsh suggests using "Extract Subset" or "Slice" but both are somewhat
> limited compared to an arbitrary clip plane. Extract subset results in
> clipping only along the data set axes and slice doesn't show internal
> structure around the slice plane.
>
> Are there any other alternatives to getting an arbitrary clip plane on a
> large volume data set without blowing out the memory? Has anyone come up
> with any clever tricks?
>
> For me this is mostly curiosity driven, as I am no longer working on the
> project that sparked my Dec 2012 email, but clearly there are some people
> that are struggling with this issue still (e.g. Robert).
>
> Brian
>
>
>
> On 6/10/2014 11:03 AM, David E DeMarle wrote:
>
>> On Tue, Jun 10, 2014 at 12:41 PM, Brian Corrie <bcorrie at sfu.ca
>> <mailto:bcorrie at sfu.ca>> wrote:
>>
>>     Hi All,
>>
>>     I have one question regarding this conversation, as I have run into
>>     this issue before. It intrigues me as to why one can't clip a
>>     structured data set efficiently.
>>
>>     http://www.paraview.org/__pipermail/paraview/2012-__
>> December/026976.html
>>
>>     <http://www.paraview.org/pipermail/paraview/2012-December/026976.html
>> >
>>
>>     The question I have is - does clip REALLY require all that memory -
>>     in my case it was a structured grid of 698x693x665 growing to in
>>     excess of 20 GB (see the question on the list above). That seems
>>     pretty excessive given the original size of the data set... I
>>     haven't done the math on the data set size of an unstructured data
>>     set, but 320MB to 20GB is a pretty big step.
>>
>>
>> Very likely.
>>
>> imagedata
>> origin + extent + spacing ~= 36 bytes #(x,y,z + ni,nj,nk + sx,sy,sz) *
>> 4bytes per word
>> (700^3 * 4)/(1024^3) = 1.27 GB #to store one scalar value on each point
>>
>> unstructured grid
>> points array = (700^3 * 3 * 4)/(1024^3) = 3.8 GB #3=x,y,z, 4=4bytes per
>> word
>> cellarray = (699^3 * 9 * 4)/*(1024^3) = 11.45GB, #1 numverts + 8 vertex
>> index
>> celltype = (699^3 * 4)/(1024^3) = 1.27GB
>> celllink = (700^3*9*4)/(1024^3) = 11.5GB #1 numcells + 8 using cells per
>> vert
>> (700^3 * 4)/(1024^3) = 1.27 GB #to store one scalar value on each point
>> (Someone check me on all of that please).
>>
>
> T
>
>> But yes, a clip like widget to something like Extract Subset that is
>> easy to use AND preserves the data type would be a really nice thing to
>> have in ParaView.
>>
>>  _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/
> opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at:
> http://paraview.org/Wiki/ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://www.paraview.org/mailman/listinfo/paraview
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20140617/224d84ba/attachment.html>