<div dir="ltr">Going back to the Merge Blocks filter, I'd say that if the input grid is a vtkUnstructuredGrid and there's only a single block on that process in this case the Merge Blocks filter should do a shallow copy and exit quickly, which it doesn't do. This isn't a silver bullet fix and probably won't matter after the composite dataset rework but for what LANL wants in this case, this would be a very simple and efficient solution.<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Mar 20, 2018 at 6:00 PM, Boonthanome Nouanesengsy <span dir="ltr"><<a href="mailto:boonth@lanl.gov" target="_blank">boonth@lanl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Unless I am missing something, no reason. Seems to me that's just the<br>
way it was written. I can't see why the filter can't request 1 extra<br>
ghost-level from its input. (cc, Berk).<br>
</blockquote></span>
Because we start with cell data, we actually require two layers of ghost cells. The ghost cells filter was developed to allocate as many layers of ghost cells as needed for a distributed dataset.<span class=""><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Any other ideas how to decrease the bloat of about 4 GB of data to be about<br>
18 GB of data?<br>
Is this for a Catalyst or pvbatch pipeline or for an interactive session?<br>
</blockquote></span>
It's for an interactive session. We have a user running out of memory trying to use a certain pipeline. More information is in the issue at <a href="https://gitlab.kitware.com/paraview/paraview/issues/18035" rel="noreferrer" target="_blank">https://gitlab.kitware.com/par<wbr>aview/paraview/issues/18035</a><br>
<br>
Boonth<div class="HOEnZb"><div class="h5"><br>
<br>
<br>
On 03/20/2018 03:08 PM, Utkarsh Ayachit wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Why does merge blocks grow so much? Why can’t this filter just do shallow<br>
copies? Is this impossible, or just not implemented?<br>
</blockquote>
No, merge blocks is not a shallow copy, it needs to combine mutliple<br>
vtkDataSet's into a single vtkDataSet merging all points into a single<br>
array, same for cells, other attribute arrays etc. Merge Blocks should<br>
indeed be considered as a workaround for issue when handling<br>
mutliblock datasets in the pipeline and not the go-to solution. There<br>
is work currently underway that will revamp multiblock dataset support<br>
in VTK/ParaView which should ultimately make Merge Blocks obsolete.<br>
<br>
Looking at your pipeline, looks like you're doing MergeBlocks so that<br>
you can run the Ghost Cells generator filter. What is the nature of<br>
your data? Is it truly a multiblock or just has 1 block in it on each<br>
rank? If its just 1 block, what is the type of the block? If it's a<br>
unstructured grid, it will trivial to create a filter that just passes<br>
the input block out as unstructured grid without duplicating.<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Why can’t Merge Blocks do shallow copies? Is this impossible, or just not<br>
implemented?<br>
</blockquote>
It's impossible. See previous comment for justification based on how<br>
things are currently.<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
(I think I know this one, but here it is). Why does the ghost cell<br>
generator bloat memory so much? Can’t you just use shallow copies of the<br>
primary data, and then add the ghost cells?<br>
</blockquote>
adding ghost cell etc, means adding new elements to existing arrays.<br>
since vtk doesn't modify input arrays, when it needs to add new<br>
elements, it has to create a deep-copy and then add new elements to<br>
it.<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Why can’t we have the cell data to point data filter be smart enough to add<br>
ghost cells if necessary for calculations, then throw them away?<br>
</blockquote>
Unless I am missing something, no reason. Seems to me that's just the<br>
way it was written. I can't see why the filter can't request 1 extra<br>
ghost-level from its input. (cc, Berk).<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Any other ideas how to decrease the bloat of about 4 GB of data to be about<br>
18 GB of data?<br>
</blockquote>
Is this for a Catalyst or pvbatch pipeline or for an interactive session?<br>
<br>
Utkarsh<br>
______________________________<wbr>_________________<br>
Powered by <a href="http://www.kitware.com" rel="noreferrer" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" rel="noreferrer" target="_blank">http://www.kitware.com/opensou<wbr>rce/opensource.html</a><br>
<br>
Search the list archives at: <a href="http://markmail.org/search/?q=Paraview-developers" rel="noreferrer" target="_blank">http://markmail.org/search/?q=<wbr>Paraview-developers</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="https://public.kitware.com/mailman/listinfo/paraview-developers" rel="noreferrer" target="_blank">https://public.kitware.com/mai<wbr>lman/listinfo/paraview-develop<wbr>ers</a><br>
</blockquote>
<br>
______________________________<wbr>_________________<br>
Powered by <a href="http://www.kitware.com" rel="noreferrer" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" rel="noreferrer" target="_blank">http://www.kitware.com/opensou<wbr>rce/opensource.html</a><br>
<br>
Search the list archives at: <a href="http://markmail.org/search/?q=Paraview-developers" rel="noreferrer" target="_blank">http://markmail.org/search/?q=<wbr>Paraview-developers</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="https://public.kitware.com/mailman/listinfo/paraview-developers" rel="noreferrer" target="_blank">https://public.kitware.com/mai<wbr>lman/listinfo/paraview-develop<wbr>ers</a><br>
</div></div></blockquote></div><br></div>