[Paraview] caching output pipelines

[Olesen, Mark]

Hi Kent,

> You seem to be on the right track except maybe for one detail: your
> reader cannot reuse the output data structure. Actually, there is
> little value to trying to reuse it since you are already keeping the
> non-varying part (the geometry) elsewhere. In RequestData a new output
> data structure must be built.

Thanks for the extra insights. This would imply that there is no
provision for the downstream to optimize for a constant geometry source.
I'm not sure how much of a bottleneck this really represents since there
are a number of other things changing anyhow (eg, field values, viewport
..).

> I made a custom, local change to the PVD reader so it does what you
> are trying to accomplish: it creates multi-block output from an input
> data set where the geometry does not change over time. I think this
> will be included in a future release.

Would it be possible to get a copy (whole or just the relevant snippets)
for reference purposes? On-list, off-list, with any usage restrictions
... it'd all be fine with me.  I would really like to avoid coding into
a dead-end.

> But I did one thing different: I stored the geometry in a VTK data
> structure instead of the non-VTK structure you used. The main output

I'm somewhat stuck with a non-VTK structure (for tracking polyhedral
decomposition and other things), but if I understand it correctly, your
method should effectively have zero additional memory overhead anyhow.
If I assume that geometric data is in use on one form or another
somewhere in the pipeline, it shouldn't make any difference if it's
being held by the reader or by the pipeline. Could it be that it is even
a tad faster than usual, since there is no intermediate malloc/free? 

> from the reader is given a pointer to the saved geometry. I
> incremented the usage count for the geometry by one so that it was not
> deleted when the main data structure was deleted.

Hmm. Is it really necessary to increment the refcount for the cached
geometry? Or would it be possible to use a shallowcopy to send the
cached geometry to the output?

Are you saving all the pieces and blocks separately and assembling them
on-demand, or do you pre-build the final output?

I thought of this type of flow:

gather geometry bits (from file or elsewhere)

foreach part
do
  # release all fields
  part->GetCellData()->Initialize();
done


foreach field
do
  # add in each field
  foreach part
  do
    part->GetCellData()->AddArray(field);
  done
done

assemble parts to multiblockdataset.


I suspect it might be alright if the last step (assemble to
multiblockdataset) occurs directly on the output.

Am I way off base with this approach?

Thanks again,
/mark
This e-mail message and any attachments may contain 
legally privileged, confidential or proprietary Information, 
or information otherwise protected by law of EMCON 
Technologies, its affiliates, or third parties. This notice 
serves as marking of its “Confidential” status as defined 
in any confidentiality agreements concerning the sender 
and recipient. If you are not the intended recipient(s), 
or the employee or agent responsible for delivery of this 
message to the intended recipient(s), you are hereby 
notified that any dissemination, distribution or copying 
of this e-mail message is strictly prohibited. 
If you have received this message in error, please 
immediately notify the sender and delete this e-mail 
message from your computer.