[vtk-developers] Massive lag in a Pileline. Too complex pipeline ?

[Will Schroeder]

Just a quick response off of the top of my head:

vtkMapper::Static (i.e, SetStaticOn()) prevents the mapper from propagating
updates down the pipeline.

I have seen pipelines hacked as follows: executing them once, and then
perform a local shallow copy of the resulting data into data object(s)
(e.g., polydata) which is disconnected from the pipeline hence the updates
are fast.

I suspect there are more elegant ways but maybe this will get the
conversation going...



On Mon, Mar 5, 2018 at 11:56 AM, Jonathan Borduas <
jonathan.borduas at caboma.com> wrote:

>
>
> Hi all,
>
>
>
> VTK pipelines are great, they allow to dynamically create a flow of
> algorithms. However, we recently encountered a massive lag in our pipeline
> related to the pipeline mechanisms (as opposed to the algorithms’
> calculation time).  We are using a derivative of ParaView with homemade
> filters.
>
>
>
> We profiled our application in such a way that the profiling session
> started when all of the algorithms present in the pipeline had already been
> executed once and were up to date. The profiler pointed out two methods
> that were called a very significant amount of times while no updates
> were required on any of the algorithms of the pipeline. Those methods were:
>
>
>
> 1) vtkInformation::GetAsObjectBase(...)
>
> 2) vtkDemandDrivenPipeline::ComputePipelineMTime(...)
>
>
>
> This lag doesn’t appear when only filters at the beginning of the pipeline
> are shown, but as we go down in the pipeline and show the outputs of
> filters, the lag become unbearable:
>
>
>
> We think this lag is caused by the number of inputs our filters have
> combined with the length and complexity of the pipeline. In the following,
> I will explain our line of thought and propose a way to solve our pipeline
> scalability problem.
>
>
>
> *Demand Driven Pipelines for Representation:*
>
> Our understanding is that a representation pipeline usually involves that
> the multiplicity of outputs* is greater than* the multiplicity of inputs.
> The pipeline fans out.  This mean that the Update() process is rather fast
> and scalable.
>
>
>
>
> In the previous picture, if filter 3 needs an Update(), only filter 0,1
> and 3 will have a RequestData().
>
>
>
> *Demand Driven Pipeline for Modeling:*
>
> However, in our case, we use VTK as a visualisation library as well as a
> mesh processing tool. To process mesh, you often need multiple inputs
> (boolean operations, points projections on a mesh…): the multiplicity of
> inputs is *equivalent *to the multiplicity of outputs.
>
> If filter 8 needs an Update(), all filters Upstream will require to
> Update() except (6,3,1)
>
> With high input multiplicity, the Update() process, even with a
> distributed executive pipeline, will impact performance, creating a
> significant overhead.
>
>
>
> *Demand Driven Pipeline with Event Driven Subsets:*
>
> Here we suggest an improvement over the DemandDrivenPipeline by
> introducing Event Driven (push) subsets:
>
>
>
>
> If Filter 8 needs an Update(), the signal will be passed to the subset and
> then to Filter0. Filter 2,4,5 and 7 are force to calculate in a
> Event-driven fashion.
>
>
>
> The overhead of the representation pipeline Update() when you have a high
> input multiplicity could be reduced with the creation of subsets of a
> pipeline. The subsets should be handled as Event Triggered Pipelines (Push
> Pipelines).
>
> A push subset could be considered like a filter that contains a small part
> of the pipeline and where all outputs execute a RequestData() using a Push
> Pipeline.
>
>
>
> We are aware of the vtkThreadedStreamingPipeline that had PUSH/PULL
> capability, however it is now legacy. Also, we do not require the
> threaded/scheduler capability of this pipeline model.  An extended version
> of the vtkDemandDrivenPipeline with a Event Driven (pull) pipeline that
> would apply to a subset would be ideal.
>
>
>
> Please share your thoughts  on creating such a pipeline. Please keep in
> mind that this suggestion has the goal to keep the pipeline’s dynamism, we
> are aware that we could package the subset into a single filter.
>
> Thank you,
>
> Jonathan Borduas
>
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/
> opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> https://vtk.org/mailman/listinfo/vtk-developers
>
>
>


-- 
William J. Schroeder, PhD
Kitware, Inc. - Building the World's Technical Computing Software
28 Corporate Drive
Clifton Park, NY 12065
will.schroeder at kitware.com
http://www.kitware.com
(518) 881-4902
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://vtk.org/pipermail/vtk-developers/attachments/20180305/194a5ac2/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 5915 bytes
Desc: not available
URL: <https://vtk.org/pipermail/vtk-developers/attachments/20180305/194a5ac2/attachment-0003.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 14261 bytes
Desc: not available
URL: <https://vtk.org/pipermail/vtk-developers/attachments/20180305/194a5ac2/attachment-0004.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 10479 bytes
Desc: not available
URL: <https://vtk.org/pipermail/vtk-developers/attachments/20180305/194a5ac2/attachment-0005.jpg>