[vtk-developers] Massive lag in a Pileline. Too complex pipeline ?

Ken Martin ken.martin at kitware.com
Mon Mar 5 12:56:23 EST 2018


For VTK there is no need to disconnect. In fact mapper->StaticOn() is
faster than disconnecting once you have rendered once I believe.  Also if
you know your data is opaque setting ForceOpaqueOn() helps as well. With
just those two settings on VTK master I can get 10000 actors rendering at a
rate of 11 fps on a laptop.

For VTK I really do believe some of the core information object pipeline
stuff needs to be reworked. Simple calls like GetInputObject take a huge %
of time and create multiple objects on the heap per call etc.

ParaView is a different beast as it has GUI etc.


On Mon, Mar 5, 2018 at 12:04 PM, Will Schroeder <will.schroeder at kitware.com>
wrote:

> Just a quick response off of the top of my head:
>
> vtkMapper::Static (i.e, SetStaticOn()) prevents the mapper from
> propagating updates down the pipeline.
>
> I have seen pipelines hacked as follows: executing them once, and then
> perform a local shallow copy of the resulting data into data object(s)
> (e.g., polydata) which is disconnected from the pipeline hence the updates
> are fast.
>
> I suspect there are more elegant ways but maybe this will get the
> conversation going...
>
>
>
> On Mon, Mar 5, 2018 at 11:56 AM, Jonathan Borduas <
> jonathan.borduas at caboma.com> wrote:
>
>>
>>
>> Hi all,
>>
>>
>>
>> VTK pipelines are great, they allow to dynamically create a flow of
>> algorithms. However, we recently encountered a massive lag in our pipeline
>> related to the pipeline mechanisms (as opposed to the algorithms’
>> calculation time).  We are using a derivative of ParaView with homemade
>> filters.
>>
>>
>>
>> We profiled our application in such a way that the profiling session
>> started when all of the algorithms present in the pipeline had already been
>> executed once and were up to date. The profiler pointed out two methods
>> that were called a very significant amount of times while no updates
>> were required on any of the algorithms of the pipeline. Those methods were:
>>
>>
>>
>> 1) vtkInformation::GetAsObjectBase(...)
>>
>> 2) vtkDemandDrivenPipeline::ComputePipelineMTime(...)
>>
>>
>>
>> This lag doesn’t appear when only filters at the beginning of the
>> pipeline are shown, but as we go down in the pipeline and show the outputs
>> of filters, the lag become unbearable:
>>
>>
>>
>> We think this lag is caused by the number of inputs our filters have
>> combined with the length and complexity of the pipeline. In the following,
>> I will explain our line of thought and propose a way to solve our pipeline
>> scalability problem.
>>
>>
>>
>> *Demand Driven Pipelines for Representation:*
>>
>> Our understanding is that a representation pipeline usually involves that
>> the multiplicity of outputs* is greater than* the multiplicity of
>> inputs. The pipeline fans out.  This mean that the Update() process is
>> rather fast and scalable.
>>
>>
>>
>>
>> In the previous picture, if filter 3 needs an Update(), only filter 0,1
>> and 3 will have a RequestData().
>>
>>
>>
>> *Demand Driven Pipeline for Modeling:*
>>
>> However, in our case, we use VTK as a visualisation library as well as a
>> mesh processing tool. To process mesh, you often need multiple inputs
>> (boolean operations, points projections on a mesh…): the multiplicity of
>> inputs is *equivalent *to the multiplicity of outputs.
>>
>> If filter 8 needs an Update(), all filters Upstream will require to
>> Update() except (6,3,1)
>>
>> With high input multiplicity, the Update() process, even with a
>> distributed executive pipeline, will impact performance, creating a
>> significant overhead.
>>
>>
>>
>> *Demand Driven Pipeline with Event Driven Subsets:*
>>
>> Here we suggest an improvement over the DemandDrivenPipeline by
>> introducing Event Driven (push) subsets:
>>
>>
>>
>>
>> If Filter 8 needs an Update(), the signal will be passed to the subset
>> and then to Filter0. Filter 2,4,5 and 7 are force to calculate in a
>> Event-driven fashion.
>>
>>
>>
>> The overhead of the representation pipeline Update() when you have a high
>> input multiplicity could be reduced with the creation of subsets of a
>> pipeline. The subsets should be handled as Event Triggered Pipelines (Push
>> Pipelines).
>>
>> A push subset could be considered like a filter that contains a small
>> part of the pipeline and where all outputs execute a RequestData() using a
>> Push Pipeline.
>>
>>
>>
>> We are aware of the vtkThreadedStreamingPipeline that had PUSH/PULL
>> capability, however it is now legacy. Also, we do not require the
>> threaded/scheduler capability of this pipeline model.  An extended version
>> of the vtkDemandDrivenPipeline with a Event Driven (pull) pipeline that
>> would apply to a subset would be ideal.
>>
>>
>>
>> Please share your thoughts  on creating such a pipeline. Please keep in
>> mind that this suggestion has the goal to keep the pipeline’s dynamism, we
>> are aware that we could package the subset into a single filter.
>>
>> Thank you,
>>
>> Jonathan Borduas
>>
>>
>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>>
>> Follow this link to subscribe/unsubscribe:
>> https://vtk.org/mailman/listinfo/vtk-developers
>>
>>
>>
>
>
> --
> William J. Schroeder, PhD
> Kitware, Inc. - Building the World's Technical Computing Software
> 28 Corporate Drive
> Clifton Park, NY 12065
> will.schroeder at kitware.com
> http://www.kitware.com
> (518) 881-4902
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/
> opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> https://vtk.org/mailman/listinfo/vtk-developers
>
>
>


-- 
Ken Martin PhD
Distinguished Engineer
Kitware Inc.
28 Corporate Drive
Clifton Park NY 12065

This communication, including all attachments, contains confidential and
legally privileged information, and it is intended only for the use of the
addressee.  Access to this email by anyone else is unauthorized. If you are
not the intended recipient, any disclosure, copying, distribution or any
action taken in reliance on it is prohibited and may be unlawful. If you
received this communication in error please notify us immediately and
destroy the original message.  Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://vtk.org/pipermail/vtk-developers/attachments/20180305/5c3fd8a3/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 5915 bytes
Desc: not available
URL: <https://vtk.org/pipermail/vtk-developers/attachments/20180305/5c3fd8a3/attachment-0003.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 14261 bytes
Desc: not available
URL: <https://vtk.org/pipermail/vtk-developers/attachments/20180305/5c3fd8a3/attachment-0004.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 10479 bytes
Desc: not available
URL: <https://vtk.org/pipermail/vtk-developers/attachments/20180305/5c3fd8a3/attachment-0005.jpg>


More information about the vtk-developers mailing list