[vtk-developers] VTK and Qt5 / QML

Tue May 16 11:26:35 EDT 2017

Hi Geoff, Chris,

IMHO, there are two approaches to VTK+QtQuick:

   -

   Rendering the VTK scene to a QQuickFrameBufferObject.
   - Provides better integration with the QML render stack.
      - Since it is an indirect rendering approach, performance may suffer.
      - May require specializations for some VTK interactor styles and VTK
      widget interaction.
   -

   Rendering directly to the QML created OpenGL context using
   beforeRendering() and afterRendering() calls
   - Better performance as VTK is directly rendering to the FrameBuffer
      - Interactor styles and widget interaction would work out of the box
      - Does not fit within the QML stack as the VTK scene would either be
      an underlay or an overlay

Note that in both these approaches, VTK rendering would need to be
constrained to the QSGRenderThread and Qt events would have to be queued
before being processed on the render thread.

Considering that both these approaches have their own merits, demerits and
constraints, maybe adding a compile time flag that lets you switch between
direct and indirect rendering would help. As such, there is no “correct”
solution.

Regarding queueing the events, I’d just sub-class QVTKInteractorAdapter
<http://www.vtk.org/doc/nightly/html/classQVTKInteractorAdapter.html> and
override ProcessEvent so that the event gets queued. One execution is in
the render thread, the queued events should be processed by the interactor.

One more thing as Geoff pointed out, since QML creates an OpenGL context,
each VTK item in that context would be an independent vtkViewport. Without
this, having multiple QML and VTK items in the same window would not be
possible.

Hope this helps.

Sankhesh

On Tue, May 16, 2017 at 10:14 AM <c.sell at byterefinery.de> wrote:

> Hi Geoff,
>
> as I understand it, the blog post you linked to doesn't mention
> multiple threads beyond the QML UI thread and the scenegraph render
> thread.
>
> So far, I am using that approach with QQuickFrameBufferObject as
> starting point and vtkExternalOpenGLRenderWindow for the VTK side,
> which causes VTK to use QML's OpenGL context and thus render into the
> FBO maintained by QQuickFrameBufferObject. That seems to work
> flawlessly, as long as ALL rendering is kept on the scenegraph thread,
> which is not always obvious to do at first, because VTK's API's don't
> separate between reading event values and rendering them (e.g.,
> SetSlice(2) will internally set the current slice and then do a
> Render()).
>
> But why maintain more threads?
>
> Anyway, please go ahead and explain your approach as detailed as
> possible, I am all ears.
>
> regards,
> Chris
>
> Zitat von Geoff Wright <gpwright at gmail.com>:
>
> > Hi guys,
> >
> > I would argue that for qml integration the best approach is for each vtk
> > viewport to render into a texture on a dedicated thread, not the qml
> > rendering thread.  Qml then draws the resulting texture which is more in
> > line with the Qt design intention ( see
> >
> http://blog.qt.io/blog/2015/05/11/integrating-custom-opengl-rendering-with-qt-quick-via-qquickframebufferobject/
> >  )
> >
> > I have been using this approach for a couple of years in a high
> performance
> > real time application.  The code is unfortunately proprietary but very
> > happy to explain more about the approach.  It would be great to add
> support
> > for this to VTK.
> >
> > For the interactor part I do what Sankesh said, defer the events and
> apply
> > them on the render thread of the particular vtk item.
> >
> > Regards,
> >
> > G
> >
> >
> >
> > On Tue, May 16, 2017, 3:06 PM Sankhesh Jhaveri <
> sankhesh.jhaveri at kitware.com>
> > wrote:
> >
> >> Hi Christian,
> >>
> >> Doing all VTK rendering on the QML render thread is the right thing to
> do.
> >> As far as interaction is concerned, make sure that Qt events are queued
> on
> >> the main thread and processed when execution reaches the render thread.
> >>
> >> I have done some work on VTK and QtQuick integration that I’m planning
> to
> >> add to a VTK remote module. That way, it will be available as part of
> VTK.
> >>
> >> Thanks,
> >> Sankhesh
> >> 
> >>
> >> On Tue, May 16, 2017 at 4:57 AM <c.sell at byterefinery.de> wrote:
> >>
> >>> just for the record (nobody care about this subject?):
> >>>
> >>> I found that contrary to what I said I had NOT adhered to the
> >>> principle established with protoype 1 while implementing protoype 2.
> >>> Rather, I had directly called into the VTK interactor from the QML UI
> >>> thread. And because the VTK interactors immediately go into rendering,
> >>> all hell broke loose.
> >>>
> >>> After cleaning up example 2 according to the rule laid out earlier
> >>> ("store event data in RenderDelegate, call update() on the QML item,
> >>> pass the event to VTK inside RenderDelgate.Render()", all works well.
> >>>
> >>> What I would like to do is validate the solution against findings made
> >>> elsewhere, and then establish the "canonical" way of integrating VTK
> >>> with QML. As I said, there are some architectural quirks with the
> >>> solution which I would also like to address.
> >>>
> >>> regards,
> >>> Christian
> >>>
> >>>
> >>> Zitat von c.sell at byterefinery.de:
> >>>
> >>> > Hello all,
> >>> >
> >>> > I am looking into the possibility of replacing the 3D rendering
> >>> > engine in my Qt5 / QML based mobile tablet-oriented application with
> >>> > VTK. What I need is a first class QtQuick integration (not a hack or
> >>> > workaround) that is 100% stable in every context (not an unusual
> >>> > requirement, I admit). To my amazement, nothing of that kind seems
> >>> > to exist (correct me if I'm wrong).
> >>> >
> >>> > I went on to investigate what had been done so far and to implement
> >>> > my first prototypes, using the QQuickFrameBufferObject approach.
> >>> > From the very start this felt like an uphill battle, because VTK
> >>> > seems to come from a Windowing background and is quite tightly
> >>> > integrated with concepts that are not valid in a QML context.
> >>> >
> >>> > I'll describe my findings together with the 2 prototypical QML items
> >>> > I implemented:
> >>> >
> >>> > 1st was an adaptation of the DICOM example which now runs as a QML
> >>> > item. All user interaction is handled by QML and forwarded to VTK
> >>> > (which is one thing that doesn't come naturally with VTK), and after
> >>> > some non-elegant tweaking I was able to have the UI move from slice
> >>> > to slice and re-render upon mouse wheel events as expected. The
> >>> > problem here was, that QML insists on keeping mouse event handling
> >>> > and OpenGL rendering on separate threads, with one "rendering
> >>> > thread" dedicated to OpenGL. However, the pre-existing VTK
> >>> > Interactors directly call Render() after reconfiguring the UI from
> >>> > the mouse event data, which is an absolute QML No-Go. I had to
> >>> > introduce a RenderDelegate that works somewhat like this:
> >>> >
> >>> > QML mouse event:
> >>> >     tell RenderDelegate about the event
> >>> >     call update() on the QML item, which triggers rer-endering on
> >>> > the dedicated thread
> >>> > on the QML rendering thread:
> >>> >     call Render() on the RenderWindow
> >>> >     inside the RenderDelegate, look at whether a mouse event is
> >>> > pending, and call the corresponding VTK mouse handler
> >>> >     call the renderer's default Render() function while setting the
> >>> > delegate's "Used" flag to false
> >>> >
> >>> > 2nd was a QML item that displays a model loaded from a 3ds file. My
> >>> > goal here was to move the model around using mouse drag events. I
> >>> > took the lessons learned from the first example, threw in a
> >>> > vtkInteractorStyleTrackballCamera and hoped for the best. First
> >>> > thing I found was that I needed a specialized
> >>> > RenderWindowInteractor, which I provided. When I realized that the
> >>> > most important requirement for that class was to provide access to
> >>> > timer functionality, I already got wary, as throwing around events
> >>> > and doing stuff offline while on the QML rendering thread is never a
> >>> > good idea. My fears came true when I finished wiring everything
> >>> > together: I was able to move the model using the mouse, but after a
> >>> > few moments things got whacky with error output written to the
> >>> > console and finally a segemtation fault from deep inside VTK.
> >>> >
> >>> > I still need to investigate the second example, but would like to
> >>> > synchronize with the community at this point in order to avoid
> >>> > errors/misconceptions on my side, to seek help if possible and to
> >>> > offer my contribution to the VTK code base once this becomes
> >>> > functional. My first impression is that there are some issues that
> >>> > lie on an architectural level and cannot be (elegantly) dealt with
> >>> > on the QML side alone. Any comments?
> >>> >
> >>> > thanks,
> >>> > Christian Sell
> >>> >
> >>> > _______________________________________________
> >>> > Powered by www.kitware.com
> >>> >
> >>> > Visit other Kitware open-source projects at
> >>> > http://www.kitware.com/opensource/opensource.html
> >>> >
> >>> > Search the list archives at:
> >>> http://markmail.org/search/?q=vtk-developers
> >>> >
> >>> > Follow this link to subscribe/unsubscribe:
> >>> > http://public.kitware.com/mailman/listinfo/vtk-developers
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Powered by www.kitware.com
> >>>
> >>> Visit other Kitware open-source projects at
> >>> http://www.kitware.com/opensource/opensource.html
> >>>
> >>> Search the list archives at:
> http://markmail.org/search/?q=vtk-developers
> >>>
> >>> Follow this link to subscribe/unsubscribe:
> >>> http://public.kitware.com/mailman/listinfo/vtk-developers
> >>>
> >>> --
> >> Sankhesh Jhaveri *Sr. Research & Development Engineer* | Kitware
> >> <http://www.kitware.com/> | (518) 881-4417
> >> 
> >> _______________________________________________
> >> Powered by www.kitware.com
> >>
> >> Visit other Kitware open-source projects at
> >> http://www.kitware.com/opensource/opensource.html
> >>
> >> Search the list archives at:
> http://markmail.org/search/?q=vtk-developers
> >>
> >> Follow this link to subscribe/unsubscribe:
> >> http://public.kitware.com/mailman/listinfo/vtk-developers
> >>
> >>
>
>
>
> --
Sankhesh Jhaveri *Sr. Research & Development Engineer* | Kitware
<http://www.kitware.com/> | (518) 881-4417

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20170516/963b9145/attachment-0001.html>