[vtk-developers] VTK and Qt5 / QML

Geoff Wright gpwright at gmail.com
Wed May 17 01:38:56 EDT 2017 

Yes each vtk item has its own render thread that is why I would argue this
approach is superior to the others since it scales with the number of cores.

I followed the approach described here:

http://doc.qt.io/qt-5/qtquick-scenegraph-textureinthread-example.html

So basically inside each vtk qml item there is a thread drawing a vtk
viewport into a texture, then the QSGRenderThread comes along and just does
a single texture blit for each item (very cheap). The Qt main thread takes
care of other event processing, models etc.

This is sufficient if each item contains an independent scene (vtk
pipeline).  If you need to share state between two or more viewports  then
you need to be careful to control the timing of when any data objects may
mutate since the vtk api is not inherently threadsafe.

Does that make sense?

G



On Tue, May 16, 2017, 6:13 PM <c.sell at byterefinery.de> wrote:

> Hello all,
>
> @Geoff: are you saying that each pair of QML item / vtkRenderWindow
> has its own render thread? Is that something QML does?
>
> My goal is to have QML/VTK items behave as normal QML items, i.e., I
> could have a dozen items on my screen each rendering something
> different through VTK without influencing each other (apart from
> system performance). I expect that the approach I have taken so far
> will already fulfill this requirement.
>
> @Sankesh: I object to the compile time flag. IMO both approaches
> should be usable at the same time, e.g. by extending different
> superclasses. As you say, the "underlay or overlay" aspect breaks QML
> concepts and makes any item built according to this approach an
> "alien" in the QML crowd that always requires special attention.
>
> I'll look into QVTKInteractorAdapter - that sounds like seomething
> useful. Should have come across it earlier.
>
> @both: is there a chance that usable code will come out of this
> exchange any time soon? I for my part would at least publish my
> prototypical implementations somewhere (GitHub?), unless they become
> obsolete by something "official"
>
> regards,
> Chris
>
> Zitat von Geoff Wright <gpwright at gmail.com>:
>
> > Hi Chris,
> >
> > Sorry for the confusion, yes I am doing what you describe.  The
> difference
> > may be that perhaps your app has only one viewport.  In my case there
> are a
> > number of different viewports (aka vtkRenderWindow)  each has its own qml
> > item and is each rendering on it's own thread.  Plus the qml composition
> > thread which is managed by Qt.
> >
> > I was just making the point that this architecture let's you draw
> different
> > viewports in parallel provided that each one has its own mappers and
> actors.
> >
> > G
> >
> >
> > On Tue, May 16, 2017, 4:13 PM <c.sell at byterefinery.de> wrote:
> >
> >> Hi Geoff,
> >>
> >> as I understand it, the blog post you linked to doesn't mention
> >> multiple threads beyond the QML UI thread and the scenegraph render
> >> thread.
> >>
> >> So far, I am using that approach with QQuickFrameBufferObject as
> >> starting point and vtkExternalOpenGLRenderWindow for the VTK side,
> >> which causes VTK to use QML's OpenGL context and thus render into the
> >> FBO maintained by QQuickFrameBufferObject. That seems to work
> >> flawlessly, as long as ALL rendering is kept on the scenegraph thread,
> >> which is not always obvious to do at first, because VTK's API's don't
> >> separate between reading event values and rendering them (e.g.,
> >> SetSlice(2) will internally set the current slice and then do a
> >> Render()).
> >>
> >> But why maintain more threads?
> >>
> >> Anyway, please go ahead and explain your approach as detailed as
> >> possible, I am all ears.
> >>
> >> regards,
> >> Chris
> >>
> >> Zitat von Geoff Wright <gpwright at gmail.com>:
> >>
> >> > Hi guys,
> >> >
> >> > I would argue that for qml integration the best approach is for each
> vtk
> >> > viewport to render into a texture on a dedicated thread, not the qml
> >> > rendering thread.  Qml then draws the resulting texture which is more
> in
> >> > line with the Qt design intention ( see
> >> >
> >>
> http://blog.qt.io/blog/2015/05/11/integrating-custom-opengl-rendering-with-qt-quick-via-qquickframebufferobject/
> >> >  )
> >> >
> >> > I have been using this approach for a couple of years in a high
> >> performance
> >> > real time application.  The code is unfortunately proprietary but very
> >> > happy to explain more about the approach.  It would be great to add
> >> support
> >> > for this to VTK.
> >> >
> >> > For the interactor part I do what Sankesh said, defer the events and
> >> apply
> >> > them on the render thread of the particular vtk item.
> >> >
> >> > Regards,
> >> >
> >> > G
> >> >
> >> >
> >> >
> >> > On Tue, May 16, 2017, 3:06 PM Sankhesh Jhaveri <
> >> sankhesh.jhaveri at kitware.com>
> >> > wrote:
> >> >
> >> >> Hi Christian,
> >> >>
> >> >> Doing all VTK rendering on the QML render thread is the right thing
> to
> >> do.
> >> >> As far as interaction is concerned, make sure that Qt events are
> queued
> >> on
> >> >> the main thread and processed when execution reaches the render
> thread.
> >> >>
> >> >> I have done some work on VTK and QtQuick integration that I’m
> planning
> >> to
> >> >> add to a VTK remote module. That way, it will be available as part of
> >> VTK.
> >> >>
> >> >> Thanks,
> >> >> Sankhesh
> >> >> ​
> >> >>
> >> >> On Tue, May 16, 2017 at 4:57 AM <c.sell at byterefinery.de> wrote:
> >> >>
> >> >>> just for the record (nobody care about this subject?):
> >> >>>
> >> >>> I found that contrary to what I said I had NOT adhered to the
> >> >>> principle established with protoype 1 while implementing protoype 2.
> >> >>> Rather, I had directly called into the VTK interactor from the QML
> UI
> >> >>> thread. And because the VTK interactors immediately go into
> rendering,
> >> >>> all hell broke loose.
> >> >>>
> >> >>> After cleaning up example 2 according to the rule laid out earlier
> >> >>> ("store event data in RenderDelegate, call update() on the QML item,
> >> >>> pass the event to VTK inside RenderDelgate.Render()", all works
> well.
> >> >>>
> >> >>> What I would like to do is validate the solution against findings
> made
> >> >>> elsewhere, and then establish the "canonical" way of integrating VTK
> >> >>> with QML. As I said, there are some architectural quirks with the
> >> >>> solution which I would also like to address.
> >> >>>
> >> >>> regards,
> >> >>> Christian
> >> >>>
> >> >>>
> >> >>> Zitat von c.sell at byterefinery.de:
> >> >>>
> >> >>> > Hello all,
> >> >>> >
> >> >>> > I am looking into the possibility of replacing the 3D rendering
> >> >>> > engine in my Qt5 / QML based mobile tablet-oriented application
> with
> >> >>> > VTK. What I need is a first class QtQuick integration (not a hack
> or
> >> >>> > workaround) that is 100% stable in every context (not an unusual
> >> >>> > requirement, I admit). To my amazement, nothing of that kind seems
> >> >>> > to exist (correct me if I'm wrong).
> >> >>> >
> >> >>> > I went on to investigate what had been done so far and to
> implement
> >> >>> > my first prototypes, using the QQuickFrameBufferObject approach.
> >> >>> > From the very start this felt like an uphill battle, because VTK
> >> >>> > seems to come from a Windowing background and is quite tightly
> >> >>> > integrated with concepts that are not valid in a QML context.
> >> >>> >
> >> >>> > I'll describe my findings together with the 2 prototypical QML
> items
> >> >>> > I implemented:
> >> >>> >
> >> >>> > 1st was an adaptation of the DICOM example which now runs as a QML
> >> >>> > item. All user interaction is handled by QML and forwarded to VTK
> >> >>> > (which is one thing that doesn't come naturally with VTK), and
> after
> >> >>> > some non-elegant tweaking I was able to have the UI move from
> slice
> >> >>> > to slice and re-render upon mouse wheel events as expected. The
> >> >>> > problem here was, that QML insists on keeping mouse event handling
> >> >>> > and OpenGL rendering on separate threads, with one "rendering
> >> >>> > thread" dedicated to OpenGL. However, the pre-existing VTK
> >> >>> > Interactors directly call Render() after reconfiguring the UI from
> >> >>> > the mouse event data, which is an absolute QML No-Go. I had to
> >> >>> > introduce a RenderDelegate that works somewhat like this:
> >> >>> >
> >> >>> > QML mouse event:
> >> >>> >     tell RenderDelegate about the event
> >> >>> >     call update() on the QML item, which triggers rer-endering on
> >> >>> > the dedicated thread
> >> >>> > on the QML rendering thread:
> >> >>> >     call Render() on the RenderWindow
> >> >>> >     inside the RenderDelegate, look at whether a mouse event is
> >> >>> > pending, and call the corresponding VTK mouse handler
> >> >>> >     call the renderer's default Render() function while setting
> the
> >> >>> > delegate's "Used" flag to false
> >> >>> >
> >> >>> > 2nd was a QML item that displays a model loaded from a 3ds file.
> My
> >> >>> > goal here was to move the model around using mouse drag events. I
> >> >>> > took the lessons learned from the first example, threw in a
> >> >>> > vtkInteractorStyleTrackballCamera and hoped for the best. First
> >> >>> > thing I found was that I needed a specialized
> >> >>> > RenderWindowInteractor, which I provided. When I realized that the
> >> >>> > most important requirement for that class was to provide access to
> >> >>> > timer functionality, I already got wary, as throwing around events
> >> >>> > and doing stuff offline while on the QML rendering thread is
> never a
> >> >>> > good idea. My fears came true when I finished wiring everything
> >> >>> > together: I was able to move the model using the mouse, but after
> a
> >> >>> > few moments things got whacky with error output written to the
> >> >>> > console and finally a segemtation fault from deep inside VTK.
> >> >>> >
> >> >>> > I still need to investigate the second example, but would like to
> >> >>> > synchronize with the community at this point in order to avoid
> >> >>> > errors/misconceptions on my side, to seek help if possible and to
> >> >>> > offer my contribution to the VTK code base once this becomes
> >> >>> > functional. My first impression is that there are some issues that
> >> >>> > lie on an architectural level and cannot be (elegantly) dealt with
> >> >>> > on the QML side alone. Any comments?
> >> >>> >
> >> >>> > thanks,
> >> >>> > Christian Sell
> >> >>> >
> >> >>> > _______________________________________________
> >> >>> > Powered by www.kitware.com
> >> >>> >
> >> >>> > Visit other Kitware open-source projects at
> >> >>> > http://www.kitware.com/opensource/opensource.html
> >> >>> >
> >> >>> > Search the list archives at:
> >> >>> http://markmail.org/search/?q=vtk-developers
> >> >>> >
> >> >>> > Follow this link to subscribe/unsubscribe:
> >> >>> > http://public.kitware.com/mailman/listinfo/vtk-developers
> >> >>>
> >> >>>
> >> >>>
> >> >>> _______________________________________________
> >> >>> Powered by www.kitware.com
> >> >>>
> >> >>> Visit other Kitware open-source projects at
> >> >>> http://www.kitware.com/opensource/opensource.html
> >> >>>
> >> >>> Search the list archives at:
> >> http://markmail.org/search/?q=vtk-developers
> >> >>>
> >> >>> Follow this link to subscribe/unsubscribe:
> >> >>> http://public.kitware.com/mailman/listinfo/vtk-developers
> >> >>>
> >> >>> --
> >> >> Sankhesh Jhaveri *Sr. Research & Development Engineer* | Kitware
> >> >> <http://www.kitware.com/> | (518) 881-4417
> >> >> ​
> >> >> _______________________________________________
> >> >> Powered by www.kitware.com
> >> >>
> >> >> Visit other Kitware open-source projects at
> >> >> http://www.kitware.com/opensource/opensource.html
> >> >>
> >> >> Search the list archives at:
> >> http://markmail.org/search/?q=vtk-developers
> >> >>
> >> >> Follow this link to subscribe/unsubscribe:
> >> >> http://public.kitware.com/mailman/listinfo/vtk-developers
> >> >>
> >> >>
> >>
> >>
> >>
> >>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20170517/4b63a5e5/attachment-0001.html>

More information about the vtk-developers mailing list