Hi Chris,<div><br></div><div>For the interactor there is a way to basically queue the event information you need e.g. from qmls onPressed and then pass it to an interactor that belongs to the particular viewport that was pressed/clicked so that it will be handled once that thread is being rendered.</div><div><br></div><div>However, I found that I quickly wanted a more advanced interaction. For example if your vtk qml item is inside a qml Flickable you can easily do very nice interactions for touch such as rotational momentum on flick that continues after release.</div><div><br></div><div>I am not sure of the latest touch interactor in vtk but when I was working on this a couple of years ago it was easier to handle all of the interactor part in QML (e.g. utilizing Flickable) and then pass info to the rendering thread that would directly mutate the camera, effectively making the vtkCamera a slave to some qml properties.</div><div><br></div><div>However if vtk has caught up with momentum based touch interactors it may be worthwhile to use these. I am not caught up with what's been added to VTK in recent months. Another advantage of keeping the camera master state in QML is that you can use Qt animation framework easily (e.g. they recently added an orientation animation that can be useful for controlling a camera).</div><div><br></div><div>I think it depends how complex are the user interactions that you are planning for.</div><div><br></div><div>G</div><div><br></div><div><br></div><div><br></div><div><br><br><div class="gmail_quote"><div dir="ltr">On Wed, May 17, 2017, 10:15 AM <<a href="mailto:c.sell@byterefinery.de">c.sell@byterefinery.de</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I have to correct myself with regard to the "direct rendering" option.<br>
There is, of course, an example that demonstrates this<br>
(<a href="http://doc.qt.io/qt-5/qtquick-scenegraph-openglunderqml-example.html" rel="noreferrer" target="_blank">http://doc.qt.io/qt-5/qtquick-scenegraph-openglunderqml-example.html</a>). I<br>
doubt whether this could be implemented in conjunction with<br>
thread-per-item, so it only adds one architectural option. I still<br>
think this is the least versatile and generic one, though.<br>
<br>
chris<br>
<br>
Zitat von <a href="mailto:c.sell@byterefinery.de" target="_blank">c.sell@byterefinery.de</a>:<br>
<br>
> ok, things are clearing up a little more now. You ARE creating a<br>
> separate thread for each QML item, which gives us these threads:<br>
><br>
> - QML GUI thread (where the events come from)<br>
> - QML rendering thread (aka scenegraph rendering thread aka<br>
> QSGRenderThread), of which theres is only one<br>
> - a dedicated thread for each item (RenderThread class in the example)<br>
><br>
> the example also uses a QSGSimpleTextureNode subclass that does the<br>
> blit'ing together. All of that decoupled via QueuedConnections and<br>
> events signalled here and there. Not an easy beast, that.<br>
><br>
> What remains unclear is how you deal with user input. For example,<br>
> if the user were to drag something around in the scene (and your app<br>
> were to support that), where do the MouseMove events go? This is<br>
> really the main crux in this discussion IMO.<br>
><br>
> To sum up the architectural choices:<br>
><br>
> a) render into FBO<br>
> a1)- do all rendering on the scenegraph thread (exemplified in<br>
> <a href="http://doc.qt.io/qt-5/qtquick-scenegraph-textureinsgnode-example.html" rel="noreferrer" target="_blank">http://doc.qt.io/qt-5/qtquick-scenegraph-textureinsgnode-example.html</a>)<br>
><br>
> a2) render on a dedicated thread for each item (exemplified in<br>
> <a href="http://doc.qt.io/qt-5/qtquick-scenegraph-textureinthread-example.html" rel="noreferrer" target="_blank">http://doc.qt.io/qt-5/qtquick-scenegraph-textureinthread-example.html</a>)<br>
><br>
> b) render into the QML context directly (not sure about this. Is<br>
> there a Qt example demonstrating the practicability?)<br>
> b?) undetermined<br>
><br>
><br>
> The user intaction part is undetermined for me at this point. I am<br>
> trying to get something going using QVTKInteractorAdapter, but<br>
> havent had luck so far.<br>
><br>
> In terms of ready usable code that demonstrates QML/VTK integration<br>
> (including interaction and all) I see nothing yet on the horizon.<br>
><br>
> chris<br>
><br>
><br>
><br>
> Zitat von Geoff Wright <<a href="mailto:gpwright@gmail.com" target="_blank">gpwright@gmail.com</a>>:<br>
><br>
>> Yes each vtk item has its own render thread that is why I would argue this<br>
>> approach is superior to the others since it scales with the number of cores.<br>
>><br>
>> I followed the approach described here:<br>
>><br>
>> <a href="http://doc.qt.io/qt-5/qtquick-scenegraph-textureinthread-example.html" rel="noreferrer" target="_blank">http://doc.qt.io/qt-5/qtquick-scenegraph-textureinthread-example.html</a><br>
>><br>
>> So basically inside each vtk qml item there is a thread drawing a vtk<br>
>> viewport into a texture, then the QSGRenderThread comes along and just does<br>
>> a single texture blit for each item (very cheap). The Qt main thread takes<br>
>> care of other event processing, models etc.<br>
>><br>
>> This is sufficient if each item contains an independent scene (vtk<br>
>> pipeline). If you need to share state between two or more viewports then<br>
>> you need to be careful to control the timing of when any data objects may<br>
>> mutate since the vtk api is not inherently threadsafe.<br>
>><br>
>> Does that make sense?<br>
>><br>
>> G<br>
>><br>
>><br>
>><br>
>> On Tue, May 16, 2017, 6:13 PM <<a href="mailto:c.sell@byterefinery.de" target="_blank">c.sell@byterefinery.de</a>> wrote:<br>
>><br>
>>> Hello all,<br>
>>><br>
>>> @Geoff: are you saying that each pair of QML item / vtkRenderWindow<br>
>>> has its own render thread? Is that something QML does?<br>
>>><br>
>>> My goal is to have QML/VTK items behave as normal QML items, i.e., I<br>
>>> could have a dozen items on my screen each rendering something<br>
>>> different through VTK without influencing each other (apart from<br>
>>> system performance). I expect that the approach I have taken so far<br>
>>> will already fulfill this requirement.<br>
>>><br>
>>> @Sankesh: I object to the compile time flag. IMO both approaches<br>
>>> should be usable at the same time, e.g. by extending different<br>
>>> superclasses. As you say, the "underlay or overlay" aspect breaks QML<br>
>>> concepts and makes any item built according to this approach an<br>
>>> "alien" in the QML crowd that always requires special attention.<br>
>>><br>
>>> I'll look into QVTKInteractorAdapter - that sounds like seomething<br>
>>> useful. Should have come across it earlier.<br>
>>><br>
>>> @both: is there a chance that usable code will come out of this<br>
>>> exchange any time soon? I for my part would at least publish my<br>
>>> prototypical implementations somewhere (GitHub?), unless they become<br>
>>> obsolete by something "official"<br>
>>><br>
>>> regards,<br>
>>> Chris<br>
>>><br>
>>> Zitat von Geoff Wright <<a href="mailto:gpwright@gmail.com" target="_blank">gpwright@gmail.com</a>>:<br>
>>><br>
>>>> Hi Chris,<br>
>>>><br>
>>>> Sorry for the confusion, yes I am doing what you describe. The<br>
>>> difference<br>
>>>> may be that perhaps your app has only one viewport. In my case there<br>
>>> are a<br>
>>>> number of different viewports (aka vtkRenderWindow) each has its own qml<br>
>>>> item and is each rendering on it's own thread. Plus the qml composition<br>
>>>> thread which is managed by Qt.<br>
>>>><br>
>>>> I was just making the point that this architecture let's you draw<br>
>>> different<br>
>>>> viewports in parallel provided that each one has its own mappers and<br>
>>> actors.<br>
>>>><br>
>>>> G<br>
>>>><br>
>>>><br>
>>>> On Tue, May 16, 2017, 4:13 PM <<a href="mailto:c.sell@byterefinery.de" target="_blank">c.sell@byterefinery.de</a>> wrote:<br>
>>>><br>
>>>>> Hi Geoff,<br>
>>>>><br>
>>>>> as I understand it, the blog post you linked to doesn't mention<br>
>>>>> multiple threads beyond the QML UI thread and the scenegraph render<br>
>>>>> thread.<br>
>>>>><br>
>>>>> So far, I am using that approach with QQuickFrameBufferObject as<br>
>>>>> starting point and vtkExternalOpenGLRenderWindow for the VTK side,<br>
>>>>> which causes VTK to use QML's OpenGL context and thus render into the<br>
>>>>> FBO maintained by QQuickFrameBufferObject. That seems to work<br>
>>>>> flawlessly, as long as ALL rendering is kept on the scenegraph thread,<br>
>>>>> which is not always obvious to do at first, because VTK's API's don't<br>
>>>>> separate between reading event values and rendering them (e.g.,<br>
>>>>> SetSlice(2) will internally set the current slice and then do a<br>
>>>>> Render()).<br>
>>>>><br>
>>>>> But why maintain more threads?<br>
>>>>><br>
>>>>> Anyway, please go ahead and explain your approach as detailed as<br>
>>>>> possible, I am all ears.<br>
>>>>><br>
>>>>> regards,<br>
>>>>> Chris<br>
>>>>><br>
>>>>> Zitat von Geoff Wright <<a href="mailto:gpwright@gmail.com" target="_blank">gpwright@gmail.com</a>>:<br>
>>>>><br>
>>>>> > Hi guys,<br>
>>>>> ><br>
>>>>> > I would argue that for qml integration the best approach is for each<br>
>>> vtk<br>
>>>>> > viewport to render into a texture on a dedicated thread, not the qml<br>
>>>>> > rendering thread. Qml then draws the resulting texture which is more<br>
>>> in<br>
>>>>> > line with the Qt design intention ( see<br>
>>>>> ><br>
>>>>><br>
>>> <a href="http://blog.qt.io/blog/2015/05/11/integrating-custom-opengl-rendering-with-qt-quick-via-qquickframebufferobject/" rel="noreferrer" target="_blank">http://blog.qt.io/blog/2015/05/11/integrating-custom-opengl-rendering-with-qt-quick-via-qquickframebufferobject/</a><br>
>>>>> > )<br>
>>>>> ><br>
>>>>> > I have been using this approach for a couple of years in a high<br>
>>>>> performance<br>
>>>>> > real time application. The code is unfortunately proprietary but very<br>
>>>>> > happy to explain more about the approach. It would be great to add<br>
>>>>> support<br>
>>>>> > for this to VTK.<br>
>>>>> ><br>
>>>>> > For the interactor part I do what Sankesh said, defer the events and<br>
>>>>> apply<br>
>>>>> > them on the render thread of the particular vtk item.<br>
>>>>> ><br>
>>>>> > Regards,<br>
>>>>> ><br>
>>>>> > G<br>
>>>>> ><br>
>>>>> ><br>
>>>>> ><br>
>>>>> > On Tue, May 16, 2017, 3:06 PM Sankhesh Jhaveri <<br>
>>>>> <a href="mailto:sankhesh.jhaveri@kitware.com" target="_blank">sankhesh.jhaveri@kitware.com</a>><br>
>>>>> > wrote:<br>
>>>>> ><br>
>>>>> >> Hi Christian,<br>
>>>>> >><br>
>>>>> >> Doing all VTK rendering on the QML render thread is the right thing<br>
>>> to<br>
>>>>> do.<br>
>>>>> >> As far as interaction is concerned, make sure that Qt events are<br>
>>> queued<br>
>>>>> on<br>
>>>>> >> the main thread and processed when execution reaches the render<br>
>>> thread.<br>
>>>>> >><br>
>>>>> >> I have done some work on VTK and QtQuick integration that I’m<br>
>>> planning<br>
>>>>> to<br>
>>>>> >> add to a VTK remote module. That way, it will be available as part of<br>
>>>>> VTK.<br>
>>>>> >><br>
>>>>> >> Thanks,<br>
>>>>> >> Sankhesh<br>
>>>>> >> <br>
>>>>> >><br>
>>>>> >> On Tue, May 16, 2017 at 4:57 AM <<a href="mailto:c.sell@byterefinery.de" target="_blank">c.sell@byterefinery.de</a>> wrote:<br>
>>>>> >><br>
>>>>> >>> just for the record (nobody care about this subject?):<br>
>>>>> >>><br>
>>>>> >>> I found that contrary to what I said I had NOT adhered to the<br>
>>>>> >>> principle established with protoype 1 while implementing protoype 2.<br>
>>>>> >>> Rather, I had directly called into the VTK interactor from the QML<br>
>>> UI<br>
>>>>> >>> thread. And because the VTK interactors immediately go into<br>
>>> rendering,<br>
>>>>> >>> all hell broke loose.<br>
>>>>> >>><br>
>>>>> >>> After cleaning up example 2 according to the rule laid out earlier<br>
>>>>> >>> ("store event data in RenderDelegate, call update() on the QML item,<br>
>>>>> >>> pass the event to VTK inside RenderDelgate.Render()", all works<br>
>>> well.<br>
>>>>> >>><br>
>>>>> >>> What I would like to do is validate the solution against findings<br>
>>> made<br>
>>>>> >>> elsewhere, and then establish the "canonical" way of integrating VTK<br>
>>>>> >>> with QML. As I said, there are some architectural quirks with the<br>
>>>>> >>> solution which I would also like to address.<br>
>>>>> >>><br>
>>>>> >>> regards,<br>
>>>>> >>> Christian<br>
>>>>> >>><br>
>>>>> >>><br>
>>>>> >>> Zitat von <a href="mailto:c.sell@byterefinery.de" target="_blank">c.sell@byterefinery.de</a>:<br>
>>>>> >>><br>
>>>>> >>> > Hello all,<br>
>>>>> >>> ><br>
>>>>> >>> > I am looking into the possibility of replacing the 3D rendering<br>
>>>>> >>> > engine in my Qt5 / QML based mobile tablet-oriented application<br>
>>> with<br>
>>>>> >>> > VTK. What I need is a first class QtQuick integration (not a hack<br>
>>> or<br>
>>>>> >>> > workaround) that is 100% stable in every context (not an unusual<br>
>>>>> >>> > requirement, I admit). To my amazement, nothing of that kind seems<br>
>>>>> >>> > to exist (correct me if I'm wrong).<br>
>>>>> >>> ><br>
>>>>> >>> > I went on to investigate what had been done so far and to<br>
>>> implement<br>
>>>>> >>> > my first prototypes, using the QQuickFrameBufferObject approach.<br>
>>>>> >>> > From the very start this felt like an uphill battle, because VTK<br>
>>>>> >>> > seems to come from a Windowing background and is quite tightly<br>
>>>>> >>> > integrated with concepts that are not valid in a QML context.<br>
>>>>> >>> ><br>
>>>>> >>> > I'll describe my findings together with the 2 prototypical QML<br>
>>> items<br>
>>>>> >>> > I implemented:<br>
>>>>> >>> ><br>
>>>>> >>> > 1st was an adaptation of the DICOM example which now runs as a QML<br>
>>>>> >>> > item. All user interaction is handled by QML and forwarded to VTK<br>
>>>>> >>> > (which is one thing that doesn't come naturally with VTK), and<br>
>>> after<br>
>>>>> >>> > some non-elegant tweaking I was able to have the UI move from<br>
>>> slice<br>
>>>>> >>> > to slice and re-render upon mouse wheel events as expected. The<br>
>>>>> >>> > problem here was, that QML insists on keeping mouse event handling<br>
>>>>> >>> > and OpenGL rendering on separate threads, with one "rendering<br>
>>>>> >>> > thread" dedicated to OpenGL. However, the pre-existing VTK<br>
>>>>> >>> > Interactors directly call Render() after reconfiguring the UI from<br>
>>>>> >>> > the mouse event data, which is an absolute QML No-Go. I had to<br>
>>>>> >>> > introduce a RenderDelegate that works somewhat like this:<br>
>>>>> >>> ><br>
>>>>> >>> > QML mouse event:<br>
>>>>> >>> > tell RenderDelegate about the event<br>
>>>>> >>> > call update() on the QML item, which triggers rer-endering on<br>
>>>>> >>> > the dedicated thread<br>
>>>>> >>> > on the QML rendering thread:<br>
>>>>> >>> > call Render() on the RenderWindow<br>
>>>>> >>> > inside the RenderDelegate, look at whether a mouse event is<br>
>>>>> >>> > pending, and call the corresponding VTK mouse handler<br>
>>>>> >>> > call the renderer's default Render() function while setting<br>
>>> the<br>
>>>>> >>> > delegate's "Used" flag to false<br>
>>>>> >>> ><br>
>>>>> >>> > 2nd was a QML item that displays a model loaded from a 3ds file.<br>
>>> My<br>
>>>>> >>> > goal here was to move the model around using mouse drag events. I<br>
>>>>> >>> > took the lessons learned from the first example, threw in a<br>
>>>>> >>> > vtkInteractorStyleTrackballCamera and hoped for the best. First<br>
>>>>> >>> > thing I found was that I needed a specialized<br>
>>>>> >>> > RenderWindowInteractor, which I provided. When I realized that the<br>
>>>>> >>> > most important requirement for that class was to provide access to<br>
>>>>> >>> > timer functionality, I already got wary, as throwing around events<br>
>>>>> >>> > and doing stuff offline while on the QML rendering thread is<br>
>>> never a<br>
>>>>> >>> > good idea. My fears came true when I finished wiring everything<br>
>>>>> >>> > together: I was able to move the model using the mouse, but after<br>
>>> a<br>
>>>>> >>> > few moments things got whacky with error output written to the<br>
>>>>> >>> > console and finally a segemtation fault from deep inside VTK.<br>
>>>>> >>> ><br>
>>>>> >>> > I still need to investigate the second example, but would like to<br>
>>>>> >>> > synchronize with the community at this point in order to avoid<br>
>>>>> >>> > errors/misconceptions on my side, to seek help if possible and to<br>
>>>>> >>> > offer my contribution to the VTK code base once this becomes<br>
>>>>> >>> > functional. My first impression is that there are some issues that<br>
>>>>> >>> > lie on an architectural level and cannot be (elegantly) dealt with<br>
>>>>> >>> > on the QML side alone. Any comments?<br>
>>>>> >>> ><br>
>>>>> >>> > thanks,<br>
>>>>> >>> > Christian Sell<br>
>>>>> >>> ><br>
>>>>> >>> > _______________________________________________<br>
>>>>> >>> > Powered by <a href="http://www.kitware.com" rel="noreferrer" target="_blank">www.kitware.com</a><br>
>>>>> >>> ><br>
>>>>> >>> > Visit other Kitware open-source projects at<br>
>>>>> >>> > <a href="http://www.kitware.com/opensource/opensource.html" rel="noreferrer" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
>>>>> >>> ><br>
>>>>> >>> > Search the list archives at:<br>
>>>>> >>> <a href="http://markmail.org/search/?q=vtk-developers" rel="noreferrer" target="_blank">http://markmail.org/search/?q=vtk-developers</a><br>
>>>>> >>> ><br>
>>>>> >>> > Follow this link to subscribe/unsubscribe:<br>
>>>>> >>> > <a href="http://public.kitware.com/mailman/listinfo/vtk-developers" rel="noreferrer" target="_blank">http://public.kitware.com/mailman/listinfo/vtk-developers</a><br>
>>>>> >>><br>
>>>>> >>><br>
>>>>> >>><br>
>>>>> >>> _______________________________________________<br>
>>>>> >>> Powered by <a href="http://www.kitware.com" rel="noreferrer" target="_blank">www.kitware.com</a><br>
>>>>> >>><br>
>>>>> >>> Visit other Kitware open-source projects at<br>
>>>>> >>> <a href="http://www.kitware.com/opensource/opensource.html" rel="noreferrer" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
>>>>> >>><br>
>>>>> >>> Search the list archives at:<br>
>>>>> <a href="http://markmail.org/search/?q=vtk-developers" rel="noreferrer" target="_blank">http://markmail.org/search/?q=vtk-developers</a><br>
>>>>> >>><br>
>>>>> >>> Follow this link to subscribe/unsubscribe:<br>
>>>>> >>> <a href="http://public.kitware.com/mailman/listinfo/vtk-developers" rel="noreferrer" target="_blank">http://public.kitware.com/mailman/listinfo/vtk-developers</a><br>
>>>>> >>><br>
>>>>> >>> --<br>
>>>>> >> Sankhesh Jhaveri *Sr. Research & Development Engineer* | Kitware<br>
>>>>> >> <<a href="http://www.kitware.com/" rel="noreferrer" target="_blank">http://www.kitware.com/</a>> | (518) 881-4417<br>
>>>>> >> <br>
>>>>> >> _______________________________________________<br>
>>>>> >> Powered by <a href="http://www.kitware.com" rel="noreferrer" target="_blank">www.kitware.com</a><br>
>>>>> >><br>
>>>>> >> Visit other Kitware open-source projects at<br>
>>>>> >> <a href="http://www.kitware.com/opensource/opensource.html" rel="noreferrer" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
>>>>> >><br>
>>>>> >> Search the list archives at:<br>
>>>>> <a href="http://markmail.org/search/?q=vtk-developers" rel="noreferrer" target="_blank">http://markmail.org/search/?q=vtk-developers</a><br>
>>>>> >><br>
>>>>> >> Follow this link to subscribe/unsubscribe:<br>
>>>>> >> <a href="http://public.kitware.com/mailman/listinfo/vtk-developers" rel="noreferrer" target="_blank">http://public.kitware.com/mailman/listinfo/vtk-developers</a><br>
>>>>> >><br>
>>>>> >><br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>><br>
>>><br>
>>><br>
>>><br>
>>><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Powered by <a href="http://www.kitware.com" rel="noreferrer" target="_blank">www.kitware.com</a><br>
><br>
> Visit other Kitware open-source projects at<br>
> <a href="http://www.kitware.com/opensource/opensource.html" rel="noreferrer" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
><br>
> Search the list archives at: <a href="http://markmail.org/search/?q=vtk-developers" rel="noreferrer" target="_blank">http://markmail.org/search/?q=vtk-developers</a><br>
><br>
> Follow this link to subscribe/unsubscribe:<br>
> <a href="http://public.kitware.com/mailman/listinfo/vtk-developers" rel="noreferrer" target="_blank">http://public.kitware.com/mailman/listinfo/vtk-developers</a><br>
<br>
<br>
<br>
_______________________________________________<br>
Powered by <a href="http://www.kitware.com" rel="noreferrer" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" rel="noreferrer" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Search the list archives at: <a href="http://markmail.org/search/?q=vtk-developers" rel="noreferrer" target="_blank">http://markmail.org/search/?q=vtk-developers</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://public.kitware.com/mailman/listinfo/vtk-developers" rel="noreferrer" target="_blank">http://public.kitware.com/mailman/listinfo/vtk-developers</a><br>
<br>
</blockquote></div></div>