[vtkusers] Advice on mitigating choppy resize with Qt 5 + VTK 7.1
Aron Helser
aron.helser at kitware.com
Fri Jun 9 09:34:34 EDT 2017
Hi Elvis,
In my experience on Windows, resize is an extremely expensive operation for
an OpenGL pipeline, because essentially everthing has to be rebuilt because
the frame buffer changed. I do think my previous experience may be a bit
out of date, but I think this issue would need some profiling evaluation or
some outside advice/examples to give guidance on how to proceed.
I haven't followed the twists of this thread, but is there an example (or
counter example) that you have found that says resize could be fast? I'm
afraid I haven't taken the time to investigate recently.
I'm sorry to be a bit of a wet blanket, but I am concerned that there just
may not be a fast path for resize (most games don't resize their windows :)
)
Aron
On Fri, Jun 9, 2017 at 8:24 AM, Elvis Stansvik <elvis.stansvik at orexplore.com
> wrote:
> 2017-06-08 8:45 GMT+02:00 Elvis Stansvik <elvis.stansvik at orexplore.com>:
>
>>
>> Den 8 juni 2017 8:26 fm skrev "Manfred Gratt" <gratt at intales.com>:
>> >
>> >
>> >
>> > On 07/06/17 14:15, Elvis Stansvik wrote:
>> >>
>> >> 2017-03-13 10:37 GMT+01:00 Elvis Stansvik <
>> elvis.stansvik at orexplore.com>:
>> >>>
>> >>> 2017-03-13 10:32 GMT+01:00 Elvis Stansvik <
>> elvis.stansvik at orexplore.com>:
>> >>> > 2017-03-07 16:10 GMT+01:00 Elvis Stansvik <
>> elvis.stansvik at orexplore.com>:
>> >>> >> 2017-03-07 15:58 GMT+01:00 Elvis Stansvik <
>> elvis.stansvik at orexplore.com>:
>> >>> >>> 2017-03-07 15:53 GMT+01:00 David Cole <DLRdave at aol.com>:
>> >>> >>>> On Windows, we make resizing our windows interactive by getting
>> the
>> >>> >>>> vtkInteractorStyle associated with the render window containing
>> the
>> >>> >>>> volume rendering, and then calling StartState at resize begin
>> time
>> >>> >>>> (OnEnterSizeMove) and StopState at resize end time
>> (OnExitSizeMove).
>> >>> >>>>
>> >>> >>>> I suppose there may be a Qt equivalent which works on many
>> platforms
>> >>> >>>> for beginning and ending a resize action. If not, there are
>> definitely
>> >>> >>>> platform-specific hooks you can intercept to achieve smooth
>> resizing
>> >>> >>>> with this technique.
>> >>> >>>>
>> >>> >>>> Wrapping anything in a StartState/StopState pair on the
>> >>> >>>> vtkInteractorStyle will cause "interactive frame rate rendering"
>> to be
>> >>> >>>> in effect in between the calls. The volume rendering is not as
>> nice
>> >>> >>>> looking during interactions, but it is definitely speedier.
>> >>> >>>
>> >>> >>> Ah, I was just about to reply to myself with some further
>> information:
>> >>> >>>
>> >>> >>> I know that during interaction, the quality of the rendering is
>> >>> >>> decreased, and that this can account for some of the performance
>> >>> >>> discrepancy I'm seeing between camera movement vs window resize.
>> >>> >>>
>> >>> >>> But, I've experimented with disabling the quality degradation
>> during
>> >>> >>> interactions (so that the two should be on "equal footing"), and
>> the
>> >>> >>> resizing is still much more choppy than when interacting with the
>> >>> >>> camera. So there must be something else.
>> >>> >>>
>> >>> >>> In fact, in the screencast I showed, I wasn't using the volume
>> >>> >>> renderer's default built-in quality degradation during
>> interaction.
>> >>> >>> I'm using my own since I've found that VTKs own is a little too
>> >>> >>> aggressive in degrading the quality to maintain frame rate.
>> >>> >>
>> >>> >> To illustrate, look at the attached screen recording. In this test
>> >>> >> case, I'm using
>> >>> >>
>> >>> >> mapper->AutoAdjustSampleDistancesOff();
>> >>> >> mapper->SetSampleDistance(0.0002);
>> >>> >>
>> >>> >> on my vtkGPUVolumeRayCastMapper, to turn off the automatic
>> adjustment
>> >>> >> of sample distance during interactions, and hardcode the sample
>> >>> >> distance to 0.0002.
>> >>> >>
>> >>> >> Notice how the rendering is still reasonably smooth during
>> >>> >> interaction, but during resize, the rendering sometimes lags with
>> 100s
>> >>> >> of milliseconds. During the resizing I was getting warnings like
>> >>> >>
>> >>> >> Warning: In /buildbot/vtk7-builder/build/R
>> endering/OpenGL2/vtkXOpenGLRenderWindow.cxx,
>> >>> >> line 1207
>> >>> >> vtkXOpenGLRenderWindow (0x2a629d0): warning window did not resize
>> in
>> >>> >> the allotted time
>> >>> >>
>> >>> >> printed.
>> >>> >
>> >>> > No ideas where this discrepancy in refresh rate, despite automatic
>> >>> > adjustment of sample distance being turned off, comes from?
>> >>> >
>> >>> > At the moment I've worked around it by simply resorting to bounding
>> >>> > box rendering when my VTK windows are resized, since the choppy
>> resize
>> >>> > behavior was quite jarring. But that's a kludge and I'd much rather
>> >>> > find the real problem :/
>> >>> >
>> >>> > Below is a minimal test case, and I'm attaching a video where I
>> first
>> >>> > interact with the volume rendering, giving a smooth albeit a little
>> >>> > slow frame rate, and then resizing the window, which gives a jerky
>> and
>> >>> > choppy rendering.
>> >>> >
>> >>> > This was on a Thinkpad T440s laptop with Intel HD 4400 graphics. VTK
>> >>> > 7.1 and Qt 5.5.1 running on Kubuntu 16.04.
>> >>> >
>> >>> > Elvis
>> >>> >
>> >>> >
>> >>> > main.cpp:
>> >>> >
>> >>> > #include <random>
>> >>> >
>> >>> > #include <vtkCamera.h>
>> >>> > #include <vtkColorTransferFunction.h>
>> >>> > #include <vtkGPUVolumeRayCastMapper.h>
>> >>> > #include <vtkImageData.h>
>> >>> > #include <vtkNew.h>
>> >>> > #include <vtkPiecewiseFunction.h>
>> >>> > #include <vtkRenderer.h>
>> >>> > #include <vtkRenderWindow.h>
>> >>> > #include <vtkRenderWindowInteractor.h>
>> >>> > #include <vtkVolume.h>
>> >>> > #include <vtkVolumeProperty.h>
>> >>> >
>> >>> > int main(int argc, char *argv[])
>> >>> > {
>> >>> > vtkNew<vtkColorTransferFunction> colorFunction;
>> >>> > colorFunction->AddRGBPoint(0.0, 0.0, 0.0, 0.0);
>> >>> > colorFunction->AddRGBPoint(1.0, 0.0, 0.0, 0.0);
>> >>> >
>> >>> > vtkNew<vtkPiecewiseFunction> opacityFunction;
>> >>> > opacityFunction->AddPoint(0.0, 0.0);
>> >>> > opacityFunction->AddPoint(1.0, 0.02);
>> >>> >
>> >>> > vtkNew<vtkImageData> data;
>> >>> > data->SetExtent(0, 250, 0, 250, 0, 500);
>> >>> > data->AllocateScalars(VTK_FLOAT, 1);
>> >>> > std::random_device device;
>> >>> > std::mt19937 engine(device());
>> >>> > std::uniform_real_distribution<> distribution(0, 1);
>> >>> > auto dataPointer = static_cast<float
>> *>(data->GetScalarPointer());
>> >>> > for (int i = 0; i < 250*250*500; ++i) {
>> >>> > *dataPointer = distribution(engine);
>> >>> > ++dataPointer;
>> >>> > }
>> >>> >
>> >>> > vtkNew<vtkGPUVolumeRayCastMapper> mapper;
>> >>> > mapper->SetInputData(data.Get());
>> >>>
>> >>> Sorry, the test case should have had
>> >>>
>> >>> mapper->AutoAdjustSampleDistancesOff();
>> >>> mapper->SetSampleDistance(1);
>> >>>
>> >>> here, to show that the discrepancy is there even when automatic
>> >>> adjustment of sample distance is turned off.
>> >>>
>> >>> The result is the same, the rendering is keeping up much better during
>> >>> interaction that it is during window resize.
>> >>>
>> >>> I can understand if it's a little heavier to re-render after window
>> >>> resize, but not by that much, so I think something is wrong
>> >>
>> >>
>> >> Anybody have some more advise on this? I'm seeing the same problem in
>> PV 5.4.
>> >>
>> >> I had hoped that maybe the new QVTKOpenGLWidget would change things,
>> but it hasn't. I would really like to remove the workarounds I have in
>> place that degrades the rendering to bounding boxes during widget resizing.
>> >>
>> >> Elvis
>> >
>> > Hi Elvis,
>> > if you are using Qt5.5 and VTK the problem may be related to this bug:
>> >
>> > http://vtk.1045678.n5.nabble.com/Delayed-mouse-interaction-d
>> ue-to-event-loop-bug-in-Qt5-td5730474.html
>> >
>> > https://bugreports.qt.io/browse/QTBUG-40889
>> >
>> > I have this problem with VTK6 and VTK7 using Ubuntu 16.04 but it was
>> fixed in Ubuntu 16.10
>> > that has Qt5.6 which fixes the bug. Have you tried your program on a
>> newer version of Ubuntu?
>>
>> Sorry that my thread has gotten so long, but I mentioned QTBUG-40889 in
>> an earlier mail, and that's unfortunately not it. I'm well aware of that
>> bug and even submitted an SRU to backport the upstream fix to 16.04 (not
>> sure of the status of that).
>>
>> I also see the problem both with my AppImage build, which uses 5.6, and
>> on macOS, where I use 5.8.
>>
> I just tested on Windows 7 as well (Qt 5.6, VTK 8.0.0.rc1), and if
> anything, the problem is even worse than on Linux :/
>
> Elvis
>
>> What I would expect is that resizing a window, VTK should be able to
>> maintain roughly the same framerate as during normal interaction (sample
>> distance reduction turned off). But instead, each frame is seemingly taking
>> a lot more time, which I think is what leads to the stuttering.
>>
>> The test case I sent earlier should show the problem, even on newer Qt
>> versions.
>>
>> Elvis
>>
>> >
>> > Manfred
>> >
>> >
>> >
>> >>
>> >>>
>> >>>
>> >>> Elvis
>> >>>
>> >>> >
>> >>> > vtkNew<vtkVolumeProperty> property;
>> >>> > property->SetScalarOpacity(opacityFunction.Get());
>> >>> > property->SetColor(colorFunction.Get());
>> >>> >
>> >>> > vtkNew<vtkVolume> volume;
>> >>> > volume->SetMapper(mapper.Get());
>> >>> > volume->SetProperty(property.Get());
>> >>> >
>> >>> > vtkNew<vtkRenderer> renderer;
>> >>> > renderer->AddVolume(volume.Get());
>> >>> > renderer->SetBackground(1.0, 1.0, 1.0);
>> >>> >
>> >>> > vtkNew<vtkRenderWindow> window;
>> >>> > window->AddRenderer(renderer.Get());
>> >>> >
>> >>> > renderer->ResetCamera();
>> >>> >
>> >>> > vtkNew<vtkRenderWindowInteractor> interactor;
>> >>> > interactor->SetRenderWindow(window.Get());
>> >>> > interactor->Start();
>> >>> >
>> >>> > return 0;
>> >>> > }
>> >>> >
>> >>> >
>> >>> > CMakeLists.txt:
>> >>> >
>> >>> > cmake_minimum_required(VERSION 3.1)
>> >>> >
>> >>> > project(TestCase)
>> >>> >
>> >>> > set(CMAKE_PREFIX_PATH ${CMAKE_PREFIX_PATH} /opt/VTK7)
>> >>> >
>> >>> > find_package(VTK 7.1 COMPONENTS
>> >>> > vtkCommonCore
>> >>> > vtkCommonDataModel
>> >>> > vtkCommonExecutionModel
>> >>> > vtkCommonMath
>> >>> > vtkInteractionStyle
>> >>> > vtkRenderingCore
>> >>> > vtkRenderingOpenGL2
>> >>> > vtkRenderingVolume
>> >>> > vtkRenderingVolumeOpenGL2
>> >>> > REQUIRED
>> >>> > )
>> >>> >
>> >>> > add_executable(TestCase WIN32 main.cpp)
>> >>> >
>> >>> > target_link_libraries(TestCase PUBLIC
>> >>> > vtkCommonCore
>> >>> > vtkCommonDataModel
>> >>> > vtkCommonExecutionModel
>> >>> > vtkCommonMath
>> >>> > vtkInteractionStyle
>> >>> > vtkRenderingCore
>> >>> > vtkRenderingOpenGL2
>> >>> > vtkRenderingVolume
>> >>> > vtkRenderingVolumeOpenGL2
>> >>> > )
>> >>> >
>> >>> > target_include_directories(TestCase PUBLIC
>> >>> > ${VTK_INCLUDE_DIRS}
>> >>> > )
>> >>> >
>> >>> > target_compile_definitions(TestCase PUBLIC
>> >>> > ${VTK_DEFINITIONS}
>> >>> > )
>> >>> >
>> >>> > set_target_properties(TestCase PROPERTIES
>> >>> > CXX_STANDARD 14
>> >>> > CXX_STANDARD_REQUIRED ON
>> >>> > )
>> >>> >
>> >>> >>
>> >>> >> Elvis
>> >>> >>
>> >>> >>>
>> >>> >>> Elvis
>> >>> >>>
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> HTH,
>> >>> >>>> David C.
>> >>> >>>>
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> On Tue, Mar 7, 2017 at 9:33 AM, Elvis Stansvik
>> >>> >>>> <elvis.stansvik at orexplore.com> wrote:
>> >>> >>>>> 2017-03-07 15:17 GMT+01:00 Elvis Stansvik <
>> elvis.stansvik at orexplore.com>:
>> >>> >>>>>> Hi all,
>> >>> >>>>>>
>> >>> >>>>>> I'm using Qt 5.5.1 and VTK 7.1. The program has a couple of
>> VTKWidget
>> >>> >>>>>> showing volume renderings, as well as a window with a chart.
>> >>> >>>>>
>> >>> >>>>> Actually, I'm able to reproduce this behavior when not using Qt
>> at
>> >>> >>>>> all, but just a regular render window + interactor setup with a
>> single
>> >>> >>>>> volume rendered. Camera interaction is nice and fast, but
>> resizing the
>> >>> >>>>> window, the rendering is very choppy. I'm also getting
>> >>> >>>>>
>> >>> >>>>> Warning: In /buildbot/vtk7-builder/build/R
>> endering/OpenGL2/vtkXOpenGLRenderWindow.cxx,
>> >>> >>>>> line 1207
>> >>> >>>>> vtkXOpenGLRenderWindow (0x1353a10): warning window did not
>> resize in
>> >>> >>>>> the allotted time
>> >>> >>>>>
>> >>> >>>>> printed to the console, so it seems VTK detects what I'm seeing
>> >>> >>>>> visually. I did not get this warning printed in the Qt + VTK
>> app.
>> >>> >>>>>
>> >>> >>>>> Elvis
>> >>> >>>>>
>> >>> >>>>>>
>> >>> >>>>>> Have a look at the attached screen recording. Notice how camera
>> >>> >>>>>> interaction in both VTK render windows is nice and smooth, but
>> when
>> >>> >>>>>> resizing the windows, the updating of the renderings is very
>> >>> >>>>>> slow/choppy.
>> >>> >>>>>>
>> >>> >>>>>> I've been trying to debug this, or at least finding a way of
>> >>> >>>>>> mitigating it. Could it be that Qt is delivering too many
>> resize
>> >>> >>>>>> events? Has anyone else dealt with this problem?
>> >>> >>>>>>
>> >>> >>>>>> Thanks in advance for any advice,
>> >>> >>>>>> Elvis
>> >>> >>>>> _______________________________________________
>> >>> >>>>> Powered by www.kitware.com
>> >>> >>>>>
>> >>> >>>>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>> >>> >>>>>
>> >>> >>>>> Please keep messages on-topic and check the VTK FAQ at:
>> http://www.vtk.org/Wiki/VTK_FAQ
>> >>> >>>>>
>> >>> >>>>> Search the list archives at: http://markmail.org/search/?q=
>> vtkusers
>> >>> >>>>>
>> >>> >>>>> Follow this link to subscribe/unsubscribe:
>> >>> >>>>> http://public.kitware.com/mailman/listinfo/vtkusers
>> >>
>> >>
>> >>
>> >>
>> >> _______________________________________________ Powered by
>> www.kitware.com Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html Please keep messages
>> on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ
>> Search the list archives at: http://markmail.org/search/?q=vtkusers
>> Follow this link to subscribe/unsubscribe: http://public.kitware.com/mail
>> man/listinfo/vtkusers
>> >
>> >
>> > _______________________________________________
>> > Powered by www.kitware.com
>> >
>> > Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>> >
>> > Please keep messages on-topic and check the VTK FAQ at:
>> http://www.vtk.org/Wiki/VTK_FAQ
>> >
>> > Search the list archives at: http://markmail.org/search/?q=vtkusers
>> >
>> > Follow this link to subscribe/unsubscribe:
>> > http://public.kitware.com/mailman/listinfo/vtkusers
>> >
>>
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/
> opensource/opensource.html
>
> Please keep messages on-topic and check the VTK FAQ at:
> http://www.vtk.org/Wiki/VTK_FAQ
>
> Search the list archives at: http://markmail.org/search/?q=vtkusers
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/vtkusers
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtkusers/attachments/20170609/b98c306b/attachment-0001.html>
More information about the vtkusers
mailing list