[Paraview-developers] What I have learned about providing a VR system that is intuitive to use while keeping discomfort to a minimum.

[Magnus Elden]

Hello!

I sent a mail to Kitware and got a reply that I should send this info to the VTK and Paraview developer mailing lists.
I would like to start with saying that the following words are based on my personal experience and the feedback I have gotten from the small group of people around me who have tried VR systems. As such, please think about what I have to say and then do your own testing to see what works for your own systems and use cases.

I have been working on using Paraview with UE4 to provide a platform for VR to be used as a tool of scientific visualisation. I have been working on this for only half a year, and I started by looking into maybe modifying Paraview to render to OpenVR. However, I was defeated by the fact that I could not for the life of me understand how to use VTK to open xdmf files. I got errors complaining about time all the time. I eventually hacked a python script together to render and create a bunch of slices that I imported to UE4 and I wrote my own volume renderer there.

You are basically doing what I am doing for my master's thesis and I believe that having VR support directly integrated into VTK or Paraview is the best way to go for VR visualization for researchers. Since I am not that well versed with the Paraview and VTK systems I was unable to help directly by adding code, especially since I had nobody to really turn to, but I can at least provide some hints based on my experience.

First and foremost, when running in a VR environment it is very important for the user that they have a floor and maybe some walls which they can use for reference. Blurring, depth of field and motion blur are also enemies of a good VR experience. The Paraview overlap detection also seemed slightly wonky as I had to re-grab several times. Unlike a standard screen a VR system covers the entire visual field and makes it impossible for the brain to calibrate using peripheral vision. This leads to mismatch between visual cues and the other signals the brain receives. This mismatch leads to the users feeling nausea and other discomforts. This makes the standard systems used for visualizing 3D models and volumes are not optimal. In short, make it feel like you have the model in front of you inside a world you have created. Imagine a hologram in a science fiction film.

Finally, from feedback, the handling should be similar to handling objects in real life in order to minimize confusion and make it natural. This means that the model is not necessarily in the center of the space around which the camera rotates. The models can be moved by grabbing with the controllers and scaled in the models local space, not world space, by grabbing the model with both hands. Handling things like cutting planes will also have to be like moving a stick or something else physical we handle in the real world with our hands. The way the cutting plane in Paraview is already working is a very good example of how it can be. Grab the center with your left hand and place it where you want it to be while you grab the rotational "stick" with your right hand and orient it.

Quickly I want to answer what I find to be the strengths and weaknesses of both the Paraview, and similar platforms, and game engines like UE4. Paraview is great for volumetric visualisations and quick iterative analysis. Colour encoding and iso-surface generation and all the other powerful tools of Paraview is great for the researchers who can understand the colours, symbols and encoded data. It is not good for creating a interactive, wholesome experience that can be experiences by several users at once. This is where UE4 can shine. As a research tool it is not great, but it is great at presentation. Simplified and processed models can be made to move and interact with any number of concurrent users while showing animations and playing sounds. This allows UE4 to be a great tool to Conway the processed data to those learning or otherwise without the required specialized knowledge. For education and presentation I can see a lot of use for these simplified and processed models.

I wrote this because I believe VR can be a great aid for research. I also know that Paraview and similar software are made for the researchers and I want to stress that Paraview should keep that focus. It can be among the best tools a researcher has and as such it should not diminish itself by trying to also be something it is not.
I hope that this will lead to Paraview and VTK being used to give even better functionality for the scientific world.

Yours,
Magnus Elden

If you want some info on what I am doing and the state of the volume rendering in VR in UE4, please refer to this forum thread: https://forums.unrealengine.com/showthread.php?119267-Your-thoughts-on-and-comments-to-Volume-Rendering-in-Unreal-Engine-4/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview-developers/attachments/20161102/44e5d075/attachment.html>