[vtk-developers] OpenVR and HTC Vive support

[Ken Martin]

First off the HTC Vive is awesome. It gives a glimpse at what can be done
and it is a game changer. The ability to go into VR and look at, and
interact with a dataset, model, or point cloud is incredibly satisfying and
I suspect productive. In the next few years I believe it will continue to
improve because it is just that compelling.  Better VR hardware and a lot
more rendering performance is only going to make it more compelling.

So if you have a Vive or other OpenVR compatible device you should be able
to use it with VTK now. I have created a vtkOpenVR remote module that
includes drop in replacements for the core rendering and interaction
classes. It is alpha code and has only been tested with the Vive but it
seems to work. I have attached the VTK Remote Module
<http://www.vtk.org/Wiki/VTK/Remote_Modules> file to this email.  It
requires master git VTK. Note in VR basically you are always doing an
interactive render and you have to maintain really fast interaction rates,
as an example just using the cell picker for the dragon.ply model was
causing interruptions. We will have to look more into asynchronous/threaded
processing etc as we move forward. Below find the current README.md for the
module. Feel free to improve/fix/question/etc.

Enjoy!
Ken



** BETA/ALPHA code here, feel free to use and improve **
<https://gitlab.kitware.com/ken-martin/vtkVive#compiling>Compiling

This module is part of VTK
<https://gitlab.kitware.com/ken-martin/vtkVive/blob/master/vtk.org> and
relies on two external libraries. You will need to download/compile/install
SDL2 <https://wiki.libsdl.org/Installation> as well as OpenVR
<https://github.com/ValveSoftware/openvr>. The CMake build process for this
module will try to find those two libraries. Build an optimized version as
you will need the performance.
<https://gitlab.kitware.com/ken-martin/vtkVive#what-is-included>What is
included

These classes are designed to be drop in replacements for VTK. As long as
you link your executable to this module the object factory mechanism should
replace the core rendering classes such as vtkRenderWindow with OpenVR
specialized versions. The goal is for VTK programs to be able to use the
OpenVR library with little to no changes.

vtkOpenVRRenderWIndow handles the bulk of interfacing to OpenVR. It
supports one renderer currently. The renderer is assumed to cover the
entire window which is what makes sense to VR. Overlay renderers can
probably be made to work with this but consider how overlays will appear in
a HMD if they do not track the viewpoint etc. This class is based on sample
code from the OpenVR project so it may not be completely ideal for VTK in
its current form.

The vtkOpenVRCamera gets the matrices from OpenVR to use for rendering. It
contains a scale and translation that are designed to map world coordinates
into the HMD space (meters around 0,0,0 I believe) This way world
coordinates can be kept in the units that makes sense for your problem
domain and the camera will handle shifting and scaling those coordinates
into the units that make sens for the HMD.

vtkOpenVRRenderer that exists to make ResetCamera do something reasonable.
In this case it computes a reasonable scale and translation and sets that
on the OpenVRCmaera. It also sets the default clipping range expansion to
be more reasonable as the VTK default is way too generous.

Supporting interaction is a little odd. All of VTK is designed to pick and
interact based on X,Y mouse window coordinates. With OpenVR we actually
have XYZ world coordinates and WXYZ orientations. The
vtkOpenVRRenderWindowInteractor catches controller events and converts them
to mouse window events (yuck) but it also stores the world coordinate
positions and orientations for styles/pickers that can use them (yay). It
supports multiple controller through the standard PointerIndex approach
used already by VTK for multitouch.

To work with the vtkOpenVRRenderWindowInteractor we have a
vtkInteractorStyleOpenVR that uses world coordinate XYZ position, WXYZ
orientation to adjust actors. This provides a nice grab and move style of
interaction that is common to OpenVR.

To determine what prop has been picked we have a vtkOpenVRPropPicker that
uses the XYZ world coordinate as the picking value as opposed to
intersecting a ray which is slower etc.
<https://gitlab.kitware.com/ken-martin/vtkVive#todo>TODO

there are a few areas that are ripe for improvement.

   -

   Add support for the OpenVR overlay, it is a great way to display a user
   interface and OpenVR has a lot of support built in for it.
   -

   Think seriously about picking and interactors/widgets in the context of
   having a XYZ position at the beginning of the pick operation.
   -

   Performance, make it faster, make sure we are doing it the best way
   -

   Performance, make sure long operations happen asynchronously
   -

   Add more event interactions




-- 
Ken Martin PhD
Chairman & CFO
Kitware Inc.
28 Corporate Drive
Clifton Park NY 12065
518 371 3971

This communication, including all attachments, contains confidential and
legally privileged information, and it is intended only for the use of the
addressee.  Access to this email by anyone else is unauthorized. If you are
not the intended recipient, any disclosure, copying, distribution or any
action taken in reliance on it is prohibited and may be unlawful. If you
received this communication in error please notify us immediately and
destroy the original message.  Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20160505/e887f4e0/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: vtkOpenVR.remote.cmake
Type: application/octet-stream
Size: 205 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20160505/e887f4e0/attachment.obj>