[Paraview] Is there any point to compiling with MPI but without offscreen rendering?

Tue Oct 9 10:43:35 EDT 2012

Pat

Great summary.

Did writing that give you any ideas on the best way to simplify the options and #ifdefs.

JB

From: paraview-bounces at paraview.org [mailto:paraview-bounces at paraview.org] On Behalf Of Pat Marion
Sent: 09 October 2012 15:28
To: Cook, Rich; paraview at paraview.org
Subject: Re: [Paraview] Is there any point to compiling with MPI but without offscreen rendering?

Hi Rich,

I totally agreee with you, I find it very confusing that there is a compile time flag VTK_USE_OFFSCREEN, and a runtime flag --use-offscreen-rendering, and an OSMesa compile flag VTK_OPENGL_HAS_OSMESA.  I think that the options could be presented in a way that makes it more clear what is happening.

I want to clarify some things here.  There are actually several very different concepts that are all kinda mixed up by the way that VTK presents the options:

1) hardware accelerated rendering vs. software rendering
2) regular rendering vs. offscreen rendering vs. OSMesa rendering

Just saying "offscreen rendering" is totally not specific enough to have any idea what is really happening under the hood, a lot of people assume different things when they hear "offscreen rendering", and I think that's where the problem is.

First, hardware vs. software rendering.  If you use the Mesa OpenGL driver (not to be confused with the OSMesa library!), you are using software rendering.  But, the mesa driver renders into a buffer that is owned by the operating system and window manager (X11).  So you are using a software implementation of OpenGL, but rendering "on screen", and you depend on X11 APIs and an X server.  If you have a GPU, then you can use a hardware driver, and similarly you depend on X11.  You can switch between a hardware driver and a Mesa driver without recompiling any software.

Second, regular rendering vs. offscreen rendering.  Here is an old, but very informative, article on the many different implementations of offscreen rendering:   http://www.mesa3d.org/brianp/sig97/offscrn.htm
Note that Mesa offscreen rendering, OSMesa, is just one of many different offscreen rendering APIs described in the article.  You can do hardware accelerated offscreen rendering, and that is totally different from OSMesa.  For example, using X11, you can render into a GLX pixmap.  A GLX pixmap is a render buffer that is not associated with a window, it is "offscreen", but the rendering uses your GL driver, which could be a hardware accelerated driver or a Mesa software driver, but not OSMesa.

Finally, there is OSMesa.  It uses Mesa software rendering, and renders into a special OSMesa buffer.  Using OSMesa, you have zero dependency on X11.  You don't need X11 at build time or at run time.  Here's more information about OSMesa: http://www.mesa3d.org/osmesa.html

So, in summary,

-you can use a GPU or Mesa (not OSMesa) and do on screen rendering into an X11 window.
-you can use a GPU or Mesa (not OSMesa) and do offscreen rendering into an X11 managed offscreen buffer.
-you can use OSMesa for software rendering without any dependency on X11.

For MPI cluster runs of paraview, the two most common configurations are GPU with X11, or OSMesa without X11.  If you didn't have a GPU on your nodes, you might also use Mesa with X11, but render on screen.

If you grep VTK source code for "OSMESA", you will find multiple confusing flags, VTK_OPENGL_HAS_OSMESA and VTK_USE_OSMESA, and lots of confusing code to decide whether or not we want to use on screen or offscreen rendering with gpu, or OSMesa rendering.  If VTK is truly compiled for OSMesa, it means that ldd pvserver lists only OSMesa and no X11 or OpenGL dependencies.  When you have compiled with OSMesa, the --use-offscreen-rendering flag can be completely ignored, VTK will always use OSMesa.

I think the --use-offscreen-rendering flag is mostly for switching between on screen and offscreen while still using a regular GL driver, and I'm really not sure if anybody actually uses that feature, I certainly have not.  To make things even more confusing, in ParaView there is a preferences setting for "Use offscreen rendering for screenshots".  Ouch.

Hope this helps!

Pat

p.s. If anybody has corrections, please let me know!

On Mon, Oct 8, 2012 at 2:23 PM, Burlen Loring <bloring at lbl.gov<mailto:bloring at lbl.gov>> wrote:
Hi Rich,

if you are installing on cluster, I would say that it's easier to use an MPI build. Of course when you build with MPI support you need to launch pvserver with mpirun (or the system equivalent). When you need a serial run just specify -n 1. You may be aware that, providing a server config will let your users select the number of mpi ranks and the "layout" of mpi ranks across cluster nodes from ParaView's GUI. ParaView should run on any number of mpi ranks. When you say you're running in serial, are you using the mpirun command? what errors/complaints are you experiencing?

My recollection, is that the --use-offscreen-rendering is used to select between osmesa open gl and the system open gl when both are used in the same build. I haven't built ParaView this way in many years, hence my recollection is fuzzy. On the other hand, if you have two builds, one built with osmesa and the other using a system provided hardware accelerated open gl, this command line flag is not necessary.

Burlen

On 10/08/2012 10:21 AM, Cook, Rich wrote:
Are you assuming we are always running in parallel on the cluster?  I find that Paraview for some reason complains when run in serial if you compile it with MPI.  So currently I'm doing four builds, opengl, osmesa, opengl-mpi, osmesa-mpi.  It's a PITA but now it's all automated.  Still I think it's confusing for our users.

I'm also personally unsure about where there is a switch for "--use-offscreen-rendering."  If I compile with OSMesa, I'm always going to want that, right?  And if OpenGL, I'm always going to disable that, right?  Why is there a command line switch for that?

It does sound like yes, pvservers will render to graphics cards in parallel so that's good to know.

Thanks!
-- Rich

On Oct 6, 2012, at 7:16 PM, Burlen Loring wrote:
Hi Rich,

You're always going to want to build with MPI when running on a cluster. Leveraging your cluster's graphics cards comes down to building with or without os mesa. you probably want to provide both builds, and select the build to run depending on whether the user has requested nodes that contain graphics cards or not. Hardware accelerated rendering can be faster than software based rendering depending on the amount of contention there is for the graphics card. eg 16 mpi ranks hitting the same card will likely be slower than 16 cpu cores running os mesa. When submitting a job you could always control this by limiting the number of mpi ranks per graphics card. Also, some rendering algorithms, such as surface LIC, are disabled when using os mesa. it's nice to have the hardware accelerated build in order to access the other algorithms when you need them.

Burlen

On 10/1/2012 5:05 PM, Cook, Rich wrote:
We have large clusters and some of them have graphics cards on them.  Most don't.  So normally I expect to be using -DVTK_USE_OFFSCREEN:BOOL=ON   -DPARAVIEW_USE_MPI:BOOL=ON with cmake.
I got to thinking.  First, if I compile with -DVTK_USE_OFFSCREEN:BOOL=ON then why do I have to use --use-offscreen-rendering to launch the pvservers.  Secondly, can pvservers render to graphics cards for distributed rendering under MPI?  If so, does it make sense to do -DVTK_USE_OFFSCREEN:BOOL=OFF   -DPARAVIEW_USE_MPI:BOOL=ON ??

_______________________________________________
Powered by www.kitware.com<http://www.kitware.com>

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView

Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20121009/0c1678e4/attachment.htm>