[Paraview] Crash using openmpi with more than 4 processes

[Bart Janssens]

On Wednesday 03 February 2010 09:58:53 pm Berk Geveci wrote:
> Any chance you can attach with a debugger to one of the nodes that
> crash? It is hard to tell what is wrong from the errors you posted. We

I have recompiled with debugging symbols and attached a backtrace from the first pvserver process. The crash is in Mesa, and happens both with the stock Mesa 6.5 from CentOS and after upgrading to the 
latest stable version from source (7.6.1). The relevant piece of mesa code appears to be the jmp call below (Mesa 7.6.1):

#elif defined(PTHREADS)
        pushq   %rdi
        call    _x86_64_get_dispatch at PLT
        popq    %rdi
        movq    2344(%rax), %r11
        jmp     *%r11
#else

I have absolutely no clue what this bit of code does, or why it fails here (especially only when more than 4 processes are running). I have tried using both 8 processes on the same node and 8 processes on 
different nodes. This looks very similar to these threads:
http://markmail.org/thread/owglrjmzkqlfmuhm
and
http://markmail.org/message/5eekpxkkh5iput2w

Somehow, I get the feeling this is not going to be easy... :)

> have definitely run ParaView with OpenMPI with 4 core and more. Also,
> what do you mean by "simple VTK file"? Small file? Big file?
 It's a very small file (10700 cells), in this case the "airFoil2D"  OpenFOAM tutorial file after conversion. It loads fine with exactly the same setup using 4 processes.

Cheers,

-- 
Bart
-------------- next part --------------
Program received signal SIGSEGV, Segmentation fault.
glMatrixMode () at x86-64/glapi_x86-64.S:10535
10535           jmp     *%r11
Current language:  auto; currently asm
(gdb) bt
#0  glMatrixMode () at x86-64/glapi_x86-64.S:10535
#1  0x00002b7d8618fe6a in vtkOpenGLRenderWindow::OpenGLInit (this=0x9dbe290)
    at /home/bjanssens/src/paraview/ParaView3/VTK/Rendering/vtkOpenGLRenderWindow.cxx:241
#2  0x00002b7d861ec5ff in vtkXOpenGLRenderWindow::CreateOffScreenWindow (this=0x9dbe290, width=1098, height=786)
    at /home/bjanssens/src/paraview/ParaView3/VTK/Rendering/vtkXOpenGLRenderWindow.cxx:952
#3  0x00002b7d861ec703 in vtkXOpenGLRenderWindow::Initialize (this=0x9dbe290)
    at /home/bjanssens/src/paraview/ParaView3/VTK/Rendering/vtkXOpenGLRenderWindow.cxx:1067
#4  0x00002b7d861e6a9e in vtkXOpenGLRenderWindow::Start (this=0x9dbe290)
    at /home/bjanssens/src/paraview/ParaView3/VTK/Rendering/vtkXOpenGLRenderWindow.cxx:1186
#5  0x00002b7d860bb014 in vtkRenderWindow::DoStereoRender (this=0x9dbe290)
    at /home/bjanssens/src/paraview/ParaView3/VTK/Rendering/vtkRenderWindow.cxx:688
#6  0x00002b7d860bb701 in vtkRenderWindow::DoFDRender (this=0x9dbe290)
    at /home/bjanssens/src/paraview/ParaView3/VTK/Rendering/vtkRenderWindow.cxx:677
#7  0x00002b7d860bbedd in vtkRenderWindow::DoAARender (this=0x9dbe290)
    at /home/bjanssens/src/paraview/ParaView3/VTK/Rendering/vtkRenderWindow.cxx:564
#8  0x00002b7d860bc70f in vtkRenderWindow::Render (this=0x9dbe290)
    at /home/bjanssens/src/paraview/ParaView3/VTK/Rendering/vtkRenderWindow.cxx:377
#9  0x00002b7d861e7cb6 in vtkXOpenGLRenderWindow::Render (this=0x9dbe290)
    at /home/bjanssens/src/paraview/ParaView3/VTK/Rendering/vtkXOpenGLRenderWindow.cxx:1846
#10 0x00002b7d84df3f3a in vtkParallelRenderManager::RenderRMI (this=0x9d720a0)
    at /home/bjanssens/src/paraview/ParaView3/VTK/Parallel/vtkParallelRenderManager.cxx:757
#11 0x00002b7d7b02a1b2 in vtkPVClientServerRenderManager::RenderRMI (this=0x9d720a0)
    at /home/bjanssens/src/paraview/ParaView3/Servers/Filters/vtkPVClientServerRenderManager.h:55
#12 0x00002b7d7b028440 in RenderRMI (arg=0x9d720a0)
    at /home/bjanssens/src/paraview/ParaView3/Servers/Filters/vtkPVClientServerRenderManager.cxx:57
#13 0x00002b7d84d48658 in vtkMultiProcessController::ProcessRMI (this=0x9d62b50, remoteProcessId=1, arg=0x0, argLength=0,
    rmiTag=34532) at /home/bjanssens/src/paraview/ParaView3/VTK/Parallel/vtkMultiProcessController.cxx:592
#14 0x00002b7d84d49278 in vtkMultiProcessController::ProcessRMIs (this=0x9d62b50, reportErrors=0, dont_loop=1)
    at /home/bjanssens/src/paraview/ParaView3/VTK/Parallel/vtkMultiProcessController.cxx:545
#15 0x00002b7d7aad5576 in vtkRemoteConnection::ProcessCommunication (this=0x9d62770)
    at /home/bjanssens/src/paraview/ParaView3/Servers/Common/vtkRemoteConnection.cxx:75
#16 0x00002b7d7aa69f19 in vtkProcessModuleConnectionManager::MonitorConnections (this=0x9d6e640, msec=0)
    at /home/bjanssens/src/paraview/ParaView3/Servers/Common/vtkProcessModuleConnectionManager.cxx:429
#17 0x00002b7d7aa76cd9 in vtkProcessModule::StartServer (this=0x98dbb90, msec=0)
    at /home/bjanssens/src/paraview/ParaView3/Servers/Common/vtkProcessModule.cxx:444
#18 0x00002b7d7aa777c4 in vtkProcessModule::Start (this=0x98dbb90, argc=1, argv=0x98da360)
    at /home/bjanssens/src/paraview/ParaView3/Servers/Common/vtkProcessModule.cxx:355
#19 0x00002b7d7b06cecc in vtkPVMain::Run (this=0x98cc1b0, options=0x98da1e0)
    at /home/bjanssens/src/paraview/ParaView3/Servers/Filters/vtkPVMain.cxx:273
#20 0x0000000000401296 in main (argc=4, argv=0x7fffd3aa8da8)
    at /home/bjanssens/src/paraview/ParaView3/Servers/Executables/pvserver.cxx:45