[Paraview] crash when picking with 'p' for stream tracer : Tag mismatch: got 1, expecting 22222
Utkarsh Ayachit
utkarsh.ayachit at kitware.com
Tue Jun 4 10:30:16 EDT 2013
Burlen,
I think I know what's causing the segfaults. I'll have a fix soon.
Thanks for catching (and reporting) this :).
Utkarsh
On Mon, Jun 3, 2013 at 8:05 PM, Burlen Loring <bloring at lbl.gov> wrote:
> another data point on this, a similar crash occurs seemingly randomly (see
> output below). I'm seeing this crash or a variant of it on pretty much all
> the systems I'm using. Sometimes I can run for 5min and other a few hours,
> but I'm crashing alot always with: Tag mismatch: got 1, expecting 22222.
>
>
> ERROR: In
> /home/bloring/builds/ParaView/VTK/Parallel/Core/vtkSocketCommunicator.cxx,
> line 878
> vtkSocketCommunicator (0xf2d270): Tag mismatch: got 1, expecting 22222.
>
> pvserver:
> /home/bloring/builds/ParaView/VTK/Parallel/Core/vtkMultiProcessStream.cxx:637:
> vtkMultiProcessStream& vtkMultiProcessStream::operator>>(unsigned int&):
> Assertion `this->Internals->Data.front() == vtkInternals::uint32_value'
> failed.
> Generic Warning: In
> /home/bloring/builds/ParaView/VTK/Parallel/MPI/vtkMPICommunicator.cxx, line
> 99
>
> MPI had an error
> ------------------------------------------------
>
> A process has failed, error stack:
> PMPI_Barrier(426)...........: MPI_Barrier(MPI_COMM_WORLD) failed
> MPIR_Barrier_impl(294)......:
> MPIR_Barrier_or_coll_fn(121):
> MPIR_Barrier_intra(83)......:
> dequeue_and_set_error(888)..: Communication error with rank 0
> MPIR_Barrier_impl(308)......:
> MPIR_Bcast_impl(1369).......:
> MPIR_Bcast_intra(1199)......:
> MPIR_Bcast_binomial(220)....: Failure during collective
>
> Program Stack:
> ------------------------------------------------
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelMPI-pv4.0.so.1
> [0x2b773f94aea7]
> /home/bloring/installs/mpich/3.0.4/lib/libmpich.so.10(MPIR_Err_return_comm+0x1bd)
> [0x2b77418465bd]
> /home/bloring/installs/mpich/3.0.4/lib/libmpich.so.10(MPI_Barrier+0xda)
> [0x2b77418bd7ca]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelMPI-pv4.0.so.1
> [0x2b773f94ac95]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelMPI-pv4.0.so.1(_ZN18vtkMPICommunicator18BroadcastVoidArrayEPvxii+0x2f)
> [0x2b773f94e3ab]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelCore-pv4.0.so.1
> [0x2b773fb9b4bc]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelCore-pv4.0.so.1(_ZN15vtkCommunicator9BroadcastER21vtkMultiProcessStreami+0x110)
> [0x2b773fb9a732]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVClientServerCoreRendering-pv4.0.so.1
> [0x2b774d5c7901]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVClientServerCoreRendering-pv4.0.so.1(_ZN30vtkPVSynchronizedRenderWindows20SatelliteStartRenderEP15vtkRenderWindow+0x46)
> [0x2b774d60938a]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVClientServerCoreRendering-pv4.0.so.1(_ZN30vtkPVSynchronizedRenderWindows17HandleStartRenderEP15vtkRenderWindow+0xc0)
> [0x2b774d608f36]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVClientServerCoreRendering-pv4.0.so.1
> [0x2b774d60db58]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkCommonCore-pv4.0.so.1
> [0x2b77431ba183]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkCommonCore-pv4.0.so.1(_ZN9vtkObject11InvokeEventEmPv+0x3d)
> [0x2b77431ba693]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkRenderingCore-pv4.0.so.1(_ZN15vtkRenderWindow6RenderEv+0x1e7)
> [0x2b773b696c2d]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkRenderingCore-pv4.0.so.1(_ZN19vtkHardwareSelector14CaptureBuffersEv+0x473)
> [0x2b773b5fb60b]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVVTKExtensionsRendering-pv4.0.so.1(_ZN21vtkPVHardwareSelector13PrepareSelectEv+0xef)
> [0x2b774e61ce93]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVVTKExtensionsRendering-pv4.0.so.1(_ZN21vtkPVHardwareSelector6SelectEPi+0x1c)
> [0x2b774e61cee8]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVClientServerCoreRendering-pv4.0.so.1(_ZN15vtkPVRenderView6SelectEiPi+0x9f)
> [0x2b774d5f714b]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVClientServerCoreRendering-pv4.0.so.1(_ZN15vtkPVRenderView11SelectCellsEPi+0x25)
> [0x2b774d5f6e99]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVServerManagerApplication-pv4.0.so.1(_ZN15vtkPVRenderView11SelectCellsEiiii+0x44)
> [0x2b7739ab6736]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVServerManagerApplication-pv4.0.so.1(_Z22vtkPVRenderViewCommandP26vtkClientServerInterpreterP13vtkObjectBasePKcRK21vtkClientServerStreamRS5_+0x22dc)
> [0x2b7739ab214f]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkClientServer-pv4.0.so.1(_ZN26vtkClientServerInterpreter20ProcessCommandInvokeERK21vtkClientServerStreami+0x18e)
> [0x2b773ec67564]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkClientServer-pv4.0.so.1(_ZN26vtkClientServerInterpreter17ProcessOneMessageERK21vtkClientServerStreami+0x135)
> [0x2b773ec66c9d]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkClientServer-pv4.0.so.1(_ZN26vtkClientServerInterpreter13ProcessStreamERK21vtkClientServerStream+0x2f)
> [0x2b773ec66b37]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVServerImplementationCore-pv4.0.so.1(_ZN16vtkPVSessionCore21ExecuteStreamInternalERK21vtkClientServerStreamb+0xdb)
> [0x2b773a91b7bf]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVServerImplementationCore-pv4.0.so.1(_ZN16vtkPVSessionCore30ExecuteStreamSatelliteCallbackEv+0xd5)
> [0x2b773a91b6a3]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVServerImplementationCore-pv4.0.so.1
> [0x2b773a91938f]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelCore-pv4.0.so.1(_ZN25vtkMultiProcessController10ProcessRMIEiPvii+0x3a6)
> [0x2b773fba72f2]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelCore-pv4.0.so.1(_ZN25vtkMultiProcessController11ProcessRMIsEii+0x63c)
> [0x2b773fba6e88]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelCore-pv4.0.so.1(_ZN25vtkMultiProcessController11ProcessRMIsEv+0x22)
> [0x2b773fba684a]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/pvserver [0x401d30]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/pvserver(main+0x25)
> [0x401dca]
> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3fab01d994]
> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/pvserver [0x401a09]
>
>
>
>
>
> On 05/31/2013 03:15 PM, Burlen Loring wrote:
>>
>> can also reproduce with the point source (open data, change to surface
>> rep, add point source, pick)
>> vtk dataset that can be used to reproduce:
>> http://hpcvis.com/downloads/pr1-vti.tar.gz
>>
>> On 05/31/2013 01:54 PM, Burlen Loring wrote:
>>>
>>> Hi All,
>>>
>>> With git master (first noticed a few days ago, rebased this am) I'm
>>> hitting crash when attempting to use the stream tracer interactive picking a
>>> seed.
>>>
>>> 1) load the data,
>>> 2) change to surface rep,
>>> 3) add stream tracer filter
>>> 4) hit the 'p' key to select a seed -> crash.
>>>
>>> I've only been able to reproduce with a real dataset (1024^3 image mesh).
>>> the small data I've tried from ParaViewData doesn't lead to the crash,
>>> However, there's plenty of ram (first encountered using 32 ranks 4 ranks per
>>> node 48g per node on longhorn reduced to 4 ranks 24g on my workstation for
>>> debugging). I can share this dataset if it helps to debug, or you may have
>>> to use a large dataset to trigger.
>>>
>>> filed a bug report: http://paraview.org/Bug/view.php?id=14107
>>>
>>> here's cerr output and a stack
>>>
>>> 3437 ERROR: In
>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Parallel/Core/vtkSocketCommunicator.cxx,
>>> line 877
>>> 3438 vtkSocketCommunicator (0x2efd2d0): Tag mismatch: got 1, expecting
>>> 22222.
>>> 3439
>>> 3440 pvserver:
>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Parallel/Core/vtkMultiProcessStream.cxx:637:
>>> vtkMultiProcessStream &vtkMultiProcessStream::operator>>(unsigned int &):
>>> Assertion `this->Internals->Data.front() == vtkInter nals::uint32_value'
>>> failed.
>>>
>>> #0 0x00000033aea30215 in raise () from /lib64/libc.so.6
>>> #1 0x00000033aea31cc0 in abort () from /lib64/libc.so.6
>>> #2 0x00000033aea29696 in __assert_fail () from /lib64/libc.so.6
>>> #3 0x00002b6ae159c3ed in vtkMultiProcessStream::operator>>
>>> (this=0x7fffea94ed68, value=@0x7fffea94ed1c)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Parallel/Core/vtkMultiProcessStream.cxx:637
>>> #4 0x00002b6aef371cdd in
>>> vtkPVSynchronizedRenderWindows::LoadWindowAndLayout (this=0x2835d50,
>>> window=0x26e9650, stream=@0x7fffea94ed68)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVSynchronizedRenderWindows.cxx:998
>>> #5 0x00002b6aef37155b in vtkPVSynchronizedRenderWindows::RootStartRender
>>> (this=0x2835d50, renWin=0x26e9650)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVSynchronizedRenderWindows.cxx:901
>>> #6 0x00002b6aef3711aa in
>>> vtkPVSynchronizedRenderWindows::HandleStartRender (this=0x2835d50,
>>> renWin=0x26e9650)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVSynchronizedRenderWindows.cxx:814
>>> #7 0x00002b6aef36dc5a in
>>> vtkPVSynchronizedRenderWindows::vtkObserver::Execute (this=0x282f660,
>>> ocaller=0x26e9650, eventId=3)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVSynchronizedRenderWindows.cxx:219
>>> #8 0x00002b6ae374c9ff in vtkSubjectHelper::InvokeEvent (this=0x282e970,
>>> event=3, callData=0x0, self=0x26e9650) at
>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Common/Core/vtkObject.cxx:620
>>> #9 0x00002b6ae374d028 in vtkObject::InvokeEvent (this=0x26e9650,
>>> event=3, callData=0x0) at
>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Common/Core/vtkObject.cxx:789
>>> #10 0x00002b6adc9ee77a in vtkRenderWindow::Render (this=0x26e9650) at
>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Rendering/Core/vtkRenderWindow.cxx:313
>>> #11 0x00002b6af42443f7 in vtkXOpenGLRenderWindow::Render (this=0x26e9650)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Rendering/OpenGL/vtkXOpenGLRenderWindow.cxx:1826
>>> #12 0x00002b6adc9183d6 in vtkHardwareSelector::CaptureBuffers
>>> (this=0x26e9460) at
>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Rendering/Core/vtkHardwareSelector.cxx:288
>>> #13 0x00002b6af05119ef in vtkPVHardwareSelector::PrepareSelect
>>> (this=0x26e9460) at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/VTKExtensions/Rendering/vtkPVHardwareSelector.cxx:62
>>> #14 0x00002b6af0511a54 in vtkPVHardwareSelector::Select (this=0x26e9460,
>>> region=0x7fffea94f750)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/VTKExtensions/Rendering/vtkPVHardwareSelector.cxx:75
>>> #15 0x00002b6aef353e31 in vtkPVRenderView::Select (this=0x2833ce0,
>>> fieldAssociation=1, region=0x7fffea94f750)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVRenderView.cxx:597
>>> #16 0x00002b6aef353a19 in vtkPVRenderView::SelectCells (this=0x2833ce0,
>>> region=0x7fffea94f750)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVRenderView.cxx:556
>>> #17 0x00002b6ada8b1064 in vtkPVRenderView::SelectCells (this=0x2833ce0,
>>> region0=218, region1=142, region2=218, region3=142)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVRenderView.h:246
>>> #18 0x00002b6ada8ab300 in vtkPVRenderViewCommand (arlu=0x227b4f0,
>>> ob=0x2833ce0, method=0x3dfbd19 "SelectCells", msg=@0x7fffea951538,
>>> resultStream=@0x227bdf0)
>>> at
>>> /scratch/01237/bloring/ParaView/next/PV/Wrapping/ClientServer/vtkPVRenderViewClientServer.cxx:545
>>> #19 0x00002b6ae04b896e in
>>> vtkClientServerInterpreter::ProcessCommandInvoke (this=0x227b4f0,
>>> css=@0x7fffea951b90, midx=0)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerStream/vtkClientServerInterpreter.cxx:379
>>> #20 0x00002b6ae04b7d5b in vtkClientServerInterpreter::ProcessOneMessage
>>> (this=0x227b4f0, css=@0x7fffea951b90, message=0)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerStream/vtkClientServerInterpreter.cxx:214
>>> #21 0x00002b6ae04b7c0e in vtkClientServerInterpreter::ProcessStream
>>> (this=0x227b4f0, css=@0x7fffea951b90)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerStream/vtkClientServerInterpreter.cxx:183
>>> #22 0x00002b6adba173cd in vtkPVSessionCore::ExecuteStreamInternal
>>> (this=0x227b990, stream=@0x7fffea951b90, ignore_errors=false)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ServerImplementation/Core/vtkPVSessionCore.cxx:665
>>> #23 0x00002b6adba1711e in vtkPVSessionCore::ExecuteStream
>>> (this=0x227b990, location=21, stream=@0x7fffea951b90, ignore_errors=false)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ServerImplementation/Core/vtkPVSessionCore.cxx:635
>>> ---Type <return> to continue, or q <return> to quit---
>>> #24 0x00002b6adba11dde in vtkPVSessionBase::ExecuteStream
>>> (this=0x1a6ae50, location=21, stream=@0x7fffea951b90, ignore_errors=false)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ServerImplementation/Core/vtkPVSessionBase.cxx:173
>>> #25 0x00002b6adba284b4 in vtkPVSessionServer::OnClientServerMessageRMI
>>> (this=0x1a6ae50, message=0x3cd3d00, message_length=16)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ServerImplementation/Core/vtkPVSessionServer.cxx:510
>>> #26 0x00002b6adba25d2e in (anonymous namespace)::RMICallback
>>> (localArg=0x1a6ae50, remoteArg=0x3cd3d00, remoteArgLength=16)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ServerImplementation/Core/vtkPVSessionServer.cxx:56
>>> #27 0x00002b6ae158abcc in vtkMultiProcessController::ProcessRMI
>>> (this=0x2efd1a0, remoteProcessId=1, arg=0x3cd3d00, argLength=16,
>>> rmiTag=55625)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Parallel/Core/vtkMultiProcessController.cxx:650
>>> #28 0x00002b6ae158a48f in vtkMultiProcessController::ProcessRMIs
>>> (this=0x2efd1a0, reportErrors=0, dont_loop=1)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Parallel/Core/vtkMultiProcessController.cxx:596
>>> #29 0x00002b6adbec2d07 in
>>> vtkTCPNetworkAccessManager::ProcessEventsInternal (this=0x174f090,
>>> timeout_msecs=0, do_processing=true)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Core/vtkTCPNetworkAccessManager.cxx:268
>>> #30 0x00002b6adbec25d0 in vtkTCPNetworkAccessManager::ProcessEvents
>>> (this=0x174f090, timeout_msecs=0)
>>> at
>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Core/vtkTCPNetworkAccessManager.cxx:165
>>> #31 0x00000000004035e0 in RealMain (argc=5, argv=0x7fffea953418,
>>> type=vtkProcessModule::PROCESS_SERVER) at
>>> /scratch/01237/bloring/ParaView/next/ParaView/CommandLineExecutables/pvserver_common.h:90
>>> #32 0x000000000040305d in main (argc=5, argv=0x7fffea953418) at
>>> /scratch/01237/bloring/ParaView/next/ParaView/CommandLineExecutables/pvserver.cxx:21
>>>
>>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at:
> http://paraview.org/Wiki/ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://www.paraview.org/mailman/listinfo/paraview
More information about the ParaView
mailing list