[Paraview] crash when picking with 'p' for stream tracer : Tag mismatch: got 1, expecting 22222
Burlen Loring
bloring at lbl.gov
Tue Jun 4 12:43:11 EDT 2013
Thanks for looking at it Utkarsh! I will try it out and let you know if
it comes up again.
On 06/04/2013 08:27 AM, Utkarsh Ayachit wrote:
> Burlen,
>
> I've pushed a fix (patch attached) http://paraview.org/Bug/view.php?id=14112
> Let me know if you're still seeing the issues.
>
> Thanks,
> Utkarsh
>
>
> On Tue, Jun 4, 2013 at 10:30 AM, Utkarsh Ayachit
> <utkarsh.ayachit at kitware.com> wrote:
>> Burlen,
>>
>> I think I know what's causing the segfaults. I'll have a fix soon.
>> Thanks for catching (and reporting) this :).
>>
>> Utkarsh
>>
>> On Mon, Jun 3, 2013 at 8:05 PM, Burlen Loring <bloring at lbl.gov> wrote:
>>> another data point on this, a similar crash occurs seemingly randomly (see
>>> output below). I'm seeing this crash or a variant of it on pretty much all
>>> the systems I'm using. Sometimes I can run for 5min and other a few hours,
>>> but I'm crashing alot always with: Tag mismatch: got 1, expecting 22222.
>>>
>>>
>>> ERROR: In
>>> /home/bloring/builds/ParaView/VTK/Parallel/Core/vtkSocketCommunicator.cxx,
>>> line 878
>>> vtkSocketCommunicator (0xf2d270): Tag mismatch: got 1, expecting 22222.
>>>
>>> pvserver:
>>> /home/bloring/builds/ParaView/VTK/Parallel/Core/vtkMultiProcessStream.cxx:637:
>>> vtkMultiProcessStream& vtkMultiProcessStream::operator>>(unsigned int&):
>>> Assertion `this->Internals->Data.front() == vtkInternals::uint32_value'
>>> failed.
>>> Generic Warning: In
>>> /home/bloring/builds/ParaView/VTK/Parallel/MPI/vtkMPICommunicator.cxx, line
>>> 99
>>>
>>> MPI had an error
>>> ------------------------------------------------
>>>
>>> A process has failed, error stack:
>>> PMPI_Barrier(426)...........: MPI_Barrier(MPI_COMM_WORLD) failed
>>> MPIR_Barrier_impl(294)......:
>>> MPIR_Barrier_or_coll_fn(121):
>>> MPIR_Barrier_intra(83)......:
>>> dequeue_and_set_error(888)..: Communication error with rank 0
>>> MPIR_Barrier_impl(308)......:
>>> MPIR_Bcast_impl(1369).......:
>>> MPIR_Bcast_intra(1199)......:
>>> MPIR_Bcast_binomial(220)....: Failure during collective
>>>
>>> Program Stack:
>>> ------------------------------------------------
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelMPI-pv4.0.so.1
>>> [0x2b773f94aea7]
>>> /home/bloring/installs/mpich/3.0.4/lib/libmpich.so.10(MPIR_Err_return_comm+0x1bd)
>>> [0x2b77418465bd]
>>> /home/bloring/installs/mpich/3.0.4/lib/libmpich.so.10(MPI_Barrier+0xda)
>>> [0x2b77418bd7ca]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelMPI-pv4.0.so.1
>>> [0x2b773f94ac95]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelMPI-pv4.0.so.1(_ZN18vtkMPICommunicator18BroadcastVoidArrayEPvxii+0x2f)
>>> [0x2b773f94e3ab]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelCore-pv4.0.so.1
>>> [0x2b773fb9b4bc]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelCore-pv4.0.so.1(_ZN15vtkCommunicator9BroadcastER21vtkMultiProcessStreami+0x110)
>>> [0x2b773fb9a732]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVClientServerCoreRendering-pv4.0.so.1
>>> [0x2b774d5c7901]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVClientServerCoreRendering-pv4.0.so.1(_ZN30vtkPVSynchronizedRenderWindows20SatelliteStartRenderEP15vtkRenderWindow+0x46)
>>> [0x2b774d60938a]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVClientServerCoreRendering-pv4.0.so.1(_ZN30vtkPVSynchronizedRenderWindows17HandleStartRenderEP15vtkRenderWindow+0xc0)
>>> [0x2b774d608f36]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVClientServerCoreRendering-pv4.0.so.1
>>> [0x2b774d60db58]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkCommonCore-pv4.0.so.1
>>> [0x2b77431ba183]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkCommonCore-pv4.0.so.1(_ZN9vtkObject11InvokeEventEmPv+0x3d)
>>> [0x2b77431ba693]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkRenderingCore-pv4.0.so.1(_ZN15vtkRenderWindow6RenderEv+0x1e7)
>>> [0x2b773b696c2d]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkRenderingCore-pv4.0.so.1(_ZN19vtkHardwareSelector14CaptureBuffersEv+0x473)
>>> [0x2b773b5fb60b]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVVTKExtensionsRendering-pv4.0.so.1(_ZN21vtkPVHardwareSelector13PrepareSelectEv+0xef)
>>> [0x2b774e61ce93]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVVTKExtensionsRendering-pv4.0.so.1(_ZN21vtkPVHardwareSelector6SelectEPi+0x1c)
>>> [0x2b774e61cee8]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVClientServerCoreRendering-pv4.0.so.1(_ZN15vtkPVRenderView6SelectEiPi+0x9f)
>>> [0x2b774d5f714b]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVClientServerCoreRendering-pv4.0.so.1(_ZN15vtkPVRenderView11SelectCellsEPi+0x25)
>>> [0x2b774d5f6e99]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVServerManagerApplication-pv4.0.so.1(_ZN15vtkPVRenderView11SelectCellsEiiii+0x44)
>>> [0x2b7739ab6736]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVServerManagerApplication-pv4.0.so.1(_Z22vtkPVRenderViewCommandP26vtkClientServerInterpreterP13vtkObjectBasePKcRK21vtkClientServerStreamRS5_+0x22dc)
>>> [0x2b7739ab214f]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkClientServer-pv4.0.so.1(_ZN26vtkClientServerInterpreter20ProcessCommandInvokeERK21vtkClientServerStreami+0x18e)
>>> [0x2b773ec67564]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkClientServer-pv4.0.so.1(_ZN26vtkClientServerInterpreter17ProcessOneMessageERK21vtkClientServerStreami+0x135)
>>> [0x2b773ec66c9d]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkClientServer-pv4.0.so.1(_ZN26vtkClientServerInterpreter13ProcessStreamERK21vtkClientServerStream+0x2f)
>>> [0x2b773ec66b37]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVServerImplementationCore-pv4.0.so.1(_ZN16vtkPVSessionCore21ExecuteStreamInternalERK21vtkClientServerStreamb+0xdb)
>>> [0x2b773a91b7bf]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVServerImplementationCore-pv4.0.so.1(_ZN16vtkPVSessionCore30ExecuteStreamSatelliteCallbackEv+0xd5)
>>> [0x2b773a91b6a3]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkPVServerImplementationCore-pv4.0.so.1
>>> [0x2b773a91938f]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelCore-pv4.0.so.1(_ZN25vtkMultiProcessController10ProcessRMIEiPvii+0x3a6)
>>> [0x2b773fba72f2]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelCore-pv4.0.so.1(_ZN25vtkMultiProcessController11ProcessRMIsEii+0x63c)
>>> [0x2b773fba6e88]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/libvtkParallelCore-pv4.0.so.1(_ZN25vtkMultiProcessController11ProcessRMIsEv+0x22)
>>> [0x2b773fba684a]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/pvserver [0x401d30]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/pvserver(main+0x25)
>>> [0x401dca]
>>> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3fab01d994]
>>> /home/bloring/installs/ParaView/4.0.0/lib/paraview-4.0/pvserver [0x401a09]
>>>
>>>
>>>
>>>
>>>
>>> On 05/31/2013 03:15 PM, Burlen Loring wrote:
>>>> can also reproduce with the point source (open data, change to surface
>>>> rep, add point source, pick)
>>>> vtk dataset that can be used to reproduce:
>>>> http://hpcvis.com/downloads/pr1-vti.tar.gz
>>>>
>>>> On 05/31/2013 01:54 PM, Burlen Loring wrote:
>>>>> Hi All,
>>>>>
>>>>> With git master (first noticed a few days ago, rebased this am) I'm
>>>>> hitting crash when attempting to use the stream tracer interactive picking a
>>>>> seed.
>>>>>
>>>>> 1) load the data,
>>>>> 2) change to surface rep,
>>>>> 3) add stream tracer filter
>>>>> 4) hit the 'p' key to select a seed -> crash.
>>>>>
>>>>> I've only been able to reproduce with a real dataset (1024^3 image mesh).
>>>>> the small data I've tried from ParaViewData doesn't lead to the crash,
>>>>> However, there's plenty of ram (first encountered using 32 ranks 4 ranks per
>>>>> node 48g per node on longhorn reduced to 4 ranks 24g on my workstation for
>>>>> debugging). I can share this dataset if it helps to debug, or you may have
>>>>> to use a large dataset to trigger.
>>>>>
>>>>> filed a bug report: http://paraview.org/Bug/view.php?id=14107
>>>>>
>>>>> here's cerr output and a stack
>>>>>
>>>>> 3437 ERROR: In
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Parallel/Core/vtkSocketCommunicator.cxx,
>>>>> line 877
>>>>> 3438 vtkSocketCommunicator (0x2efd2d0): Tag mismatch: got 1, expecting
>>>>> 22222.
>>>>> 3439
>>>>> 3440 pvserver:
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Parallel/Core/vtkMultiProcessStream.cxx:637:
>>>>> vtkMultiProcessStream &vtkMultiProcessStream::operator>>(unsigned int &):
>>>>> Assertion `this->Internals->Data.front() == vtkInter nals::uint32_value'
>>>>> failed.
>>>>>
>>>>> #0 0x00000033aea30215 in raise () from /lib64/libc.so.6
>>>>> #1 0x00000033aea31cc0 in abort () from /lib64/libc.so.6
>>>>> #2 0x00000033aea29696 in __assert_fail () from /lib64/libc.so.6
>>>>> #3 0x00002b6ae159c3ed in vtkMultiProcessStream::operator>>
>>>>> (this=0x7fffea94ed68, value=@0x7fffea94ed1c)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Parallel/Core/vtkMultiProcessStream.cxx:637
>>>>> #4 0x00002b6aef371cdd in
>>>>> vtkPVSynchronizedRenderWindows::LoadWindowAndLayout (this=0x2835d50,
>>>>> window=0x26e9650, stream=@0x7fffea94ed68)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVSynchronizedRenderWindows.cxx:998
>>>>> #5 0x00002b6aef37155b in vtkPVSynchronizedRenderWindows::RootStartRender
>>>>> (this=0x2835d50, renWin=0x26e9650)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVSynchronizedRenderWindows.cxx:901
>>>>> #6 0x00002b6aef3711aa in
>>>>> vtkPVSynchronizedRenderWindows::HandleStartRender (this=0x2835d50,
>>>>> renWin=0x26e9650)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVSynchronizedRenderWindows.cxx:814
>>>>> #7 0x00002b6aef36dc5a in
>>>>> vtkPVSynchronizedRenderWindows::vtkObserver::Execute (this=0x282f660,
>>>>> ocaller=0x26e9650, eventId=3)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVSynchronizedRenderWindows.cxx:219
>>>>> #8 0x00002b6ae374c9ff in vtkSubjectHelper::InvokeEvent (this=0x282e970,
>>>>> event=3, callData=0x0, self=0x26e9650) at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Common/Core/vtkObject.cxx:620
>>>>> #9 0x00002b6ae374d028 in vtkObject::InvokeEvent (this=0x26e9650,
>>>>> event=3, callData=0x0) at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Common/Core/vtkObject.cxx:789
>>>>> #10 0x00002b6adc9ee77a in vtkRenderWindow::Render (this=0x26e9650) at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Rendering/Core/vtkRenderWindow.cxx:313
>>>>> #11 0x00002b6af42443f7 in vtkXOpenGLRenderWindow::Render (this=0x26e9650)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Rendering/OpenGL/vtkXOpenGLRenderWindow.cxx:1826
>>>>> #12 0x00002b6adc9183d6 in vtkHardwareSelector::CaptureBuffers
>>>>> (this=0x26e9460) at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Rendering/Core/vtkHardwareSelector.cxx:288
>>>>> #13 0x00002b6af05119ef in vtkPVHardwareSelector::PrepareSelect
>>>>> (this=0x26e9460) at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/VTKExtensions/Rendering/vtkPVHardwareSelector.cxx:62
>>>>> #14 0x00002b6af0511a54 in vtkPVHardwareSelector::Select (this=0x26e9460,
>>>>> region=0x7fffea94f750)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/VTKExtensions/Rendering/vtkPVHardwareSelector.cxx:75
>>>>> #15 0x00002b6aef353e31 in vtkPVRenderView::Select (this=0x2833ce0,
>>>>> fieldAssociation=1, region=0x7fffea94f750)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVRenderView.cxx:597
>>>>> #16 0x00002b6aef353a19 in vtkPVRenderView::SelectCells (this=0x2833ce0,
>>>>> region=0x7fffea94f750)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVRenderView.cxx:556
>>>>> #17 0x00002b6ada8b1064 in vtkPVRenderView::SelectCells (this=0x2833ce0,
>>>>> region0=218, region1=142, region2=218, region3=142)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVRenderView.h:246
>>>>> #18 0x00002b6ada8ab300 in vtkPVRenderViewCommand (arlu=0x227b4f0,
>>>>> ob=0x2833ce0, method=0x3dfbd19 "SelectCells", msg=@0x7fffea951538,
>>>>> resultStream=@0x227bdf0)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/PV/Wrapping/ClientServer/vtkPVRenderViewClientServer.cxx:545
>>>>> #19 0x00002b6ae04b896e in
>>>>> vtkClientServerInterpreter::ProcessCommandInvoke (this=0x227b4f0,
>>>>> css=@0x7fffea951b90, midx=0)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerStream/vtkClientServerInterpreter.cxx:379
>>>>> #20 0x00002b6ae04b7d5b in vtkClientServerInterpreter::ProcessOneMessage
>>>>> (this=0x227b4f0, css=@0x7fffea951b90, message=0)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerStream/vtkClientServerInterpreter.cxx:214
>>>>> #21 0x00002b6ae04b7c0e in vtkClientServerInterpreter::ProcessStream
>>>>> (this=0x227b4f0, css=@0x7fffea951b90)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerStream/vtkClientServerInterpreter.cxx:183
>>>>> #22 0x00002b6adba173cd in vtkPVSessionCore::ExecuteStreamInternal
>>>>> (this=0x227b990, stream=@0x7fffea951b90, ignore_errors=false)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ServerImplementation/Core/vtkPVSessionCore.cxx:665
>>>>> #23 0x00002b6adba1711e in vtkPVSessionCore::ExecuteStream
>>>>> (this=0x227b990, location=21, stream=@0x7fffea951b90, ignore_errors=false)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ServerImplementation/Core/vtkPVSessionCore.cxx:635
>>>>> ---Type <return> to continue, or q <return> to quit---
>>>>> #24 0x00002b6adba11dde in vtkPVSessionBase::ExecuteStream
>>>>> (this=0x1a6ae50, location=21, stream=@0x7fffea951b90, ignore_errors=false)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ServerImplementation/Core/vtkPVSessionBase.cxx:173
>>>>> #25 0x00002b6adba284b4 in vtkPVSessionServer::OnClientServerMessageRMI
>>>>> (this=0x1a6ae50, message=0x3cd3d00, message_length=16)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ServerImplementation/Core/vtkPVSessionServer.cxx:510
>>>>> #26 0x00002b6adba25d2e in (anonymous namespace)::RMICallback
>>>>> (localArg=0x1a6ae50, remoteArg=0x3cd3d00, remoteArgLength=16)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ServerImplementation/Core/vtkPVSessionServer.cxx:56
>>>>> #27 0x00002b6ae158abcc in vtkMultiProcessController::ProcessRMI
>>>>> (this=0x2efd1a0, remoteProcessId=1, arg=0x3cd3d00, argLength=16,
>>>>> rmiTag=55625)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Parallel/Core/vtkMultiProcessController.cxx:650
>>>>> #28 0x00002b6ae158a48f in vtkMultiProcessController::ProcessRMIs
>>>>> (this=0x2efd1a0, reportErrors=0, dont_loop=1)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/VTK/Parallel/Core/vtkMultiProcessController.cxx:596
>>>>> #29 0x00002b6adbec2d07 in
>>>>> vtkTCPNetworkAccessManager::ProcessEventsInternal (this=0x174f090,
>>>>> timeout_msecs=0, do_processing=true)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Core/vtkTCPNetworkAccessManager.cxx:268
>>>>> #30 0x00002b6adbec25d0 in vtkTCPNetworkAccessManager::ProcessEvents
>>>>> (this=0x174f090, timeout_msecs=0)
>>>>> at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/ParaViewCore/ClientServerCore/Core/vtkTCPNetworkAccessManager.cxx:165
>>>>> #31 0x00000000004035e0 in RealMain (argc=5, argv=0x7fffea953418,
>>>>> type=vtkProcessModule::PROCESS_SERVER) at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/CommandLineExecutables/pvserver_common.h:90
>>>>> #32 0x000000000040305d in main (argc=5, argv=0x7fffea953418) at
>>>>> /scratch/01237/bloring/ParaView/next/ParaView/CommandLineExecutables/pvserver.cxx:21
>>>>>
>>> _______________________________________________
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>>
>>> Please keep messages on-topic and check the ParaView Wiki at:
>>> http://paraview.org/Wiki/ParaView
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://www.paraview.org/mailman/listinfo/paraview
More information about the ParaView
mailing list