[Paraview] Python interpreter not being created for coprocessing?

Vanmoer, Mark W mvanmoer at illinois.edu
Tue Sep 11 11:21:43 EDT 2012

Previous message: [Paraview] How to visualize this 2d tecplot file?
Next message: [Paraview] Python interpreter not being created for coprocessing?
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Hi Andy,

Sorry, I lost track of where I was in debugging. What I had was an openmpi/pgi debug build that passed the ctest -R CoPro tests. But the app I'm working on still crashes when trying to create a python interpreter.

I started over from scratch working with the Wiki example, using gcc 4.4.6 and mpich2 1.4.1p1.

I have the C++ wiki example working with both the ParaView-3.14.1-Source tarball and with git commit 2537cabbf0cc9b39f17f347105ee18c559d0041a.

I then made a Fortran version of the wiki example. It works with the ParaView-3.14.1-Source tarball build, but crashes when built against the git branch build. Seems to be at the same spot as the pgi build, when it tries to return the python interpreter instance. I stepped through with gdb and don't see anything obvious. It seems to crash when it tries to create an interpreter to add the python script to the module search path.

Mark
From: Andy Bauer [mailto:andy.bauer at kitware.com]<mailto:[mailto:andy.bauer at kitware.com]>
Sent: Thursday, August 30, 2012 10:26 PM
To: Vanmoer, Mark W
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: Re: [Paraview] Python interpreter not being created for coprocessing?

Glad to hear that it's working. As for the parallel test failure, I think you can ignore that for now. I need to fix it but I just haven't gotten around to it yet. I think it's either a parallel rendering/compositing issue or a problem with the actors's visibility in the views getting screwed up. Both of those things are not my area of expertise so I've been reluctant to go and fix it. Now that you've publicly shamed me I'm going to have to bite the bullet and fix it :)  Hopefully by the end of next week it will be fixed. I created a mantis issue (http://paraview.org/Bug/view.php?id=13417) if you want to follow it.

Andy
On Thu, Aug 30, 2012 at 6:19 PM, Vanmoer, Mark W <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>> wrote:
Hi Andy,
Not sure what changed, but I was able to build pgi/openmpi and got to the point you did below, where the parallel test fails with "CPGrid0 image compare failed."
Can I ignore this error? Anyway, the python interpreter seems to be created, so that issue is closed.
Mark

From: Andy Bauer [mailto:andy.bauer at kitware.com<mailto:andy.bauer at kitware.com>]
Sent: Wednesday, August 29, 2012 12:15 PM

To: Vanmoer, Mark W
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: Re: [Paraview] Python interpreter not being created for coprocessing?

Ok, glad to hear that coprocessing is working in some fashion. Now to figure out how to get it running properly with your desired configuration. Unfortunately at this point I don't know how to move forward except for trying to piecemeal from having it working with GCC without everything you need to working with PGI, CUDA and everything else.

Andy
On Tue, Aug 28, 2012 at 2:51 PM, Vanmoer, Mark W <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>> wrote:
[mvanmoer at forge ParaView]$ git log origin/master | head -1
commit 2537cabbf0cc9b39f17f347105ee18c559d0041a

I should mention I do have a gcc, mvapich2 compiled version of coprocessing compiled on Forge and it works great. However any program built with PGI/openmpi code (required because of the CUDA Fortran) crashes with MPI errors. That's why I was trying to build ParaView with PGI/openmpi. I guess I could try gcc/openmpi and see if that linked to pgi/openmpi will run.

Mark


From: Andy Bauer [mailto:andy.bauer at kitware.com<mailto:andy.bauer at kitware.com>]
Sent: Tuesday, August 28, 2012 12:19 PM

To: Vanmoer, Mark W
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: Re: [Paraview] Python interpreter not being created for coprocessing?

Hmm, ParaView has been undergoing a lot of changes lately so it's been a bit unstable. I just checked out a clean version and built it without any problems. The SHA from that was 2537cabbf0cc9b39f17f347105ee18c559d0041a. What version are you on?

Have you tried building with GCC instead of PGI? I'm thinking of ways to simplify the build since I haven't tried building with PGI+CUDA+MPI+Python myself.

Andy
On Mon, Aug 27, 2012 at 3:31 PM, Vanmoer, Mark W <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>> wrote:
Hi Andy, I just did a fresh git clone and ccmake gives me
Make Error at VTK/CMake/vtkModuleTop.cmake:29 (message):
   No such module "vtkWrappingTools" needed by "vtkUtilitiesWrapClientServer"
Call Stack (most recent call first):
   VTK/CMake/vtkModuleTop.cmake:45 (vtk_module_check)
   VTK/CMake/vtkModuleTop.cmake:52 (vtk_module_check)
   VTK/CMakeLists.txt:334 (include)

Mark

From: Andy Bauer [mailto:andy.bauer at kitware.com<mailto:andy.bauer at kitware.com>]
Sent: Monday, August 27, 2012 11:59 AM

To: Vanmoer, Mark W
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: Re: [Paraview] Python interpreter not being created for coprocessing?

Hi Mark,

With the modularization of VTK and ParaView the coprocessing tests were temporarily in a bad state. The serial ones are fixed now (the parallel runs but gives incorrect results for the screenshot). Is it possible for you to update your version of ParaView and retry this?

Thanks,
Andy
On Fri, Aug 24, 2012 at 4:03 PM, Vanmoer, Mark W <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>> wrote:
Hi, I rebuilt with ParaViewData directory set and it's only finding 1 out 3 of the CoProcessing tests. Did I forget a step?

$ ctest --verbose --output-on-failure --output-log ctest.out -R CoProcess

UpdateCTestConfiguration  from :/usr/apps/vis/ParaView/repo/withTesting/DartConfiguration.tcl
Parse Config file:/usr/apps/vis/ParaView/repo/withTesting/DartConfiguration.tcl
UpdateCTestConfiguration  from :/usr/apps/vis/ParaView/repo/withTesting/DartConfiguration.tcl
Parse Config file:/usr/apps/vis/ParaView/repo/withTesting/DartConfiguration.tcl
Test project /usr/apps/vis/ParaView/repo/withTesting
Constructing a list of tests
Done constructing a list of tests
Checking test dependency graph...
Checking test dependency graph end
test 29
    Start  29: vtkCoProcessor-HeaderTest

29: Test command: /usr/bin/python "/usr/apps/vis/ParaView/repo/ParaView/VTK/Testing/Core/HeaderTesting.py" "/usr/apps/vis/ParaView/repo/ParaView/CoProcessing/Core" "VTKCOPROCESSOR_EXPORT"
29: Test timeout computed to be: 1500
29: Use export macro: VTKCOPROCESSOR_EXPORT
1/3 Test  #29: vtkCoProcessor-HeaderTest ........   Passed    0.37 sec
test 100
    Start 100: CoProcessingTestPythonScript

100: Test command: /usr/apps/vis/CMake/2.8.9/bin/cmake "-Dcfg=Release" "-P" "/usr/apps/vis/ParaView/repo/withTesting/CoProcessing/CoProcessingTestPythonScript.cmake"
100: Test timeout computed to be: 1500
100: CMake Error at /usr/apps/vis/ParaView/repo/withTesting/CoProcessing/CoProcessingTestPythonScript.cmake:6 (message):
100:
100:   '/usr/apps/vis/ParaView/repo/withTesting/ThirdParty/protobuf/vtkprotobuf/bin/Release/CoProcessingPythonScriptExample.exe'
100:   does not exist
100:
100:
2/3 Test #100: CoProcessingTestPythonScript .....***Failed    0.79 sec
CMake Error at /usr/apps/vis/ParaView/repo/withTesting/CoProcessing/CoProcessingTestPythonScript.cmake:6 (message):

  '/usr/apps/vis/ParaView/repo/withTesting/ThirdParty/protobuf/vtkprotobuf/bin/Release/CoProcessingPythonScriptExample.exe'
  does not exist



test 101
    Start 101: PCoProcessingTestPythonScript

101: Test command: /usr/apps/vis/CMake/2.8.9/bin/cmake "-Dcfg=Release" "-P" "/usr/apps/vis/ParaView/repo/withTesting/CoProcessing/PCoProcessingTestPythonScript.cmake"
101: Test timeout computed to be: 1500
101: CMake Error at /usr/apps/vis/ParaView/repo/withTesting/CoProcessing/PCoProcessingTestPythonScript.cmake:6 (message):
101:
101:   '/usr/apps/vis/ParaView/repo/withTesting/ThirdParty/protobuf/vtkprotobuf/bin/Release/CoProcessingPythonScriptExample.exe'
101:   does not exist
101:
101:
3/3 Test #101: PCoProcessingTestPythonScript ....***Failed    0.01 sec
CMake Error at /usr/apps/vis/ParaView/repo/withTesting/CoProcessing/PCoProcessingTestPythonScript.cmake:6 (message):

  '/usr/apps/vis/ParaView/repo/withTesting/ThirdParty/protobuf/vtkprotobuf/bin/Release/CoProcessingPythonScriptExample.exe'
  does not exist




The following tests passed:
                vtkCoProcessor-HeaderTest

33% tests passed, 2 tests failed out of 3

Total Test time (real) =   8.14 sec

The following tests FAILED:
                100 - CoProcessingTestPythonScript (Failed)
                101 - PCoProcessingTestPythonScript (Failed)
Errors while running CTest

Mark
From: Andy Bauer [mailto:andy.bauer at kitware.com<mailto:andy.bauer at kitware.com>]
Sent: Thursday, August 23, 2012 3:06 PM

To: Vanmoer, Mark W
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: Re: [Paraview] Python interpreter not being created for coprocessing?


On Thu, Aug 23, 2012 at 4:00 PM, Vanmoer, Mark W <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>> wrote:
Two questions - is there a way to tell ctest where the ParaViewData is without having to go back into cmake and rebuilding?

Not possible for most, if not all, tests. The reason is that the test doesn't get added unless ParaViewData is available.

And 2, is there a way to tell ctest to not use MPIEXEC_MAX_NUMPROCS when running tests?

Many parallel tests, including the parallel coprocessing test, usually just have it hard coded to use 2 or 4 processes for testing.


Mark
From: Andy Bauer [mailto:andy.bauer at kitware.com<mailto:andy.bauer at kitware.com>]
Sent: Thursday, August 23, 2012 1:32 PM

To: Vanmoer, Mark W
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: Re: [Paraview] Python interpreter not being created for coprocessing?

It looks like you've built it properly. At least I don't see anything that looks wrong. I think the reason you didn't get any coprocessing tests is that they depend on ParaViewDAta. Maybe the next step is to download ParaViewData (git://paraview.org/ParaViewData.git<http://paraview.org/ParaViewData.git>) and then set the location ParaViewData in cmake. After that, run the coprocessing tests (ctest -R CoProcess) and show me the output from that.

Andy
On Wed, Aug 22, 2012 at 12:26 PM, Vanmoer, Mark W <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>> wrote:
Hi Andy,

I set PARAVIEW_ENABLE_PYTHON to ON, is more needed? It did build pvpython, I don't know if that's separate thing entirely, though.
$ pvserver -version
paraview version 3.14.1-1549-g5f34f40

$ python -version
Python 2.6.6

$ cmake -version
cmake version 2.8.9

CMakeCache.txt is attached.

I also just noticed that ctest -R CoProcessing returns No tests were found!!!, yet it does have other tests.
Let me know what else you need.

Mark

From: Andy Bauer [mailto:andy.bauer at kitware.com<mailto:andy.bauer at kitware.com>]
Sent: Wednesday, August 22, 2012 10:49 AM
To: Vanmoer, Mark W
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: Re: [Paraview] Python interpreter not being created for coprocessing?

Hi Mark,

I think I can guess what the problem is but can't be certain without some more information. My guess is that you're using ParaView 3.14.1 for this and you're trying to use the fortran interface but haven't built ParaView with Python enabled. There were some dependency issues there that have been ironed out since then though. If that's not the case though I'll need more information on versions and maybe sending your CMakeCache.txt in your ParaView build directory.

Andy
On Wed, Aug 22, 2012 at 11:12 AM, Vanmoer, Mark W <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>> wrote:
Hello,

I'm setting up someone's code to use coprocessing with OSMesa and openmpi/pgi/cuda fortran on NCSA's Forge using ParaView from the git repo. I'm getting segfaults and pgdbg gives me this error when coprocessorinitialize is called:

[0] Signalled SIGSEGV at 0x2B62188CFD9A, function vtkCPPythonHelper::GetPythonInterpretor(), file /usr/apps/vis/ParaView/repo/ParaView/CoProcessing/CoProcessor/vtkCPPythonHelper.cxx, line 136
0x2B62188CFD9A:  48 8B 41 38            movq   56(%rcx),%rax

which is:
vtkPVPythonInterpretor* vtkCPPythonHelper::GetPythonInterpretor()
{
  return Instance->PythonInterpretor;
}

The call stack is
vtkCPPythonHelper::GetPythonInterpretor() line 136 in vtkCPPythonHelper.cxx
vtkCPPythonScriptPipeline::Initialize(const char *) line 69 in vtkCPPythonScriptPipeline.cxx
coprocessorinitialize_ line 88 in FortranAdaptorAPI.cxx


So is an Instance not getting created? I have the system python in my path.

Thanks,
Mark

_______________________________________________
Powered by www.kitware.com<http://www.kitware.com>

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView

Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20120911/93168298/attachment-0001.htm>

Previous message: [Paraview] How to visualize this 2d tecplot file?
Next message: [Paraview] Python interpreter not being created for coprocessing?
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

More information about the ParaView mailing list