[Paraview] Python interpreter not being created for coprocessing?
Andy Bauer
andy.bauer at kitware.com
Tue Sep 25 11:12:46 EDT 2012
Did you get my changes from ParaView's stage branch? My changes didn't make
it into ParaView master yet so I'm not sure what changed that caused things
to work now. Well, if it's working I'm glad -- I'm not sure I can
personally take credit for it though :)
Let me know how it goes with the GPU stuff.
Andy
On Mon, Sep 24, 2012 at 5:48 PM, Vanmoer, Mark W <mvanmoer at illinois.edu>wrote:
> Hi Andy,****
>
> ** **
>
> I’ve tested this and it works with both gcc/mpich2 and pgi/openmpi. ****
>
> ** **
>
> Thanks again!****
>
> Mark****
>
> ** **
>
> *From:* Vanmoer, Mark W
> *Sent:* Wednesday, September 19, 2012 1:30 PM
> *To:* Andy Bauer
> *Cc:* paraview at paraview.org
> *Subject:* RE: [Paraview] Python interpreter not being created for
> coprocessing?****
>
> ** **
>
> Hi Andy, that sounds like some deep debugging, thanks for your effort.
> I'll test this and let you know. It's on my to do list to learn CMake, I've
> had the book on my desk for a month now.
>
> Also, I made another Fortran example that's closer in spirit to the wiki
> example, which uses the DataGenerator class as the C++ does, if you're
> interested.
>
> Thanks,
> Mark****
> ------------------------------
>
> *From: *Andy Bauer
> *Sent: *9/19/2012 1:12 PM
> *To: *Vanmoer, Mark W
> *Cc: *paraview at paraview.org
> *Subject: *Re: [Paraview] Python interpreter not being created for
> coprocessing?****
>
> Hi Mark,
>
> I finally got around to fixing this. It's in the stage branch
> (remotes/stage/13460_CP_FortranAdaptor) and hopefully will go into the
> master branch on the next gatekeeper review. There were 2 issues fixed in
> this:
> 1) for using newer GCC compilers and other compilers that hide symbols in
> the library (http://gcc.gnu.org/wiki/Visibility) the FortranAdaptor API
> methods weren't available to other libraries and executables trying to link
> to them.
> 2) Since 3.14.1 I changed around the way ParaView gets initialized through
> the coprocessing library (now through vtkCPProcessor::Initialize() as it
> should be instead of vtkCPPythonScriptPipeline::Initialize()). I forgot to
> make the change in the FortranAdaptorAPI.cxx which caused the python
> interpretor to not be initialized properly, as you noticed.
>
> Thanks for the help in finding this problem. Sorry for the slow response
> on it. I created a CMakeLists.txt file to help build as I've been using
> CMake too long and can't remember how to work with Makefiles anymore! I'm
> attaching it in case you'd like to use it. I didn't test it with an install
> of ParaView as I really only work directly with ParaView builds.
>
> Let me know if this solves your problems. I seem to remember there being
> more as far as using CUDA and PGI but hopefully this gets you close.
>
> By the way, I'll probably use what you sent me to create a Fortran test or
> example to help with making sure it doesn't get broken in the future.
>
> Andy****
>
> On Wed, Sep 12, 2012 at 4:42 PM, Vanmoer, Mark W <mvanmoer at illinois.edu>
> wrote:****
>
> Tarball attached, it’s not exactly the wiki example because I didn’t want
> to bother with passing a DataGenerator object back and forth from C++ to
> Fortran, so I just make a sphere. The pipeline script is a shrink filter
> generated by pvclient.****
>
> ****
>
> I checked CMake and it looks like it’s finding the same version of python.
> ****
>
> Mark****
>
> ****
>
> *From:* Andy Bauer [mailto:andy.bauer at kitware.com]
> *Sent:* Wednesday, September 12, 2012 1:02 PM****
>
>
> *To:* Vanmoer, Mark W
> *Cc:* paraview at paraview.org
> *Subject:* Re: [Paraview] Python interpreter not being created for
> coprocessing?****
>
> ****
>
> Can you send me your Fortran version of the wiki example?
>
> From what I remember, CMake treats the Python interpretor/executable
> different than the header files and libraries. By this I mean that it will
> pick up the python header files and libraries from one version of python
> and the python interpretor will be from a different version. I don't think
> that's what's going on but you may want to check on that in CMake.
>
> Andy****
>
> On Tue, Sep 11, 2012 at 11:21 AM, Vanmoer, Mark W <mvanmoer at illinois.edu>
> wrote:****
>
> Hi Andy,****
>
> ****
>
> Sorry, I lost track of where I was in debugging. What I had was an
> openmpi/pgi debug build that passed the ctest -R CoPro tests. But the app
> I'm working on still crashes when trying to create a python interpreter.**
> **
>
> ****
>
> I started over from scratch working with the Wiki example, using gcc 4.4.6
> and mpich2 1.4.1p1.****
>
> ****
>
> I have the C++ wiki example working with both the ParaView-3.14.1-Source
> tarball and with git commit 2537cabbf0cc9b39f17f347105ee18c559d0041a.****
>
> ****
>
> I then made a Fortran version of the wiki example. It works with the
> ParaView-3.14.1-Source tarball build, but crashes when built against the
> git branch build. Seems to be at the same spot as the pgi build, when it
> tries to return the python interpreter instance. I stepped through with gdb
> and don’t see anything obvious. It seems to crash when it tries to create
> an interpreter to add the python script to the module search path.****
>
> ****
>
> Mark****
>
> *From:* Andy Bauer [mailto:andy.bauer at kitware.com]
> *Sent:* Thursday, August 30, 2012 10:26 PM
> *To:* Vanmoer, Mark W
> *Cc:* paraview at paraview.org
> *Subject:* Re: [Paraview] Python interpreter not being created for
> coprocessing?****
>
> ****
>
> Glad to hear that it's working. As for the parallel test failure, I think
> you can ignore that for now. I need to fix it but I just haven't gotten
> around to it yet. I think it's either a parallel rendering/compositing
> issue or a problem with the actors's visibility in the views getting
> screwed up. Both of those things are not my area of expertise so I've been
> reluctant to go and fix it. Now that you've publicly shamed me I'm going to
> have to bite the bullet and fix it :) Hopefully by the end of next week it
> will be fixed. I created a mantis issue (
> http://paraview.org/Bug/view.php?id=13417) if you want to follow it.
>
> Andy****
>
> On Thu, Aug 30, 2012 at 6:19 PM, Vanmoer, Mark W <mvanmoer at illinois.edu>
> wrote:****
>
> Hi Andy, ****
>
> Not sure what changed, but I was able to build pgi/openmpi and got to the
> point you did below, where the parallel test fails with “CPGrid0 image
> compare failed.” ****
>
> Can I ignore this error? Anyway, the python interpreter seems to be
> created, so that issue is closed.****
>
> Mark****
>
> ****
>
> *From:* Andy Bauer [mailto:andy.bauer at kitware.com]
> *Sent:* Wednesday, August 29, 2012 12:15 PM****
>
>
> *To:* Vanmoer, Mark W
> *Cc:* paraview at paraview.org
> *Subject:* Re: [Paraview] Python interpreter not being created for
> coprocessing?****
>
> ****
>
> Ok, glad to hear that coprocessing is working in some fashion. Now to
> figure out how to get it running properly with your desired configuration.
> Unfortunately at this point I don't know how to move forward except for
> trying to piecemeal from having it working with GCC without everything you
> need to working with PGI, CUDA and everything else.
>
> Andy****
>
> On Tue, Aug 28, 2012 at 2:51 PM, Vanmoer, Mark W <mvanmoer at illinois.edu>
> wrote:****
>
> [mvanmoer at forge ParaView]$ git log origin/master | head -1****
>
> commit 2537cabbf0cc9b39f17f347105ee18c559d0041a****
>
> ****
>
> I should mention I do have a gcc, mvapich2 compiled version of
> coprocessing compiled on Forge and it works great. However any program
> built with PGI/openmpi code (required because of the CUDA Fortran) crashes
> with MPI errors. That’s why I was trying to build ParaView with
> PGI/openmpi. I guess I could try gcc/openmpi and see if that linked to
> pgi/openmpi will run. ****
>
> ****
>
> Mark****
>
> ****
>
> ****
>
> *From:* Andy Bauer [mailto:andy.bauer at kitware.com]
> *Sent:* Tuesday, August 28, 2012 12:19 PM****
>
>
> *To:* Vanmoer, Mark W
> *Cc:* paraview at paraview.org
> *Subject:* Re: [Paraview] Python interpreter not being created for
> coprocessing?****
>
> ****
>
> Hmm, ParaView has been undergoing a lot of changes lately so it's been a
> bit unstable. I just checked out a clean version and built it without any
> problems. The SHA from that was 2537cabbf0cc9b39f17f347105ee18c559d0041a.
> What version are you on?
>
> Have you tried building with GCC instead of PGI? I'm thinking of ways to
> simplify the build since I haven't tried building with PGI+CUDA+MPI+Python
> myself.
>
> Andy****
>
> On Mon, Aug 27, 2012 at 3:31 PM, Vanmoer, Mark W <mvanmoer at illinois.edu>
> wrote:****
>
> Hi Andy, I just did a fresh git clone and ccmake gives me****
>
> Make Error at VTK/CMake/vtkModuleTop.cmake:29 (message):****
>
> No such module "vtkWrappingTools" needed by
> "vtkUtilitiesWrapClientServer"****
>
> Call Stack (most recent call first):****
>
> VTK/CMake/vtkModuleTop.cmake:45 (vtk_module_check)****
>
> VTK/CMake/vtkModuleTop.cmake:52 (vtk_module_check)****
>
> VTK/CMakeLists.txt:334 (include)****
>
> ****
>
> Mark****
>
> ****
>
> *From:* Andy Bauer [mailto:andy.bauer at kitware.com]
> *Sent:* Monday, August 27, 2012 11:59 AM****
>
>
> *To:* Vanmoer, Mark W
> *Cc:* paraview at paraview.org
> *Subject:* Re: [Paraview] Python interpreter not being created for
> coprocessing?****
>
> ****
>
> Hi Mark,
>
> With the modularization of VTK and ParaView the coprocessing tests were
> temporarily in a bad state. The serial ones are fixed now (the parallel
> runs but gives incorrect results for the screenshot). Is it possible for
> you to update your version of ParaView and retry this?
>
> Thanks,
> Andy****
>
> On Fri, Aug 24, 2012 at 4:03 PM, Vanmoer, Mark W <mvanmoer at illinois.edu>
> wrote:****
>
> Hi, I rebuilt with ParaViewData directory set and it’s only finding 1 out
> 3 of the CoProcessing tests. Did I forget a step?****
>
> ****
>
> $ ctest --verbose --output-on-failure --output-log ctest.out -R CoProcess*
> ***
>
> ****
>
> UpdateCTestConfiguration from
> :/usr/apps/vis/ParaView/repo/withTesting/DartConfiguration.tcl****
>
> Parse Config
> file:/usr/apps/vis/ParaView/repo/withTesting/DartConfiguration.tcl****
>
> UpdateCTestConfiguration from
> :/usr/apps/vis/ParaView/repo/withTesting/DartConfiguration.tcl****
>
> Parse Config
> file:/usr/apps/vis/ParaView/repo/withTesting/DartConfiguration.tcl****
>
> Test project /usr/apps/vis/ParaView/repo/withTesting****
>
> Constructing a list of tests****
>
> Done constructing a list of tests****
>
> Checking test dependency graph...****
>
> Checking test dependency graph end****
>
> test 29****
>
> Start 29: vtkCoProcessor-HeaderTest****
>
> ****
>
> 29: Test command: /usr/bin/python
> "/usr/apps/vis/ParaView/repo/ParaView/VTK/Testing/Core/HeaderTesting.py"
> "/usr/apps/vis/ParaView/repo/ParaView/CoProcessing/Core"
> "VTKCOPROCESSOR_EXPORT"****
>
> 29: Test timeout computed to be: 1500****
>
> 29: Use export macro: VTKCOPROCESSOR_EXPORT****
>
> 1/3 Test #29: vtkCoProcessor-HeaderTest ........ Passed 0.37 sec****
>
> test 100****
>
> Start 100: CoProcessingTestPythonScript****
>
> ****
>
> 100: Test command: /usr/apps/vis/CMake/2.8.9/bin/cmake "-Dcfg=Release"
> "-P"
> "/usr/apps/vis/ParaView/repo/withTesting/CoProcessing/CoProcessingTestPythonScript.cmake"
> ****
>
> 100: Test timeout computed to be: 1500****
>
> 100: CMake Error at
> /usr/apps/vis/ParaView/repo/withTesting/CoProcessing/CoProcessingTestPythonScript.cmake:6
> (message):****
>
> 100: ****
>
> 100:
> '/usr/apps/vis/ParaView/repo/withTesting/ThirdParty/protobuf/vtkprotobuf/bin/Release/CoProcessingPythonScriptExample.exe'
> ****
>
> 100: does not exist****
>
> 100: ****
>
> 100: ****
>
> 2/3 Test #100: CoProcessingTestPythonScript .....***Failed 0.79 sec****
>
> CMake Error at
> /usr/apps/vis/ParaView/repo/withTesting/CoProcessing/CoProcessingTestPythonScript.cmake:6
> (message):****
>
> ****
>
>
> '/usr/apps/vis/ParaView/repo/withTesting/ThirdParty/protobuf/vtkprotobuf/bin/Release/CoProcessingPythonScriptExample.exe'
> ****
>
> does not exist****
>
> ****
>
> ****
>
> ****
>
> test 101****
>
> Start 101: PCoProcessingTestPythonScript****
>
> ****
>
> 101: Test command: /usr/apps/vis/CMake/2.8.9/bin/cmake "-Dcfg=Release"
> "-P"
> "/usr/apps/vis/ParaView/repo/withTesting/CoProcessing/PCoProcessingTestPythonScript.cmake"
> ****
>
> 101: Test timeout computed to be: 1500****
>
> 101: CMake Error at
> /usr/apps/vis/ParaView/repo/withTesting/CoProcessing/PCoProcessingTestPythonScript.cmake:6
> (message):****
>
> 101: ****
>
> 101:
> '/usr/apps/vis/ParaView/repo/withTesting/ThirdParty/protobuf/vtkprotobuf/bin/Release/CoProcessingPythonScriptExample.exe'
> ****
>
> 101: does not exist****
>
> 101: ****
>
> 101: ****
>
> 3/3 Test #101: PCoProcessingTestPythonScript ....***Failed 0.01 sec****
>
> CMake Error at
> /usr/apps/vis/ParaView/repo/withTesting/CoProcessing/PCoProcessingTestPythonScript.cmake:6
> (message):****
>
> ****
>
>
> '/usr/apps/vis/ParaView/repo/withTesting/ThirdParty/protobuf/vtkprotobuf/bin/Release/CoProcessingPythonScriptExample.exe'
> ****
>
> does not exist****
>
> ****
>
> ****
>
> ****
>
> ****
>
> The following tests passed:****
>
> vtkCoProcessor-HeaderTest****
>
> ****
>
> 33% tests passed, 2 tests failed out of 3****
>
> ****
>
> Total Test time (real) = 8.14 sec****
>
> ****
>
> The following tests FAILED:****
>
> 100 - CoProcessingTestPythonScript (Failed)****
>
> 101 - PCoProcessingTestPythonScript (Failed)****
>
> Errors while running CTest****
>
> ****
>
> Mark****
>
> *From:* Andy Bauer [mailto:andy.bauer at kitware.com]
> *Sent:* Thursday, August 23, 2012 3:06 PM****
>
>
> *To:* Vanmoer, Mark W
> *Cc:* paraview at paraview.org
> *Subject:* Re: [Paraview] Python interpreter not being created for
> coprocessing?****
>
> ****
>
> ****
>
> On Thu, Aug 23, 2012 at 4:00 PM, Vanmoer, Mark W <mvanmoer at illinois.edu>
> wrote:****
>
> Two questions – is there a way to tell ctest where the ParaViewData is
> without having to go back into cmake and rebuilding?****
>
>
> Not possible for most, if not all, tests. The reason is that the test
> doesn't get added unless ParaViewData is available.
> ****
>
> And 2, is there a way to tell ctest to not use MPIEXEC_MAX_NUMPROCS when
> running tests?****
>
>
> Many parallel tests, including the parallel coprocessing test, usually
> just have it hard coded to use 2 or 4 processes for testing.
> ****
>
> ****
>
> Mark****
>
> *From:* Andy Bauer [mailto:andy.bauer at kitware.com]
> *Sent:* Thursday, August 23, 2012 1:32 PM****
>
>
> *To:* Vanmoer, Mark W
> *Cc:* paraview at paraview.org
> *Subject:* Re: [Paraview] Python interpreter not being created for
> coprocessing?****
>
> ****
>
> It looks like you've built it properly. At least I don't see anything that
> looks wrong. I think the reason you didn't get any coprocessing tests is
> that they depend on ParaViewDAta. Maybe the next step is to download
> ParaViewData (git://paraview.org/ParaViewData.git) and then set the
> location ParaViewData in cmake. After that, run the coprocessing tests
> (ctest -R CoProcess) and show me the output from that.
>
> Andy****
>
> On Wed, Aug 22, 2012 at 12:26 PM, Vanmoer, Mark W <mvanmoer at illinois.edu>
> wrote:****
>
> Hi Andy,****
>
> ****
>
> I set PARAVIEW_ENABLE_PYTHON to ON, is more needed? It did build pvpython,
> I don’t know if that’s separate thing entirely, though.****
>
> $ pvserver –version****
>
> paraview version 3.14.1-1549-g5f34f40****
>
> ****
>
> $ python –version****
>
> Python 2.6.6****
>
> ****
>
> $ cmake –version****
>
> cmake version 2.8.9****
>
> ****
>
> CMakeCache.txt is attached.****
>
> ****
>
> I also just noticed that ctest –R CoProcessing returns No tests were
> found!!!, yet it does have other tests.****
>
> Let me know what else you need.****
>
> ****
>
> Mark****
>
> ****
>
> *From:* Andy Bauer [mailto:andy.bauer at kitware.com]
> *Sent:* Wednesday, August 22, 2012 10:49 AM
> *To:* Vanmoer, Mark W
> *Cc:* paraview at paraview.org
> *Subject:* Re: [Paraview] Python interpreter not being created for
> coprocessing?****
>
> ****
>
> Hi Mark,
>
> I think I can guess what the problem is but can't be certain without some
> more information. My guess is that you're using ParaView 3.14.1 for this
> and you're trying to use the fortran interface but haven't built ParaView
> with Python enabled. There were some dependency issues there that have been
> ironed out since then though. If that's not the case though I'll need more
> information on versions and maybe sending your CMakeCache.txt in your
> ParaView build directory.
>
> Andy****
>
> On Wed, Aug 22, 2012 at 11:12 AM, Vanmoer, Mark W <mvanmoer at illinois.edu>
> wrote:****
>
> Hello,****
>
> ****
>
> I'm setting up someone's code to use coprocessing with OSMesa and
> openmpi/pgi/cuda fortran on NCSA's Forge using ParaView from the git repo.
> I'm getting segfaults and pgdbg gives me this error when
> coprocessorinitialize is called:****
>
> ****
>
> [0] Signalled SIGSEGV at 0x2B62188CFD9A, function
> vtkCPPythonHelper::GetPythonInterpretor(), file
> /usr/apps/vis/ParaView/repo/ParaView/CoProcessing/CoProcessor/vtkCPPythonHelper.cxx,
> line 136****
>
> 0x2B62188CFD9A: 48 8B 41 38 movq 56(%rcx),%rax****
>
> ****
>
> which is:****
>
> vtkPVPythonInterpretor* vtkCPPythonHelper::GetPythonInterpretor()****
>
> {****
>
> return Instance->PythonInterpretor;****
>
> }****
>
> ****
>
> The call stack is****
>
> vtkCPPythonHelper::GetPythonInterpretor() line 136 in vtkCPPythonHelper.cxx
> ****
>
> vtkCPPythonScriptPipeline::Initialize(const char *) line 69 in
> vtkCPPythonScriptPipeline.cxx****
>
> coprocessorinitialize_ line 88 in FortranAdaptorAPI.cxx****
>
> ****
>
> ****
>
> So is an Instance not getting created? I have the system python in my path.
> ****
>
> ****
>
> Thanks,****
>
> Mark****
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at:
> http://paraview.org/Wiki/ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://www.paraview.org/mailman/listinfo/paraview****
>
> ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> ** **
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20120925/88664fb8/attachment-0001.htm>
More information about the ParaView
mailing list