[vtkusers] [vtk-developers] problems with VTK python module installation on OSX (after a custom build)

[Darren Weber]

On Wed, Sep 3, 2008 at 3:28 PM, Mike Jackson
<mike.jackson at bluequartz.net> wrote:
> Cross posting to vtkusers list as that may be more appropriate:
>
>
> Can you rebuild VTK with these two settings:
>
> CMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=ON \

Where is this?  I don't see it in ccmake.  How do you set it?

> VTK_USE_RPATH=ON

This I can see in ccmake

>
> Run ccmake on your vtk build directory and set those two cmake variables.
> Configure and build. After the build completes, then run "make install".
>
> The default installation location is in /usr/local. I would change it to
> /usr/local/vtk-5.2.
>
> Next, Get rid of extra Python builds on your system. The only python build
> should be in /System/Library/Frameworks.
>
> Now try to build your project. If everything goes correctly you should NOT
> need to mess with DYLD_LIBRARY_PATH. The reason why you are having to set
> that variable is because, as you found out, the vtk libraries do NOT have an
> "install_path" (same as rpath on linux) set in them. Without that set to
> anything besides the library name the linker will not be able to encode the
> correct path in anything else that depends on the vtk libraries. Which is
> some of your problems.
>
> Let us know if you can get this far.
>
> Mike
> On Sep 3, 2008, at 2:19 PM, Darren Weber wrote:
>
>> I think we may be on the "right" track with addition(s) to the DYLD path,
>> ie:
>>
>> [ dweber at elegans ~/src/kitware/VTK_build ]$ echo $DYLD_LIBRARY_PATH
>>
>> [ dweber at elegans ~/src/kitware/VTK_build ]$ export
>> DYLD_LIBRARY_PATH=/usr/local/lib/vtk-5.2
>> [ dweber at elegans ~/src/kitware/VTK_build ]$ echo $DYLD_LIBRARY_PATH
>> /usr/local/lib/vtk-5.2
>> [ dweber at elegans ~/src/kitware/VTK_build ]$ python
>> Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53)
>> [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>
>>>>> import vtk
>>
>> Traceback (most recent call last):
>>  File "<stdin>", line 1, in <module>
>>  File "/usr/local/lib/python2.5/site-packages/vtk/__init__.py", line
>> 43, in <module>
>>   from io import *
>>  File "/usr/local/lib/python2.5/site-packages/vtk/io.py", line 7, in
>> <module>
>>   from libvtkIOPython import *
>> ImportError:
>> dlopen(/usr/local/lib/python2.5/site-packages/vtk/libvtkIOPython.so,
>> 10): Library not loaded: libpq.5.dylib
>>  Referenced from:
>> /usr/local/lib/python2.5/site-packages/vtk/libvtkIOPython.so
>>  Reason: image not found
>>>>>
>>
>> This is another failure, but this time it is related to the PostgreSQL
>> library that resides in another location (under /Library path).  So, I
>> conclude from this test that dlopen is now finding the vtk dylib
>> files.
>>
>> Now, we can get the following "diagnostics" on the .so from the prior
>> import problem:
>>
>> [ dweber at elegans ~/src/kitware/VTK_build ]$ otool -L
>> /usr/local/lib/python2.5/site-packages/vtk/libvtkCommonPython.so
>> /usr/local/lib/python2.5/site-packages/vtk/libvtkCommonPython.so:
>>        libvtkCommonPythonD.5.2.dylib (compatibility version 0.0.0, current
>> version 0.0.0)
>>        /Library/Frameworks/Python.framework/Versions/2.5/Python
>> (compatibility version 2.5.0, current version 2.5.0)
>>        libvtkCommon.5.2.dylib (compatibility version 0.0.0, current
>> version 0.0.0)
>>        libvtksys.5.2.dylib (compatibility version 0.0.0, current version
>> 0.0.0)
>>        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
>> version 111.1.1)
>>        /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current
>> version 7.4.0)
>>        /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current
>> version 1.0.0)
>> [ dweber at elegans ~/src/kitware/VTK_build ]$
>>
>>
>> Notice how some of the system libaries have full paths (ie,
>> /usr/lib/....), but the vtk dylib files have no paths.  Hence, when we
>> specify the DYLD_LIBRARY_PATH, the python import process is now
>> finding these extra libraries because dlopen is now searching at least
>> the /usr/local/lib/vtk... path (recursively? maybe not, because it
>> doesn't work if we use /usr/local/lib without the specific vtk path).
>>
>> Similarly, now we can see the full library dependency for the
>> IOPython.so file using:
>>
>> [ dweber at elegans ~/src/kitware/VTK_build ]$ otool -L
>> /usr/local/lib/python2.5/site-packages/vtk/libvtkIOPython.so
>> /usr/local/lib/python2.5/site-packages/vtk/libvtkIOPython.so:
>>        libvtkIOPythonD.5.2.dylib (compatibility version 0.0.0, current
>> version 0.0.0)
>>        libvtkIO.5.2.dylib (compatibility version 0.0.0, current version
>> 0.0.0)
>>        /usr/local/lib/libpqxx-2.6.9.dylib (compatibility version 0.0.0,
>> current version 0.0.0)
>>        libpq.5.dylib (compatibility version 5.0.0, current version 5.1.0)
>>        /usr/local/mysql/lib/libmysqlclient.15.dylib (compatibility version
>> 16.0.0, current version 16.0.0)
>>        /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version
>> 1.2.3)
>>        libvtkFilteringPythonD.5.2.dylib (compatibility version 0.0.0,
>> current version 0.0.0)
>>        libvtkCommonPythonD.5.2.dylib (compatibility version 0.0.0, current
>> version 0.0.0)
>>        /Library/Frameworks/Python.framework/Versions/2.5/Python
>> (compatibility version 2.5.0, current version 2.5.0)
>>        libvtkFiltering.5.2.dylib (compatibility version 0.0.0, current
>> version 0.0.0)
>>        libvtkDICOMParser.5.2.dylib (compatibility version 0.0.0, current
>> version 0.0.0)
>>        libvtkNetCDF.5.2.dylib (compatibility version 0.0.0, current
>> version 0.0.0)
>>        libvtkmetaio.5.2.dylib (compatibility version 0.0.0, current
>> version 0.0.0)
>>        libvtksqlite.5.2.dylib (compatibility version 0.0.0, current
>> version 0.0.0)
>>        libvtkpng.5.2.dylib (compatibility version 0.0.0, current version
>> 0.0.0)
>>        libvtktiff.5.2.dylib (compatibility version 0.0.0, current version
>> 0.0.0)
>>        libvtkzlib.5.2.dylib (compatibility version 0.0.0, current version
>> 0.0.0)
>>        libvtkjpeg.5.2.dylib (compatibility version 0.0.0, current version
>> 0.0.0)
>>        libvtkexpat.5.2.dylib (compatibility version 0.0.0, current version
>> 0.0.0)
>>        libvtkCommon.5.2.dylib (compatibility version 0.0.0, current
>> version 0.0.0)
>>        libvtksys.5.2.dylib (compatibility version 0.0.0, current version
>> 0.0.0)
>>        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
>> version 111.1.1)
>>        /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current
>> version 7.4.0)
>>        /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current
>> version 1.0.0)
>> [ dweber at elegans ~/src/kitware/VTK_build ]$
>>
>>
>> It's curious that it has a full path to /usr/local/lib/libpqxx (which
>> I had to build and install on this system), but it has no full path to
>> the associated libpq.  I don't fully understand this link dependency,
>> as I thought libpqxx would be the primary link target and, in turn, it
>> would link to libpq. (This postgreSQL stuff is another topic on the
>> list somewhere.)
>>
>> It's also curious that 'locate' can find the libpq, but dlopen does
>> not.  So, something about the locate environment of default settings
>> is different from the dlopen environment, eg:
>>
>> {{{
>> [ dweber at elegans ~/src/kitware/VTK_build ]$ locate libpq.5.dylib
>> /Library/PostgreSQL/8.3/lib/libpq.5.dylib
>> /Library/PostgreSQL/8.3/pgAdmin3.app/Contents/Frameworks/libpq.5.dylib
>> [ dweber at elegans ~/src/kitware/VTK_build ]$
>> }}}
>>
>> So, now we modify the DYLD path to find libpq and WHA-LA, the import is
>> working!
>>
>> {{{
>> [ dweber at elegans ~/src/kitware/VTK_build ]$ export
>> DYLD_LIBRARY_PATH=/usr/local/lib/vtk-5.2:/Library/PostgreSQL/8.3/lib
>> [ dweber at elegans ~/src/kitware/VTK_build ]$ echo $DYLD_LIBRARY_PATH
>> /usr/local/lib/vtk-5.2:/Library/PostgreSQL/8.3/lib
>> [ dweber at elegans ~/src/kitware/VTK_build ]$ python
>> Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53)
>> [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>
>>>>> import vtk
>>>>>
>> }}}
>>
>>
>> I'll have to double check the default settings on DYLD path for the
>> system.  This is A fix on this particular system, so the problem
>> remains for any general OSX distribution to fix this in an elegant
>> manner.  What would you recommend for a system-wide fix to these path
>> settings and how would you implement this within cmake?  (Again, I
>> need to RTFM, it's overdue.  I will persist with OSX.)
>>
>> Thanks heaps for getting me back on track with OSX!  Power to the people
>> ;-)
>>
>> Darren
>>
>>
>> [PS, I suppose I could remove the /Library/Frameworks/Python.framework
>> installs to fall back on the /System/.... python installation, but it
>> probably makes little difference apart from disk space.  I know it's
>> not wise to touch the /System area, so that's off limits.  I guess the
>> only advantage to the /Library installs from python.org is to keep up
>> with patches and new releases.  Actually I can't decide on whether to
>> use the binary packages direct from python.org or the macports /opt
>> installation, the latter seems to provide an option for ipython and
>> that would be a real bonus.  The macports install may be as close as I
>> need to get to a source build for python.  I've opted to use macports
>> rather than fink, but that may be a mistake too ;-)  I do like the
>> Debian package system, which is the basis for fink.  I just don't know
>> whether fink is keeping up with things the way that Ubuntu does; it
>> takes the best of Debian on popular platforms.  I've opted for
>> macports only because I know they will be specific to the OSX way of
>> doing things - but why did they choose /opt/ instead of the default
>> /usr or /usr/local path?  Gheez.]
>>
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Sep 3, 2008 at 10:16 AM, Prabhu Ramachandran
>> <prabhu at aero.iitb.ac.in> wrote:
>>>
>>> Darren Weber wrote:
>>>>
>>>> I've discovered a minor problem with the VTK python installation (from
>>>> build) on OSX.  I'm basically having library path issues and I don't
>>>> know
>>>> how to bend VTK build-install to fit OSX python or how to bend OSX
>>>> python to
>>>> use the default VTK install paths.
>>>
>>> [...]
>>>>
>>>> However, despite my attempt to get the module installed in the "right"
>>>> place, I get this traceback during the import:
>>>>
>>>> dweber at weber-mbp ~/src/kitware/VTK_build ]$ which python
>>>
>>> [...]
>>>>
>>>> ImportError:
>>>>
>>>> dlopen(/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/vtk/libvtkCommonPython.so,
>>>> 10): Library not loaded: libvtkCommonPythonD.5.2.dylib
>>>> Referenced from:
>>>>
>>>> /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/vtk/libvtkCommonPython.so
>>>> Reason: image not found
>>>>>>>
>>>>
>>>>
>>>> This is where the libvtkCommon* libs are at:
>>>>
>>>> /usr/local/lib/python2.5/site-packages/vtk/libvtkCommonPython.so
>>>> /usr/local/lib/vtk-5.2/libvtkCommonPythonD.5.2.0.dylib
>>>> /usr/local/lib/vtk-5.2/libvtkCommonPythonD.5.2.dylib
>>>> /usr/local/lib/vtk-5.2/libvtkCommonPythonD.dylib
>>>
>>> Try this:
>>>
>>> export DYLD_LIBRARY_PATH=/usr/local/lib/vtk-5.2
>>>
>>> Then rerun Python and see if import vtk works at all.
>>>
>>> cheers,
>>> prabhu
>>>
>>
>
>  _________________________________________________
> | Mike Jackson - Principal Software Engineer      |
> | BlueQuartz Software                             |
> | mike.jackson at bluequartz.net                     |
> | www.bluequartz.net                              |
> ---------------------------------------------------
>
>
>
>
>
>
>
>