[vtkusers] python (2.7) problems compiling with MSVC 2010 64bit

David Gobbi david.gobbi at gmail.com
Wed Jun 8 18:10:49 EDT 2011 

I've distributed python apps with a bundled, self-contained python,
the way to do this on Windows similar to what you have described:
find all the dependencies, and copy them into the python directory.
That way the PATH and PYTHONPATH never have to be set.

On Linux and Mac copying the VTK libraries into the directory isn't
enough to ensure they will be found, but it is possible (not easy!) to
add a loader_path to the binaries and libraries.  It has been a while
since I have done this so I forget most of the details, and it is also
highly platform-dependent.  An easier, uglier solution is to deliver a
script that the users can run to set the LD_LIBRARY_PATH.

 - David


On Wed, Jun 8, 2011 at 3:50 PM, Dominik Szczerba <dominik at itis.ethz.ch> wrote:
> Hi David,
>
> Thanks a lot for your comments.
>
> You are right that my setup is far from the beaten track. I need to
> compile python myself (to consistently compile other packages against
> it with the same compiler/settings) but there is no installation
> option available for Windows, or is hidden so deep that I could not
> find it. I believe your argumentation does not cover two originally
> mentioned issues 1) portable self contained precompiled python+vtk
> environments, e.g. to be distributed to students or customers who will
> not want to hear about PATH or PYTHONPATH 2) vtk DLLs depending on
> externals like DLL, where vtk+python would fail out of the box (note
> that Qt is normally built in place and not registered with Windows).
>
> Regards,
> Dominik
>
> On Wed, Jun 8, 2011 at 11:24 PM, David Gobbi <david.gobbi at gmail.com> wrote:
>> Hi Dominik,
>>
>> I disagree with the "assemble the build into a packaged folder" part
>> of your discussion, because I don't think people should be doing this.
>> On Linux you can specify a prefix with "./configure --prefix=/some/directory"
>> and do a simple make install, the result will be a typical directory
>> layout that is suitable for system-wide use.  The python install
>> automatically segregates the binaries and libraries for different python
>> versions, so multiple versions of python can easily be installed this way.
>>
>> Ditto for Windows, the default install puts each python in its own
>> directory e.g. Python26, Python27, etc. so there are no collisions.
>> Making python accessible system-wide is as simple as installing it
>> where it is accessible to all users, this can be on a network drive
>> since python does not need any registry entries in order to work
>> properly.
>>
>> Even if you want to have one central network-wide installation that
>> can be used by different people running on different platforms, this
>> can be done in standard ways through ./configure.
>>
>> The bottom line is this: if you customize the directory structure of
>> your python installation then you have to set up the PATHs yourself
>> and/or copy the VTK dlls into the appropriate directories.
>>
>>  - David
>>
>>
>> On Wed, Jun 8, 2011 at 2:35 PM, Dominik Szczerba <dominik at itis.ethz.ch> wrote:
>>> On Wed, Jun 8, 2011 at 10:07 PM, David Gobbi <david.gobbi at gmail.com> wrote:
>>>> Hi Dominik,
>>>>
>>>> I'm a bit confused about the paths you give in your examples:
>>>>
>>>> C:\python2.7.1\bin
>>>> /usr/local/pack/python-2.7.1/bin
>>>>
>>>> On my Windows system (using the installer from python.org) I'd see
>>>> something like this:
>>>>
>>>> C:\Python27\   (no bin directory!)
>>>> C:\Python27\DLLs\
>>>> C:\Python27\Lib\site-packages\
>>>
>>> What I called a system installed python was not immediately meant to
>>> be the "official pre-compiled python". I actually have built one
>>> myself and "installed" it by making it accessible by any user. It has
>>> bin, include and lib subfolders, but yes, one can assemble it in any
>>> other way one likes. I do not in fact know the actual structure of the
>>> "official pre-compiled" installer.
>>>
>>>> On my Linux systems (ubuntu 8.04 and 10.04) after doing a typical
>>>> install, I'd see something like this:
>>>>
>>>> /usr/local/bin/python2.7 (the executable)
>>>> /usr/local/lib/python2.7/site-packages/
>>>>
>>>> Can you give some details about your system and how you have done
>>>> your python installation so that I can understand why your paths are so
>>>> different?
>>>
>>> What I do on linux is exactly the same as what I described above for
>>> Windows: I compile it myself and package myself - I did not mean
>>> calling "make install" on the custom compiled code.
>>>
>>> Bottom line: should another caption be added as 1) "Using official
>>> installers", where you would correct the paths; and the current first
>>> caption be renamed to "using custom system wide installation"? You may
>>> also opt to remove te current first part and replace it by the one
>>> related to the official installers. Hopefully the current second part
>>> "system independent installation" part, should not raise any concerns,
>>> and this one is most relevant for me.
>>>
>>> Regards,
>>> Dominik
>>>
>>>>
>>>>  - David
>>>>
>>>>
>>>> On Wed, Jun 8, 2011 at 1:19 PM, Dominik Szczerba <dominik at itis.ethz.ch> wrote:
>>>>> Hi David,
>>>>>
>>>>> I tried editing http://www.vtk.org/Wiki/VTK/Python_Wrapping_FAQ but
>>>>> was denied the editing rights. Below my proposed addition (as bullet
>>>>> 1.6) which I would have found useful when resolving the issue.
>>>>>
>>>>> Regards,
>>>>> Dominik
>>>>>
>>>>> === Installation ===
>>>>>
>>>>> In order to use the python enabled VTK you need to resolve the
>>>>> required DLL dependencies, i.e., you have to make sure all DLLs/shared
>>>>> objects can find each other. For simple setups "vtkpython" might work,
>>>>> but in case you compiled VTK with external dependencies (e.g. QT)
>>>>> chances are it will not, as there is no easy way such dependencies be
>>>>> resolved automatically. There are two scenarios covered here.
>>>>>
>>>>> ==== System installed python ====
>>>>>
>>>>> On Windows, you need to make sure that the path containing python.exe
>>>>> and python.dll is in the PATH variable. You can set it system wide in
>>>>> the system advanced settings, or from a terminal like this:
>>>>>
>>>>> PATH=%PATH%;C:\python2.7.1\bin
>>>>>
>>>>> On a recent linux distribution with a system installed python a
>>>>> similar step should not be needed, but just in case you have installed
>>>>> your own, you would need an equivalent like:
>>>>>
>>>>> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/pack/python-2.7.1/bin
>>>>>
>>>>> ==== System independent python ====
>>>>>
>>>>> Suppose you want to have several independent python versions along
>>>>> with their packages and you do not want to register any of them with
>>>>> the system. In such a case, for the best compatibility and
>>>>> consistency, you should compile python from source yourself, then
>>>>> assemble the build into a packaged folder containing three
>>>>> sub-folders: bin, include and lib. "bin" should contain python
>>>>> executable and the dlls/shared objects, "include" the header files and
>>>>> "lib" a subfolder site-packages. Now all there is to do is to naively
>>>>> copy all dependent dll's/so's from the vtk build bin folder into the
>>>>> custom python package's bin folder, and the vtk python package
>>>>> subfolder from Wrapping/Python/vtk into  lib/site-packages. Now double
>>>>> clicking python executable should bring up a python console, from
>>>>> which you should be able to do things like:
>>>>>
>>>>> from vtk import *
>>>>> image = vtkImageData()
>>>>> print image
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 8, 2011 at 7:41 PM, David Gobbi <david.gobbi at gmail.com> wrote:
>>>>>> Hi Dominik,
>>>>>>
>>>>>> It is a given that when using python.exe, either the DLLs have to be
>>>>>> copied into the path, or the PATH and PYTHONPATH variables have
>>>>>> to be set.  So I'll just leave that part of the question aside, hopefully
>>>>>> someone from Kitware who deals with the VTK installation process can
>>>>>> answer.
>>>>>>
>>>>>> The vtkpython executable is a different story, though, because it is
>>>>>> supposed to find the VTK dlls automatically.  It does have some
>>>>>> limitations: it cannot know all of the library dependencies, so it
>>>>>> cannot be expected to know where to find the Qt DLLs, and these
>>>>>> will have to be in the PATH.  When you used the dependency walker,
>>>>>> did you find any dependencies that were unusual or unexpected?
>>>>>>
>>>>>> If you put together a wiki page, that would definitely be useful for
>>>>>> other users, I'd be willing to review it for you.  You can also look
>>>>>> at VTK/Wrapping/Python/README.txt to see if it has relevant
>>>>>> information.
>>>>>>
>>>>>>  - David
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 8, 2011 at 10:59 AM, Dominik Szczerba <dominik at itis.ethz.ch> wrote:
>>>>>>> I finally got 2.7 to work. Please review, if this is not dirty hacking
>>>>>>> I can commit a small wiki page with a HOWTO.
>>>>>>>
>>>>>>> Both python26 and python27 will work if ALL required DLL's will be
>>>>>>> copied over to python\bin. Which ones I required can be debugged with
>>>>>>> the dependency walker. This includes, e.g., the Qt DLL's, because I
>>>>>>> was compiling with Qt GUI support. Probably compiling without Qt, or
>>>>>>> appending the missing DLL paths to PATH would work too, but I have not
>>>>>>> tested it.
>>>>>>>
>>>>>>> I do not see how executing 'vtkpython' can work in such cases. I found
>>>>>>> no documentation about possible setup scenarios and required system
>>>>>>> variables.
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Dominik
>>>>>>>
>>>>>>> On Wed, Jun 8, 2011 at 5:52 PM, David Gobbi <david.gobbi at gmail.com> wrote:
>>>>>>>> I can't speak to why python 2.7 would be different from python 2.6,
>>>>>>>> because I have never tried it... hopefully someone else can answer
>>>>>>>> that question.
>>>>>>>>
>>>>>>>>  - David
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 8, 2011 at 7:08 AM, Dominik Szczerba <dominik at itis.ethz.ch> wrote:
>>>>>>>>> I have now manually copied everything from vtk's build
>>>>>>>>> bin\RelWithDebInfo folder to python's bin folder and I have manually
>>>>>>>>> copied Wrapping\Python\vtk to python's lib\site-packages folder.
>>>>>>>>> Such treatment seems to work for python 2.6.7 (= "import vtk" works),
>>>>>>>>> but does not work for python 2.7.1. Note, I have no python/vtk in the
>>>>>>>>> system path and pythonhome and pythonpath are not set.
>>>>>>>>>
>>>>>>>>> Am I hacking and this will not work the next release, or is the right
>>>>>>>>> way to go? Why would 2.7 not work on Windows? Works on linux all
>>>>>>>>> right.
>>>>>>>>>
>>>>>>>>> Thanks for any more hints,
>>>>>>>>> Dominik
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jun 8, 2011 at 12:59 PM, Dominik Szczerba <dominik at itis.ethz.ch> wrote:
>>>>>>>>>> Hi David,
>>>>>>>>>>
>>>>>>>>>> I have tried python 2.6.7 with the same failure. Is this a known issue
>>>>>>>>>> for Windows? My configure settings are pretty straight forward: using
>>>>>>>>>> shared libs, paths to python/include/lib, enable wrapping, disable TK,
>>>>>>>>>> did I forget anything?
>>>>>>>>>>
>>>>>>>>>> Is there somewhere a comprehensive information what should reside / be
>>>>>>>>>> copied where such that I can use vtk with python? I never saw one,
>>>>>>>>>> even under http://www.vtk.org/Wiki/VTK/Python_Wrapping_FAQ
>>>>>>>>>> I do not (in fact, can not) use vtkpython (that does not work for now
>>>>>>>>>> anyway). I need to create a custom python+vtk environment, totally
>>>>>>>>>> unregistered with the system (so I can have multiple versions).
>>>>>>>>>>
>>>>>>>>>> Many thanks and best regards,
>>>>>>>>>> Dominik
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 7, 2011 at 11:46 PM, David Gobbi <david.gobbi at gmail.com> wrote:
>>>>>>>>>>> Hi Dominik,
>>>>>>>>>>>
>>>>>>>>>>> It might be as simple as cmake not properly detecting python, I know that
>>>>>>>>>>> it was unable to detect python-2.7 until very recently, in fact even my
>>>>>>>>>>> copy of cmake-2.8.2 is missing the bits that it needs to find python-2.7.
>>>>>>>>>>>
>>>>>>>>>>> Take a look through your CMakeCache.txt file to make sure that all of the
>>>>>>>>>>> python-related variables are set.
>>>>>>>>>>>
>>>>>>>>>>> Also, now that you know for certain that it is a path-related issue, make
>>>>>>>>>>> sure to clear the PYTHONHOME variable or it will get you into trouble
>>>>>>>>>>> in the future.
>>>>>>>>>>>
>>>>>>>>>>>  - David
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jun 7, 2011 at 3:36 PM, Dominik Szczerba <dominik at itis.ethz.ch> wrote:
>>>>>>>>>>>> Hi David,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks, indeed, there was no PYTHONHOME set. But setting it brings me
>>>>>>>>>>>> only one step further. The next is the error when calling "import
>>>>>>>>>>>> vtk". See the error at the end. I saw a similar thread when you wrote
>>>>>>>>>>>> it is about mixing release and debug for python. It is not the case
>>>>>>>>>>>> here, both python and vtk is self compiled with the same compiler
>>>>>>>>>>>> settings (Release). I blind tried various combinations of PATH and
>>>>>>>>>>>> PYTHONPATH to no avail. I will be happy about some more ideas to try.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Dominik
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jun 7, 2011 at 11:26 PM, David Gobbi <david.gobbi at gmail.com> wrote:
>>>>>>>>>>>>> Hi Dominik,
>>>>>>>>>>>>>
>>>>>>>>>>>>> The "site" module is the first to be imported, so if it is missing,
>>>>>>>>>>>>> then the "PYTHONHOME" (or equivalent) is not set or is incorrect and
>>>>>>>>>>>>> python is unable to find its own modules.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If you set PYTHONHOME manually (i.e. as an environment variable) to
>>>>>>>>>>>>> the directory where python-2.7 resides, then I am guessing that these
>>>>>>>>>>>>> tests will pass.
>>>>>>>>>>>>>
>>>>>>>>>>>>> So... the next question is, why is this happening?  Evidently the
>>>>>>>>>>>>> vtkpython executable is missing some critical information that allows
>>>>>>>>>>>>> it to find the python-2.7 directory on its own.  The relevant code in
>>>>>>>>>>>>> VTK is Wrapping/Python/vtkPythonAppInit.cxx
>>>>>>>>>>>>>
>>>>>>>>>>>>> None of the dashboard machines seem to use python-2.7.
>>>>>>>>>>>>>
>>>>>>>>>>>>>  - David
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Jun 7, 2011 at 2:44 PM, Dominik Szczerba <dominik at itis.ethz.ch> wrote:
>>>>>>>>>>>>>> Below some output from testing log. I get exactly the same error
>>>>>>>>>>>>>> (ImportError: No module named site) by just invoking vtkpython from
>>>>>>>>>>>>>> command line.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ----------------------------------------------------------
>>>>>>>>>>>>>> 59/251 Testing: PythonSmokePython-image
>>>>>>>>>>>>>> 59/251 Test: PythonSmokePython-image
>>>>>>>>>>>>>> Command: "E:/build/vtk-5.6.1/bin/Release/vtkpython.exe"
>>>>>>>>>>>>>> "E:/pack/vtk-5.6.1/Common/Testing/Python/PythonSmoke.py"
>>>>>>>>>>>>>> Directory: E:/build/vtk-5.6.1/Common/Testing/Python
>>>>>>>>>>>>>> "PythonSmokePython-image" start time: Jun 07 22:27 W. Europe Daylight Time
>>>>>>>>>>>>>> Output:
>>>>>>>>>>>>>> ----------------------------------------------------------
>>>>>>>>>>>>>> ImportError: No module named site
>>>>>>>>>>>>>> <end of output>
>>>>>>>>>>>>>> Test time =   2.22 sec
>>>>>>>>>>>>>> ----------------------------------------------------------
>>>>>>>>>>>>>> Test Failed.
>>>>>>>>>>>>>> "PythonSmokePython-image" end time: Jun 07 22:27 W. Europe Daylight Time
>>>>>>>>>>>>>> "PythonSmokePython-image" time elapsed: 00:00:02
>>>>>>>>>>>>>> ----------------------------------------------------------
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Jun 7, 2011 at 10:37 PM, Dominik Szczerba <dominik at itis.ethz.ch> wrote:
>>>>>>>>>>>>>>> I have compiled python 2.7 in Release mode on MSVC 2010 64bit. I have
>>>>>>>>>>>>>>> built vtk-5.6.1 against it (with wrapping enabled). However, I get the
>>>>>>>>>>>>>>> below errors when testing the build.
>>>>>>>>>>>>>>> My setup is rather standard, I was therefore wondering if there could
>>>>>>>>>>>>>>> be something obviously wrong, or 2.7 is simply not yet supported.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Dominik
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> E:\build\vtk-5.6.1>ctest -R Python
>>>>>>>>>>>>>>> Test project E:/build/vtk-5.6.1
>>>>>>>>>>>>>>>    Start  59: PythonSmokePython-image
>>>>>>>>>>>>>>> 1/6 Test  #59: PythonSmokePython-image ......................***Failed    2.22 s
>>>>>>>>>>>>>>> ec
>>>>>>>>>>>>>>>    Start  60: TestWeakrefPython-image
>>>>>>>>>>>>>>> 2/6 Test  #60: TestWeakrefPython-image ......................***Failed    0.09 s
>>>>>>>>>>>>>>> ec
>>>>>>>>>>>>>>>    Start  61: TestNumpySupportPython-image
>>>>>>>>>>>>>>> 3/6 Test  #61: TestNumpySupportPython-image .................***Failed    0.09 s
>>>>>>>>>>>>>>> ec
>>>>>>>>>>>>>>>    Start  62: TestTerminationCrashPython-image
>>>>>>>>>>>>>>> 4/6 Test  #62: TestTerminationCrashPython-image .............***Failed    0.08 s
>>>>>>>>>>>>>>> ec
>>>>>>>>>>>>>>>    Start  63: TestExecuteMethodFinalizeCrashPython-image
>>>>>>>>>>>>>>> 5/6 Test  #63: TestExecuteMethodFinalizeCrashPython-image ...***Failed    0.09 s
>>>>>>>>>>>>>>> ec
>>>>>>>>>>>>>>>    Start 237: kMeansDistanceCalculatorPython
>>>>>>>>>>>>>>> 6/6 Test #237: kMeansDistanceCalculatorPython ...............***Failed    0.08 s
>>>>>>>>>>>>>>> ec
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 0% tests passed, 6 tests failed out of 6
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Total Test time (real) =   3.50 sec
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The following tests FAILED:
>>>>>>>>>>>>>>>         59 - PythonSmokePython-image (Failed)
>>>>>>>>>>>>>>>         60 - TestWeakrefPython-image (Failed)
>>>>>>>>>>>>>>>         61 - TestNumpySupportPython-image (Failed)
>>>>>>>>>>>>>>>         62 - TestTerminationCrashPython-image (Failed)
>>>>>>>>>>>>>>>         63 - TestExecuteMethodFinalizeCrashPython-image (Failed)
>>>>>>>>>>>>>>>        237 - kMeansDistanceCalculatorPython (Failed)
>>>>>>>>>>>>>>> Errors while running CTest
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> Powered by www.kitware.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>>>>>>>>>> http://www.vtk.org/mailman/listinfo/vtkusers
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>

More information about the vtkusers mailing list