[CMake] library path stripping
Andreas Naumann
Andreas-Naumann at gmx.net
Sun Jun 12 01:08:05 EDT 2011
Am 11.06.2011 16:21, schrieb Michael Hertling:
> On 06/10/2011 12:57 PM, Andreas Naumann wrote:
>
>> Am 10.06.2011 00:16, schrieb Michael Hertling:
>>
>>> On 06/09/2011 08:18 AM, Andreas Naumann wrote:
>>>
>>>
>>>> Am 09.06.2011 07:35, schrieb Michael Hertling:
>>>>
>>>>
>>>>> On 06/09/2011 07:13 AM, Andreas Naumann wrote:
>>>>>
>>>>>
>>>>>
>>>>>> Am 08.06.2011 20:43, schrieb Andreas Pakulat:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 08.06.11 20:00:54, Andreas Naumann wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Am 08.06.2011 15:02, schrieb Eric Noulard:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> 2011/6/8 Andreas Naumann<Andreas-Naumann at gmx.net>:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Am 08.06.2011 11:56, schrieb Eric Noulard:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> 2011/6/8 Andreas Naumann<Andreas-Naumann at gmx.net>:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Hi @all,
>>>>>>>>>>>>
>>>>>>>>>>>> I have some problem with the library usage in cmake.
>>>>>>>>>>>>
>>>>>>>>>>>> It seems to me, that cmake removes the full path of the library, if the
>>>>>>>>>>>> path
>>>>>>>>>>>> is in the environment variable LIBRARY_PATH.
>>>>>>>>>>>> This behaviour cause problems at our system, such that the linker links
>>>>>>>>>>>> against the wrong library.
>>>>>>>>>>>>
>>>>>>>>>>>> Is there an option to avoid this splitting?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Did you read that:
>>>>>>>>>>> http://www.cmake.org/Wiki/CMake_RPATH_handling
>>>>>>>>>>>
>>>>>>>>>>> Which version of CMake are you using?
>>>>>>>>>>> On which system ?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> We are using version 2.8.2 and 2.8.4 on Debian and Suse Linux.
>>>>>>>>>>
>>>>>>>>>> I've read the hints on RPATH handling, but there it is said, that:
>>>>>>>>>> "By default if you don't change any RPATH related settings, CMake will link
>>>>>>>>>> the executables and shared libraries with full RPATH to all used libraries
>>>>>>>>>> in the build tree"
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> Yes.
>>>>>>>>> And you did not mention it but the probleme you have occurs when using
>>>>>>>>> the executable **directly from the build dir** right?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> what do you mean with "using"? I cannot even link the executable,
>>>>>>>> because cmake removes the path from the library without adding the
>>>>>>>> directory to the library directories.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> This means, if I don't set anything related to RPATH, cmake should not strip
>>>>>>>>>> the path from the library, should it?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> No it shoudn't for the binary in the buitd tree but...
>>>>>>>>> if you do "make install" the installed binaries will have no RPATH unless
>>>>>>>>> you set
>>>>>>>>>
>>>>>>>>> SET(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib")
>>>>>>>>>
>>>>>>>>> # add the automatically determined parts of the RPATH
>>>>>>>>> # which point to directories outside the build tree to the install RPATH
>>>>>>>>> SET(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> I don't even want to install, just build and use.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> I played with the example and it was a bit hard to understand, when the path
>>>>>>>>>> is stripped from the library.
>>>>>>>>>> The problem arises before linking the executable.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> This is a different issue.
>>>>>>>>> Could you copy/paste the
>>>>>>>>>
>>>>>>>>> add_executable
>>>>>>>>> and
>>>>>>>>> target_link_libraries
>>>>>>>>>
>>>>>>>>> statement you use for the offending executable?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> simple:
>>>>>>>> project("test")
>>>>>>>>
>>>>>>>> cmake_minimum_required(VERSION 2.8)
>>>>>>>> set(MYLIB /home/andreas/cmake_test/lib/libfoo.so)
>>>>>>>> add_executable(foo_exec test.cc)
>>>>>>>> target_link_libraries(foo_exec ${MYLIB})
>>>>>>>>
>>>>>>>> So the executable gets the full name and the example works, if the
>>>>>>>> environment variable library_path is not set to
>>>>>>>> /home/andreas/cmake_test/lib. The link command is:
>>>>>>>>
>>>>>>>> /usr/bin/c++ CMakeFiles/foo_exec.dir/test.cc.o -o foo_exec
>>>>>>>> -rdynamic /home/andreas/cmake_test/lib/libfoo.so
>>>>>>>> -Wl,-rpath,/home/andreas/cmake_test/lib
>>>>>>>>
>>>>>>>> If I set LIBRARY_PATH to /home/andreas/cmake_test/lib, the directory
>>>>>>>> is stripped and the link-command gets:
>>>>>>>>
>>>>>>>> /usr/bin/c++ CMakeFiles/foo_exec.dir/test.cc.o -o foo_exec
>>>>>>>> -rdynamic -lfoo
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> I can reproduce this here with CMake 2.8.4.
>>>>>>>
>>>>>>> And before someone else asks (since I thought that might be the error),
>>>>>>> the envvar is really "LIBRARY_PATH", i.e. not LD_LIBRARY_PATH.
>>>>>>>
>>>>>>> The strange thing is that there's no occurrence of this specific string
>>>>>>> anywhere in the cmake sources. There are all kinds of occurrence of
>>>>>>> LD_... or CMAKE_... but no string matching just LIBRARY_PATH. So its
>>>>>>> unclear why CMake respects that envvar.
>>>>>>>
>>>>>>> Andreas
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Powered by www.kitware.com
>>>>>>>
>>>>>>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>>>>>>
>>>>>>> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>>>>>>>
>>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>>> http://www.cmake.org/mailman/listinfo/cmake
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> The problem is related to the compiler, not to the linker. The
>>>>>> gcc-documentation
>>>>>> http://gcc.gnu.org/onlinedocs/gcc-4.6.0/gcc/Environment-Variables.html#Environment-Variables
>>>>>> says:
>>>>>>
>>>>>> LIBRARY_PATH
>>>>>> The value of LIBRARY_PATH is a colon-separated list of directories, much
>>>>>> like PATH. When configured as a native compiler, GCC tries the
>>>>>> directories thus specified when searching for special linker files, if
>>>>>> it can't find them using GCC_EXEC_PREFIX. Linking using GCC also uses
>>>>>> these directories when searching for ordinary libraries for the -l
>>>>>> option (but directories specified with -L come first).
>>>>>>
>>>>>> CMake seems to ask the compiler for implicitly used link directories and
>>>>>> removes each path in this list from the libraries in the directories.
>>>>>> Because this information comes from the compiler, cmake cannot
>>>>>> distinguish between working and not working directories.
>>>>>> My question is now, why does cmake remove the paths? If the use
>>>>>> specifies a library with full path, he wants EXACTLY this library.
>>>>>>
>>>>>> Currently, we print a warning, if the variable LIBRARY_PATH is set in
>>>>>> the environment.
>>>>>>
>>>>>> Andreas
>>>>>>
>>>>>>
>>>>>>
>>>>> During the initial examination of the compiler ("Detecting CXX compiler
>>>>> ABI info"), the test program is linked with -v, the output captured and
>>>>> the link line scanned for -L arguments, thus detecting the directories
>>>>> in the LIBRARY_PATH. The results go into the CMAKE_CXX_IMPLICIT_LINK_
>>>>> DIRECTORIES variable; use --debug-trycompile and look into CMakeFiles/
>>>>> CMakeOutput.log in order to see. Libraries in a directory mentioned in
>>>>> this variable are treated as system libraries and will be linked with
>>>>> -l instead of a full path, see [1] for the reasons. If you need the
>>>>> library's full path despite of this mechanism, use imported targets.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Michael
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> [1] http://www.cmake.org/Wiki/CMake_2.6_Notes#Linking_to_System_Libraries
>>>>>>
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> Powered by www.kitware.com
>>>>>
>>>>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>>>>
>>>>> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>>>>>
>>>>> Follow this link to subscribe/unsubscribe:
>>>>> http://www.cmake.org/mailman/listinfo/cmake
>>>>>
>>>>>
>>>>>
>>>>>
>>>> @Michael:
>>>>
>>>> Thank you very much for this hint. It helps to understand and track down
>>>> the problem.
>>>> In our special case, it is a bad combination of environment variable
>>>> setting and compiler configuration.
>>>>
>>>> The old libraries, we are looking for are in /usr/lib[64], which IS a
>>>> standard system path. The new libraries are on /some/other/path AND the
>>>> LIBRARY_PATH variable is set.
>>>> Now, it depends on the compiler configuration, in which order the
>>>> LIBRARY_PATH and other SYSTEM_PATHS are combined, and the paths in
>>>> $LIBRARY_PATH are after the other system paths..
>>>>
>>>> But this is a part of the compiler which cmake has no access to.
>>>>
>>>>
>>> IIRC, trouble of this kind has been the reason for CMake to prefer full
>>> paths for libraries, and if you get full-path-specified libraries into
>>> the link command - either immediately or by imported targets - it does
>>> not matter which directories in which order the compiler searches. So,
>>> I'm not sure if I understand why your problem still persists.
>>>
>>>
>> The problem persists, because the compiler tells cmake, the directory in
>> LIBRARY_PATH would be a system directory. That's why, the library is
>> linked with -l. Now, it is up to the compiler configuration to set the
>> directories in LIBRARY_PATH before the other system directories. At our
>> system, this is not the case.
>>
> And that's the reason why you must treat the concerned libraries as
> imported targets if you want to ensure that they are referred to by
> full paths. Otherwise, you are in danger that the compiler will use
> -l to link binaries against them without having a chance to prevent
> it, and the order of system directories will matter in such a case.
> Alternatively, you might perhaps remove the LIBRARY_PATH from the
> CMAKE_<LANG>_IMPLICIT_LINK_DIRECTORIES variables.
>
>
>>>> Working with imported targets gets complicated, because the libraries
>>>> with full path are given by the user and adding each one as imported
>>>> target looks a bit odd.
>>>>
>>>>
>>> If I'm not mistaken, referring to external libraries, i.e. libraries
>>> not built by the project, is quite exactly what imported targets are
>>> meant for; thus, their usage should not be odd but normal. If their
>>> additional effort with ADD_LIBRARY(... IMPORTED) and the IMPORTED_
>>> LOCATION properties is annoying, you can write a function for this
>>> purpose, or are there other reasons why using imported targets for
>>> user-supplied libraries gets complicated?
>>>
>>>
>>>
>> This means, I should call
>>
>> ADD_LIBRARY(libname IMPORTED)
>>
>> for each library I look for?
>>
> Yes.
>
>
>> And than also set IMPORTED_LOCATION?
>>
> Yes. Usually, that's slightly more work than referring to the affected
> libraries immediately, but it can be simplified with a macro/function
> and, among other things, does provide much more flexibility when the
> libraries to link against are configuration-specific.
>
>
>> CMake should prefer fullpaths of libraries, at least for libraries in
>> non-system directories, shouldn't it?
>>
> CMake *does* do so, but which directories are considered as system
> ones and which are not? As you've said yourself, it's the compiler
> that tells CMake about these directories, and the reason why this
> full-path approach is dropped for libraries therein is explained
> in the wiki I provided the link to.
>
>
>> When should I use add_library( ... IMPORTED) ?
>>
> Basically, IMO, one should use this for each external library unless
> there's a reason not to do so, typically with architecture/hardware-
> specific libraries the compiler probably knows better where to find
> them. But again: Even then, you can force to link against full-path
> specified libraries by using imported targets.
>
> Regards,
>
> Michael
>
You are right. If the user (or some FindXXX script) specify a library
with full path, I should use exactly this library. I'll give this
approach a try.
Regards,
Andreas
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake
>
>
More information about the CMake
mailing list