[CMake] library path stripping

Michael Hertling mhertling at online.de
Thu Jun 9 18:16:09 EDT 2011 

On 06/09/2011 08:18 AM, Andreas Naumann wrote:
> Am 09.06.2011 07:35, schrieb Michael Hertling:
>> On 06/09/2011 07:13 AM, Andreas Naumann wrote:
>>    
>>> Am 08.06.2011 20:43, schrieb Andreas Pakulat:
>>>      
>>>> On 08.06.11 20:00:54, Andreas Naumann wrote:
>>>>
>>>>        
>>>>> Am 08.06.2011 15:02, schrieb Eric Noulard:
>>>>>
>>>>>          
>>>>>> 2011/6/8 Andreas Naumann<Andreas-Naumann at gmx.net>:
>>>>>>
>>>>>>            
>>>>>>> Am 08.06.2011 11:56, schrieb Eric Noulard:
>>>>>>>
>>>>>>>              
>>>>>>>> 2011/6/8 Andreas Naumann<Andreas-Naumann at gmx.net>:
>>>>>>>>
>>>>>>>>
>>>>>>>>                
>>>>>>>>> Hi @all,
>>>>>>>>>
>>>>>>>>> I have some problem with the library usage in cmake.
>>>>>>>>>
>>>>>>>>> It seems to me, that cmake removes the full path of the library, if the
>>>>>>>>> path
>>>>>>>>> is in the environment variable LIBRARY_PATH.
>>>>>>>>> This behaviour cause problems at our system, such that the linker links
>>>>>>>>> against the wrong library.
>>>>>>>>>
>>>>>>>>> Is there an option to avoid this splitting?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                  
>>>>>>>> Did you read that:
>>>>>>>> http://www.cmake.org/Wiki/CMake_RPATH_handling
>>>>>>>>
>>>>>>>> Which version of CMake are you using?
>>>>>>>> On which system ?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                
>>>>>>> We are using version 2.8.2 and 2.8.4 on Debian and Suse Linux.
>>>>>>>
>>>>>>> I've read the hints on RPATH handling, but there it is said, that:
>>>>>>> "By default if you don't change any RPATH related settings, CMake will link
>>>>>>> the executables and shared libraries with full RPATH to all used libraries
>>>>>>> in the build tree"
>>>>>>>
>>>>>>>              
>>>>>> Yes.
>>>>>> And you did not mention it but the probleme you have occurs when using
>>>>>> the executable **directly from the build dir** right?
>>>>>>
>>>>>>            
>>>>> what do you mean with "using"? I cannot even link the executable,
>>>>> because cmake removes the path from the library without adding the
>>>>> directory to the library directories.
>>>>>
>>>>>
>>>>>          
>>>>>>> This means, if I don't set anything related to RPATH, cmake should not strip
>>>>>>> the path from the library, should it?
>>>>>>>
>>>>>>>              
>>>>>> No it shoudn't for the binary in the buitd tree but...
>>>>>> if you do "make install" the installed binaries will have no RPATH unless
>>>>>> you set
>>>>>>
>>>>>> SET(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib")
>>>>>>
>>>>>> # add the automatically determined parts of the RPATH
>>>>>> # which point to directories outside the build tree to the install RPATH
>>>>>> SET(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
>>>>>>
>>>>>>
>>>>>>            
>>>>> I don't even want to install, just build and use.
>>>>>
>>>>>          
>>>>>>> I played with the example and it was a bit hard to understand, when the path
>>>>>>> is stripped from the library.
>>>>>>> The problem arises before linking the executable.
>>>>>>>
>>>>>>>              
>>>>>> This is a different issue.
>>>>>> Could you copy/paste the
>>>>>>
>>>>>> add_executable
>>>>>> and
>>>>>> target_link_libraries
>>>>>>
>>>>>> statement you use for the offending executable?
>>>>>>
>>>>>>
>>>>>>            
>>>>> simple:
>>>>> project("test")
>>>>>
>>>>> cmake_minimum_required(VERSION 2.8)
>>>>> set(MYLIB /home/andreas/cmake_test/lib/libfoo.so)
>>>>> add_executable(foo_exec test.cc)
>>>>> target_link_libraries(foo_exec ${MYLIB})
>>>>>
>>>>> So the executable gets the full name and the example works, if the
>>>>> environment variable library_path is not set to
>>>>> /home/andreas/cmake_test/lib. The link command is:
>>>>>
>>>>> /usr/bin/c++      CMakeFiles/foo_exec.dir/test.cc.o  -o foo_exec
>>>>> -rdynamic /home/andreas/cmake_test/lib/libfoo.so
>>>>> -Wl,-rpath,/home/andreas/cmake_test/lib
>>>>>
>>>>> If I set LIBRARY_PATH to /home/andreas/cmake_test/lib, the directory
>>>>> is stripped and the link-command gets:
>>>>>
>>>>> /usr/bin/c++      CMakeFiles/foo_exec.dir/test.cc.o  -o foo_exec
>>>>> -rdynamic -lfoo
>>>>>
>>>>>          
>>>> I can reproduce this here with CMake 2.8.4.
>>>>
>>>> And before someone else asks (since I thought that might be the error),
>>>> the envvar is really "LIBRARY_PATH", i.e. not LD_LIBRARY_PATH.
>>>>
>>>> The strange thing is that there's no occurrence of this specific string
>>>> anywhere in the cmake sources. There are all kinds of occurrence of
>>>> LD_... or CMAKE_... but no string matching just LIBRARY_PATH. So its
>>>> unclear why CMake respects that envvar.
>>>>
>>>> Andreas
>>>>
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> http://www.cmake.org/mailman/listinfo/cmake
>>>>
>>>>
>>>>        
>>> The problem is related to the compiler, not to the linker. The
>>> gcc-documentation
>>> http://gcc.gnu.org/onlinedocs/gcc-4.6.0/gcc/Environment-Variables.html#Environment-Variables
>>> says:
>>>
>>> LIBRARY_PATH
>>> The value of LIBRARY_PATH is a colon-separated list of directories, much
>>> like PATH. When configured as a native compiler, GCC tries the
>>> directories thus specified when searching for special linker files, if
>>> it can't find them using GCC_EXEC_PREFIX. Linking using GCC also uses
>>> these directories when searching for ordinary libraries for the -l
>>> option (but directories specified with -L come first).
>>>
>>> CMake seems to ask the compiler for implicitly used link directories and
>>> removes each path in this list from the libraries in the directories.
>>> Because this information comes from the compiler, cmake cannot
>>> distinguish between working and not working directories.
>>> My question is now, why does cmake remove the paths? If the use
>>> specifies a library with full path, he wants EXACTLY this library.
>>>
>>> Currently, we print a warning, if the variable LIBRARY_PATH is set in
>>> the environment.
>>>
>>> Andreas
>>>      
>> During the initial examination of the compiler ("Detecting CXX compiler
>> ABI info"), the test program is linked with -v, the output captured and
>> the link line scanned for -L arguments, thus detecting the directories
>> in the LIBRARY_PATH. The results go into the CMAKE_CXX_IMPLICIT_LINK_
>> DIRECTORIES variable; use --debug-trycompile and look into CMakeFiles/
>> CMakeOutput.log in order to see. Libraries in a directory mentioned in
>> this variable are treated as system libraries and will be linked with
>> -l instead of a full path, see [1] for the reasons. If you need the
>> library's full path despite of this mechanism, use imported targets.
>>
>> Regards,
>>
>> Michael
>>
>>    
>>> [1] http://www.cmake.org/Wiki/CMake_2.6_Notes#Linking_to_System_Libraries
>>>      
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>
>> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.cmake.org/mailman/listinfo/cmake
>>
>>    
> @Michael:
> 
> Thank you very much for this hint. It helps to understand and track down 
> the problem.
> In our special case, it is a bad combination of environment variable 
> setting and compiler configuration.
> 
> The old libraries, we are looking for are in /usr/lib[64], which IS a 
> standard system path. The new libraries are on /some/other/path AND the 
> LIBRARY_PATH variable is set.
> Now, it depends on the compiler configuration, in which order the 
> LIBRARY_PATH and other SYSTEM_PATHS are combined, and the paths in 
> $LIBRARY_PATH are after the other system paths..
> 
> But this is a part of the compiler which cmake has no access to.

IIRC, trouble of this kind has been the reason for CMake to prefer full
paths for libraries, and if you get full-path-specified libraries into
the link command - either immediately or by imported targets - it does
not matter which directories in which order the compiler searches. So,
I'm not sure if I understand why your problem still persists.

> Working with imported targets gets complicated, because the libraries 
> with full path are given by the user and adding each one as imported 
> target looks a bit odd.

If I'm not mistaken, referring to external libraries, i.e. libraries
not built by the project, is quite exactly what imported targets are
meant for; thus, their usage should not be odd but normal. If their
additional effort with ADD_LIBRARY(... IMPORTED) and the IMPORTED_
LOCATION properties is annoying, you can write a function for this
purpose, or are there other reasons why using imported targets for
user-supplied libraries gets complicated?

Regards,

Michael

More information about the CMake mailing list