[CMake] RPATH cross-compile issue with CHECK_*_EXISTS

[Ray Donnelly]

On Mon, Feb 27, 2017 at 8:50 PM, Jörg Krause
<joerg.krause at embedded.rocks> wrote:
> Hi Brad,
>
> On Mon, 2017-02-27 at 11:43 -0500, Brad King wrote:
>> On 02/07/2017 04:40 AM, Ray Donnelly wrote:
>> > > > I have a PR that asks the linker (via the compiler) what its
>> > > > implicit
>> > > > search directories are instead.
>> > > >
>> > > > It is the right way to do it IMHO, but I need to find time to
>> > > > finish
>> > > > it unfortunately.
>> > >
>> > > Do you have a link to the PR?
>> >
>> > The PR Is closed pending me writing a test-case, but I just now
>> > updated to the my latest version and rebased on top of master:
>>
>> The MR was:
>>
>>  https://gitlab.kitware.com/cmake/cmake/merge_requests/207
>>
>> See discussion there for why it has not yet been accepted.  Basically
>> I'd like to see a clear explanation of the use case.  The case
>> described
>> in the MR looks to me like the custom compiler should be configured
>> to
>> always pass the needed rpath flags to the linker.
>>
>> On 02/06/2017 06:16 PM, Jörg Krause wrote:
>> > I did a git bisect. The behaviour was introduced in commit
>> > 896ad251de49f167f4ce3cbbcf9a6cce85a16681 [1].
>>
>> Thanks for the bisect.  I don't think there is anything wrong with
>> that
>> change on its own.  It merely exposed some existing behavior in a new
>> case.
>
> The problem is, that we end up with a host rpath when cross-compiling
> which breaks compilations for a number of CMake packages we build on
> Buildroot.

This is why my PR asks the linker that the compiler will use for the
actual list of implicit link dirs. I'm sorry I've not had time to
write up a clear explanation yet.

>
> Buildroot uses /sysroot/usr/lib as target library path and
> /sysroot/usr/lib32 is a symlink to that path. Nothing wrong here.
>
> The addition of FIND_LIBRARY_USE_LIB32_PATHS changes the behavior of
> `find_library()`. Before the commit `/sysroot/usr/lib` was
> found as library path, but now it's `/sysroot/usr/lib32`.
>
> When determining the runtime search path, CMake compares the paths
> found by `find_library()` with a list of implicit runtime pathes. This
> list contains `/sysroot/usr/lib` but not `/sysroot/usr/lib32`.
>
> If the library path found by `find_library()` matches a search path
> from the list of implicit runtime pathes it is dropped, otherwise it is
> added to rpath after removing the `/sysroot` path.
>
> So, as the implicit runtime search paths does *not* contain
> `/sysroot/usr/lib32`, find_library() ends up with a rpath set to
> `/usr/lib32`.
>
>
>
> One example of how cross-compilation is broken is the example I already
> quoted:
>
> """
> $SYSROOT/usr/bin/i586-linux-gcc --sysroot=$SYSROOT/usr/i586-buildroot-
> linux-musl/sysroot CheckSymbolExists.c.o -o cmTC_cb8f6 -Wl,-
> rpath,/usr/lib32 -rdynamic $SYSROOT/usr/i586-buildroot-linux-
> musl/sysroot/usr/lib32/libmbedtls.so
> """
>
> If libmbedtls is linked with libz, the linker tries to link the target
> libmbedtls with host libz, which fails:
>
> """
> $SYSROOT/usr/i586-buildroot-linux-musl/bin/ld: warning:
> libc.so.6, needed by /usr/lib32/libz.so.1, not found (try using -rpath
> or -rpath-link)
> /usr/lib32/libz.so.1: undefined reference to `strcpy at GLIBC_2.0'
> /usr/lib32/libz.so.1: undefined reference to `free at GLIBC_2.0'
> /usr/lib32/libz.so.1: undefined reference to `fseeko64 at GLIBC_2.1
> """
>
> Note, that Buildroot does not use a /sysroot/usr/lib64 symbolic link.
> Therefore, this behavior was not exposed before the commit.
>
> For me, it looks like there is a problem how the rpath is created when
> cross-compiling. Maybe the logic should check, if /sysroot/usr/lib32 is
> a symlink to an implicit runtime search path?
>
> However, I am not very familiar with CMake and the insights I described
> where gathered by some hours of debugging the CMake code. Maybe I
> missed something?
>
>> > My suggestion is to set FIND_LIBRARY_USE_LIB32_PATHS and
>> > FIND_LIBRARY_USE_LIB64_PATHS to FALSE when cross-compiling on
>> > Linux.
>>
>> These are set on by default in `Modules/Platform/UnixPaths.cmake` but
>> disabled on Debian by `Modules/Platform/Linux.cmake` except when
>> cross compiling.  If a toolchain file specifies CMAKE_SYSTEM_NAME
>> such that a custom `Platform/MySystem.cmake` file is loaded then
>> the latter can set them as needed for the target platform.
>
> Thanks for the hint. We are discussing this setting as a workaround.
>
> Best regards,
> Jörg Krause