[CMake] Issues trying to use the Anaconda compiler tools with CMake
Isaiah Norton
isaiah.norton at gmail.com
Mon Aug 13 11:54:00 EDT 2018
The RPATH is not included when using Anaconda because CMake considers the
Anaconda lib path to be an implicit link directory (as reported by the
compiler). Look at `$build_dir/CMakeFiles/CMakeOutput.log` in the section
about "implicit link info".
The simplest work-around is to set BUILD_RPATH manually, and possibly also
INSTALL_RPATH, depending on what conda-build's fixup routine expects.
For example, adding this line near the top of your example CMakeList fixed
the resulting binary, at least in the build directory:
set(CMAKE_BUILD_RPATH "$ENV{CONDA_PREFIX}/lib")
On Fri, Aug 10, 2018 at 5:55 PM Sebastián Mancilla <smancill at jlab.org>
wrote:
> I am trying to use Conda as a package manager for isolated C++ development
> environments. But unfortunately, when using CMake with the
> Anaconda-provided
> compilers [1] (which are used to compile the binary packages in the
> Anaconda
> repositories), things do not work as expected.
>
> I have a small test case available here [2], with an executable calling a
> shared library and a third-party dependency installed with Conda.
>
> [1]:
> https://conda.io/docs/user-guide/tasks/build-packages/compiler-tools.html
> [2]: https://gist.github.com/smancill/b28ca07ac11fdf285b4d559545a1630b
>
> --------------------------------------------------
>
> First, when using the system compiler, all works fine (but I am not sure
> of the
> binary compatibility with the Conda packages, that's why I want to use the
> Anaconda compilers):
>
> # create the environment and install XercesC
> $ conda create -n test-system xerces-c
> ...
> environment location:
> /Users/smancill/.local/share/miniconda3/envs/test-system
> ...
> The following NEW packages will be INSTALLED:
>
> icu: 58.2-h4b95b61_1
> libcxx: 4.0.1-h579ed51_0
> libcxxabi: 4.0.1-hebd6815_0
> xerces-c: 3.2.1-h44e365a_0
> ...
>
> # activate the environment
> $ conda activate test-system
>
> $ mkdir build-osx-system
> $ cd build-osx-system
> $ cmake -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX
> -DCMAKE_PREFIX_PATH=$CONDA_PREFIX ..
> -- The CXX compiler identification is AppleClang 9.0.0.9000039
> -- Check for working CXX compiler: /usr/bin/c++
> -- Check for working CXX compiler: /usr/bin/c++ -- works
> ...
> -- Found XercesC:
> /Users/smancill/.local/share/miniconda3/envs/test-system/lib/libxerces-c.dylib
> (found version "3.2.1")
> -- Configuring done
> -- Generating done
> -- Build files have been written to:
> /Users/smancill/src/conda-test/build-osx-system
>
> $ make -j1 VERBOSE=1
> ...
> [100%] Linking CXX executable bar
> /usr/local/bin/cmake -E cmake_link_script CMakeFiles/bar.dir/link.txt
> --verbose=1
> /usr/bin/c++ -isysroot
> /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
> -mmacosx-version-min=10.12 -Wl,-search_paths_first
> -Wl,-headerpad_max_install_names CMakeFiles/bar.dir/bar.cpp.o -o bar
> -Wl,-rpath,/Users/smancill/src/conda-test/build-osx-system
> -Wl,-rpath,/Users/smancill/.local/share/miniconda3/envs/test-system/lib
> libfoo.dylib
> /Users/smancill/.local/share/miniconda3/envs/test-system/lib/libxerces-c.dylib
> ...
>
> The build directory (~/src/conda-test/build-osx-system) and the conda
> environment lib directory (~/.local/share/miniconda3/envs/test-system/lib)
> are correctly added to the RPATH in the build tree by the link command:
>
> $ ./bar
> Hello, world!
>
> $ otool -L ./bar
> ./bar:
> @rpath/libfoo.dylib (compatibility version 1.0.0, current version
> 0.0.0)
> @rpath/libxerces-c-3.2.dylib (compatibility version 0.0.0, current
> version 0.0.0)
> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current
> version 400.9.0)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 1252.0.0)
>
> $ otool -l ./bar | grep -A2 LC_RPATH
> cmd LC_RPATH
> cmdsize 56
> path /Users/smancill/src/conda-test/build-osx-system (offset
> 12)
> --
> cmd LC_RPATH
> cmdsize 80
> path
> /Users/smancill/.local/share/miniconda3/envs/test-system/lib (offset 12)
>
> If I install the binary, it fails because I haven't configured CMake to
> set the install RPATH:
>
> $ make install
> ...
> Install the project...
> -- Install configuration: ""
> -- Installing:
> /Users/smancill/.local/share/miniconda3/envs/test-system/lib/libfoo.dylib
> -- Installing:
> /Users/smancill/.local/share/miniconda3/envs/test-system/include/foo.hpp
> -- Installing:
> /Users/smancill/.local/share/miniconda3/envs/test-system/bin/bar
>
> $ bar
> dyld: Library not loaded: @rpath/libfoo.dylib
> Referenced from:
> /Users/smancill/.local/share/miniconda4/envs/test-system/bin/bar
> Reason: image not found
> [1] 84611 abort bar
>
> $ otool -L $CONDA_PREFIX/bin/bar
> /Users/smancill/.local/share/miniconda3/envs/test-system/bin/bar:
> @rpath/libfoo.dylib (compatibility version 0.0.0, current version
> 0.0.0)
> @rpath/libxerces-c-3.2.dylib (compatibility version 0.0.0, current
> version 0.0.0)
> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current
> version 400.9.0)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 1252.0.0)
>
> $ otool -l $CONDA_PREFIX/bin/bar | grep -A2 LC_RPATH
> # empty
>
> # deactivate the environment to start again
> $ conda deactivate
>
> The same can be observed on Linux. Everything works as it should.
>
> --------------------------------------------------
>
> If I try to use Anaconda compilers on macOS, the build RPATH is not set
> properly anymore:
>
> # create the environment and install the Anaconda compiler for macOS,
> and XercesC
> $ conda create -n test-conda clangxx_osx-64 xerces-c
> ...
> environment location:
> /Users/smancill/.local/share/miniconda3/envs/test-conda
> ...
> The following NEW packages will be INSTALLED:
>
> cctools: 895-h7512d6f_0
> clang: 4.0.1-h662ec87_0
> clang_osx-64: 4.0.1-h1ce6c1d_11
> clangxx: 4.0.1-hc9b4283_0
> clangxx_osx-64: 4.0.1-h22b1bf0_11
> compiler-rt: 4.0.1-h5487866_0
> icu: 58.2-h4b95b61_1
> ld64: 274.2-h7c2db76_0
> libcxx: 4.0.1-h579ed51_0
> libcxxabi: 4.0.1-hebd6815_0
> llvm: 4.0.1-hc748206_0
> llvm-lto-tapi: 4.0.1-h6701bc3_0
> xerces-c: 3.2.1-h44e365a_0
> ...
>
> # activate the environment (which sets the variables to use the
> Anaconda compiler)
> $ conda activate test-conda
>
> $ mkdir build-osx-conda
> $ cd build-osx-conda
> $ cmake -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX
> -DCMAKE_PREFIX_PATH=$CONDA_PREFIX ..
> -- The CXX compiler identification is Clang 4.0.1
> -- Check for working CXX compiler:
> /Users/smancill/.local/share/miniconda3/envs/test-conda/bin/x86_64-apple-darwin13.4.0-clang++
> -- Check for working CXX compiler:
> /Users/smancill/.local/share/miniconda3/envs/test-conda/bin/x86_64-apple-darwin13.4.0-clang++
> -- works
> ...
> -- Found XercesC:
> /Users/smancill/.local/share/miniconda3/envs/test-conda/lib/libxerces-c.dylib
> (found version "3.2.1")
> -- Configuring done
> -- Generating done
> -- Build files have been written to:
> /Users/smancill/src/conda-test/build-osx-conda
>
> $ make -j1 VERBOSE=1
> ...
> [100%] Linking CXX executable bar
> /usr/local/bin/cmake -E cmake_link_script CMakeFiles/bar.dir/link.txt
> --verbose=1
>
> /Users/smancill/.local/share/miniconda3/envs/test-conda/bin/x86_64-apple-darwin13.4.0-clang++
> -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE
> -fstack-protector-strong -O2 -pipe -stdlib=libc++
> -fvisibility-inlines-hidden -std=c++14 -fmessage-length=0 -isysroot
> /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
> -mmacosx-version-min=10.12 -Wl,-search_paths_first
> -Wl,-headerpad_max_install_names -Wl,-pie -Wl,-headerpad_max_install_names
> -Wl,-dead_strip_dylibs CMakeFiles/bar.dir/bar.cpp.o -o bar
> -Wl,-rpath,/Users/smancill/src/conda-test/build-osx-conda libfoo.dylib
> /Users/smancill/.local/share/miniconda3/envs/test-conda/lib/libxerces-c.dylib
> ...
>
> You can see that the environment lib path is not added to the RPATH by the
> link command,
> which in turns result in the executable not running from the build tree
> anymore:
>
> $ ./bar
> dyld: Library not loaded: @rpath/libxerces-c-3.2.dylib
> Referenced from: /Users/smancill/src/conda-test/build-osx-conda/./bar
> Reason: image not found
> [1] 89350 abort ./bar
>
> $ otool -L ./bar
> ./bar:
> @rpath/libfoo.dylib (compatibility version 0.0.0, current version
> 0.0.0)
> @rpath/libxerces-c-3.2.dylib (compatibility version 0.0.0, current
> version 0.0.0)
> @rpath/libc++.1.dylib (compatibility version 1.0.0, current
> version 1.0.0)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 1252.0.0)
>
> $ otool -l ./bar | grep -A2 LC_RPATH
> cmd LC_RPATH
> cmdsize 56
> path /Users/smancill/src/conda-test/build-osx-conda (offset 12)
>
> # deactivate the environment
> $ conda deactivate
>
> --------------------------------------------------
>
> If I try the Anaconda compilers on Linux, there are strange results too:
>
> # create the environment and install the Anaconda compiler for Linux,
> and XercesC
> $ conda create -n test-conda gxx_linux-64 xerces-c
> ...
> environment location: /home/vagrant/miniconda3/envs/test-conda
> ...
> The following NEW packages will be INSTALLED:
>
> binutils_impl_linux-64: 2.28.1-had2808c_3
> binutils_linux-64: 7.2.0-had2808c_27
> gcc_impl_linux-64: 7.2.0-habb00fd_3
> gcc_linux-64: 7.2.0-h550dcbe_27
> gxx_impl_linux-64: 7.2.0-hdf63c60_3
> gxx_linux-64: 7.2.0-h550dcbe_27
> icu: 58.2-h9c2bf20_1
> libgcc-ng: 7.2.0-hdf63c60_3
> libstdcxx-ng: 7.2.0-hdf63c60_3
> xerces-c: 3.2.1-hac72e42_0
>
> # activate the environment (which sets the variables to use the
> Anaconda compiler)
> $ conda activate test-conda
>
> $ mkdir build-linux-conda
> $ cd build-linux-conda
> $ cmake -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX
> -DCMAKE_PREFIX_PATH=$CONDA_PREFIX ..
> -- The CXX compiler identification is GNU 7.2.0
> -- Check for working CXX compiler:
> /home/vagrant/miniconda3/envs/test-conda/bin/x86_64-conda_cos6-linux-gnu-c++
> -- Check for working CXX compiler:
> /home/vagrant/miniconda3/envs/test-conda/bin/x86_64-conda_cos6-linux-gnu-c++
> -- works
> ...
> -- Found XercesC:
> /home/vagrant/miniconda3/envs/test-conda/lib/libxerces-c.so (found version
> "3.2.1")
> -- Configuring done
> -- Generating done
> -- Build files have been written to:
> /vagrant/conda-test/build-linux-conda
>
> $ make -j1 VERBOSE=1
> ...
> [100%] Linking CXX executable bar
> /usr/bin/cmake -E cmake_link_script CMakeFiles/bar.dir/link.txt
> --verbose=1
>
> /home/vagrant/miniconda3/envs/test-conda/bin/x86_64-conda_cos6-linux-gnu-c++
> -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona
> -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2
> -pipe -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now
> CMakeFiles/bar.dir/bar.cpp.o -o bar libfoo.so
> /home/vagrant/miniconda3/envs/test-conda/lib/libxerces-c.so
> -Wl,-rpath,/vagrant/conda-test/build-linux-conda:/home/vagrant/miniconda3/envs/test-conda/lib:
> ...
>
> You can see that the environment lib path is added to the RPATH by the link
> command (unlike macOS):
>
> $ ./bar
> Hello World!
>
> But when inspecting the dependencies, the environment lib path appears
> twice:
>
> $ readelf -d ./bar
> ...
> 0x0000000000000001 (NEEDED) Shared library: [libfoo.so]
> 0x0000000000000001 (NEEDED) Shared library: [
> libxerces-c-3.2.so]
> 0x0000000000000001 (NEEDED) Shared library:
> [libstdc++.so.6]
> 0x0000000000000001 (NEEDED) Shared library:
> [libgcc_s.so.1]
> 0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
> 0x000000000000000f (RPATH) Library rpath:
> [/home/vagrant/miniconda3/envs/test-conda/lib:/vagrant/conda-test/build-linux-conda:/home/vagrant/miniconda3/envs/test-conda/lib:]
> ...
>
> Which is wrong. Now the build tree binary will pick first any old version
> of
> the foo library installed in the environment instead of the version in the
> build tree.
>
> It seems that the Anaconda toolchain is setting the environment lib path
> into
> the RPATH by its own. It is also set when installing the binaries, even
> when
> CMake is not configured to set the install RPATH:
>
> $ make install
> ...
> Install the project...
> -- Install configuration: ""
> -- Installing: /home/vagrant/miniconda3/envs/test-conda/lib/libfoo.so
> -- Installing: /home/vagrant/miniconda3/envs/test-conda/include/foo.hpp
> -- Installing: /home/vagrant/miniconda3/envs/test-conda/bin/bar
> -- Set runtime path of
> "/home/vagrant/miniconda3/envs/test-conda/bin/bar" to ""
>
> $ bar
> Hello World!
>
> $ readelf -d $CONDA_PREFIX/bin/bar
> ...
> 0x0000000000000001 (NEEDED) Shared library: [libfoo.so]
> 0x0000000000000001 (NEEDED) Shared library: [
> libxerces-c-3.2.so]
> 0x0000000000000001 (NEEDED) Shared library:
> [libstdc++.so.6]
> 0x0000000000000001 (NEEDED) Shared library:
> [libgcc_s.so.1]
> 0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
> 0x000000000000000f (RPATH) Library rpath:
> [/home/vagrant/miniconda3/envs/test-conda/lib]
> ...
>
> # deactivate the environment
> $ conda deactivate
>
> --------------------------------------------------
>
> TL;DR I cannot get CMake and the Anaconda compilers and packages working
> correctly.
>
> - On macOS, the Conda environment library path is not added to the build
> RPATH.
> - On Linux, the Conda environment library path is always added to the RPATH
> (in both build and install) independently of CMake.
>
> Any advice or workarounds?
>
> --
> Sebastian Mancilla
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> https://cmake.org/mailman/listinfo/cmake
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://cmake.org/pipermail/cmake/attachments/20180813/f25265c5/attachment-0001.html>
More information about the CMake
mailing list