[CMake] Issues trying to use the Anaconda compiler tools with CMake
Sebastián Mancilla
smancill at jlab.org
Fri Aug 10 17:48:42 EDT 2018
I am trying to use Conda as a package manager for isolated C++ development
environments. But unfortunately, when using CMake with the Anaconda-provided
compilers [1] (which are used to compile the binary packages in the Anaconda
repositories), things do not work as expected.
I have a small test case available here [2], with an executable calling a
shared library and a third-party dependency installed with Conda.
[1]:
https://conda.io/docs/user-guide/tasks/build-packages/compiler-tools.html
[2]: https://gist.github.com/smancill/b28ca07ac11fdf285b4d559545a1630b
--------------------------------------------------
First, when using the system compiler, all works fine (but I am not sure of
the
binary compatibility with the Conda packages, that's why I want to use the
Anaconda compilers):
# create the environment and install XercesC
$ conda create -n test-system xerces-c
...
environment location:
/Users/smancill/.local/share/miniconda3/envs/test-system
...
The following NEW packages will be INSTALLED:
icu: 58.2-h4b95b61_1
libcxx: 4.0.1-h579ed51_0
libcxxabi: 4.0.1-hebd6815_0
xerces-c: 3.2.1-h44e365a_0
...
# activate the environment
$ conda activate test-system
$ mkdir build-osx-system
$ cd build-osx-system
$ cmake -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX
-DCMAKE_PREFIX_PATH=$CONDA_PREFIX ..
-- The CXX compiler identification is AppleClang 9.0.0.9000039
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
...
-- Found XercesC:
/Users/smancill/.local/share/miniconda3/envs/test-system/lib/libxerces-c.dylib
(found version "3.2.1")
-- Configuring done
-- Generating done
-- Build files have been written to:
/Users/smancill/src/conda-test/build-osx-system
$ make -j1 VERBOSE=1
...
[100%] Linking CXX executable bar
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/bar.dir/link.txt
--verbose=1
/usr/bin/c++ -isysroot
/Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
-mmacosx-version-min=10.12 -Wl,-search_paths_first
-Wl,-headerpad_max_install_names CMakeFiles/bar.dir/bar.cpp.o -o bar
-Wl,-rpath,/Users/smancill/src/conda-test/build-osx-system
-Wl,-rpath,/Users/smancill/.local/share/miniconda3/envs/test-system/lib
libfoo.dylib
/Users/smancill/.local/share/miniconda3/envs/test-system/lib/libxerces-c.dylib
...
The build directory (~/src/conda-test/build-osx-system) and the conda
environment lib directory (~/.local/share/miniconda3/envs/test-system/lib)
are correctly added to the RPATH in the build tree by the link command:
$ ./bar
Hello, world!
$ otool -L ./bar
./bar:
@rpath/libfoo.dylib (compatibility version 1.0.0, current version
0.0.0)
@rpath/libxerces-c-3.2.dylib (compatibility version 0.0.0, current
version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current
version 400.9.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.0.0)
$ otool -l ./bar | grep -A2 LC_RPATH
cmd LC_RPATH
cmdsize 56
path /Users/smancill/src/conda-test/build-osx-system (offset 12)
--
cmd LC_RPATH
cmdsize 80
path
/Users/smancill/.local/share/miniconda3/envs/test-system/lib (offset 12)
If I install the binary, it fails because I haven't configured CMake to set
the install RPATH:
$ make install
...
Install the project...
-- Install configuration: ""
-- Installing:
/Users/smancill/.local/share/miniconda3/envs/test-system/lib/libfoo.dylib
-- Installing:
/Users/smancill/.local/share/miniconda3/envs/test-system/include/foo.hpp
-- Installing:
/Users/smancill/.local/share/miniconda3/envs/test-system/bin/bar
$ bar
dyld: Library not loaded: @rpath/libfoo.dylib
Referenced from:
/Users/smancill/.local/share/miniconda4/envs/test-system/bin/bar
Reason: image not found
[1] 84611 abort bar
$ otool -L $CONDA_PREFIX/bin/bar
/Users/smancill/.local/share/miniconda3/envs/test-system/bin/bar:
@rpath/libfoo.dylib (compatibility version 0.0.0, current version
0.0.0)
@rpath/libxerces-c-3.2.dylib (compatibility version 0.0.0, current
version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current
version 400.9.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.0.0)
$ otool -l $CONDA_PREFIX/bin/bar | grep -A2 LC_RPATH
# empty
# deactivate the environment to start again
$ conda deactivate
The same can be observed on Linux. Everything works as it should.
--------------------------------------------------
If I try to use Anaconda compilers on macOS, the build RPATH is not set
properly anymore:
# create the environment and install the Anaconda compiler for macOS,
and XercesC
$ conda create -n test-conda clangxx_osx-64 xerces-c
...
environment location:
/Users/smancill/.local/share/miniconda3/envs/test-conda
...
The following NEW packages will be INSTALLED:
cctools: 895-h7512d6f_0
clang: 4.0.1-h662ec87_0
clang_osx-64: 4.0.1-h1ce6c1d_11
clangxx: 4.0.1-hc9b4283_0
clangxx_osx-64: 4.0.1-h22b1bf0_11
compiler-rt: 4.0.1-h5487866_0
icu: 58.2-h4b95b61_1
ld64: 274.2-h7c2db76_0
libcxx: 4.0.1-h579ed51_0
libcxxabi: 4.0.1-hebd6815_0
llvm: 4.0.1-hc748206_0
llvm-lto-tapi: 4.0.1-h6701bc3_0
xerces-c: 3.2.1-h44e365a_0
...
# activate the environment (which sets the variables to use the
Anaconda compiler)
$ conda activate test-conda
$ mkdir build-osx-conda
$ cd build-osx-conda
$ cmake -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX
-DCMAKE_PREFIX_PATH=$CONDA_PREFIX ..
-- The CXX compiler identification is Clang 4.0.1
-- Check for working CXX compiler:
/Users/smancill/.local/share/miniconda3/envs/test-conda/bin/x86_64-apple-darwin13.4.0-clang++
-- Check for working CXX compiler:
/Users/smancill/.local/share/miniconda3/envs/test-conda/bin/x86_64-apple-darwin13.4.0-clang++
-- works
...
-- Found XercesC:
/Users/smancill/.local/share/miniconda3/envs/test-conda/lib/libxerces-c.dylib
(found version "3.2.1")
-- Configuring done
-- Generating done
-- Build files have been written to:
/Users/smancill/src/conda-test/build-osx-conda
$ make -j1 VERBOSE=1
...
[100%] Linking CXX executable bar
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/bar.dir/link.txt
--verbose=1
/Users/smancill/.local/share/miniconda3/envs/test-conda/bin/x86_64-apple-darwin13.4.0-clang++
-march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE
-fstack-protector-strong -O2 -pipe -stdlib=libc++
-fvisibility-inlines-hidden -std=c++14 -fmessage-length=0 -isysroot
/Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
-mmacosx-version-min=10.12 -Wl,-search_paths_first
-Wl,-headerpad_max_install_names -Wl,-pie -Wl,-headerpad_max_install_names
-Wl,-dead_strip_dylibs CMakeFiles/bar.dir/bar.cpp.o -o bar
-Wl,-rpath,/Users/smancill/src/conda-test/build-osx-conda libfoo.dylib
/Users/smancill/.local/share/miniconda3/envs/test-conda/lib/libxerces-c.dylib
...
You can see that the environment lib path is not added to the RPATH by the
link command,
which in turns result in the executable not running from the build tree
anymore:
$ ./bar
dyld: Library not loaded: @rpath/libxerces-c-3.2.dylib
Referenced from: /Users/smancill/src/conda-test/build-osx-conda/./bar
Reason: image not found
[1] 89350 abort ./bar
$ otool -L ./bar
./bar:
@rpath/libfoo.dylib (compatibility version 0.0.0, current version
0.0.0)
@rpath/libxerces-c-3.2.dylib (compatibility version 0.0.0, current
version 0.0.0)
@rpath/libc++.1.dylib (compatibility version 1.0.0, current version
1.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.0.0)
$ otool -l ./bar | grep -A2 LC_RPATH
cmd LC_RPATH
cmdsize 56
path /Users/smancill/src/conda-test/build-osx-conda (offset 12)
# deactivate the environment
$ conda deactivate
--------------------------------------------------
If I try the Anaconda compilers on Linux, there are strange results too:
# create the environment and install the Anaconda compiler for Linux,
and XercesC
$ conda create -n test-conda gxx_linux-64 xerces-c
...
environment location: /home/vagrant/miniconda3/envs/test-conda
...
The following NEW packages will be INSTALLED:
binutils_impl_linux-64: 2.28.1-had2808c_3
binutils_linux-64: 7.2.0-had2808c_27
gcc_impl_linux-64: 7.2.0-habb00fd_3
gcc_linux-64: 7.2.0-h550dcbe_27
gxx_impl_linux-64: 7.2.0-hdf63c60_3
gxx_linux-64: 7.2.0-h550dcbe_27
icu: 58.2-h9c2bf20_1
libgcc-ng: 7.2.0-hdf63c60_3
libstdcxx-ng: 7.2.0-hdf63c60_3
xerces-c: 3.2.1-hac72e42_0
# activate the environment (which sets the variables to use the
Anaconda compiler)
$ conda activate test-conda
$ mkdir build-linux-conda
$ cd build-linux-conda
$ cmake -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX
-DCMAKE_PREFIX_PATH=$CONDA_PREFIX ..
-- The CXX compiler identification is GNU 7.2.0
-- Check for working CXX compiler:
/home/vagrant/miniconda3/envs/test-conda/bin/x86_64-conda_cos6-linux-gnu-c++
-- Check for working CXX compiler:
/home/vagrant/miniconda3/envs/test-conda/bin/x86_64-conda_cos6-linux-gnu-c++
-- works
...
-- Found XercesC:
/home/vagrant/miniconda3/envs/test-conda/lib/libxerces-c.so (found version
"3.2.1")
-- Configuring done
-- Generating done
-- Build files have been written to:
/vagrant/conda-test/build-linux-conda
$ make -j1 VERBOSE=1
...
[100%] Linking CXX executable bar
/usr/bin/cmake -E cmake_link_script CMakeFiles/bar.dir/link.txt
--verbose=1
/home/vagrant/miniconda3/envs/test-conda/bin/x86_64-conda_cos6-linux-gnu-c++
-fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona
-mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2
-pipe -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now
CMakeFiles/bar.dir/bar.cpp.o -o bar libfoo.so
/home/vagrant/miniconda3/envs/test-conda/lib/libxerces-c.so
-Wl,-rpath,/vagrant/conda-test/build-linux-conda:/home/vagrant/miniconda3/envs/test-conda/lib:
...
You can see that the environment lib path is added to the RPATH by the link
command (unlike macOS):
$ ./bar
Hello World!
But when inspecting the dependencies, the environment lib path appears
twice:
$ readelf -d ./bar
...
0x0000000000000001 (NEEDED) Shared library: [libfoo.so]
0x0000000000000001 (NEEDED) Shared library: [
libxerces-c-3.2.so]
0x0000000000000001 (NEEDED) Shared library:
[libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000f (RPATH) Library rpath:
[/home/vagrant/miniconda3/envs/test-conda/lib:/vagrant/conda-test/build-linux-conda:/home/vagrant/miniconda3/envs/test-conda/lib:]
...
Which is wrong. Now the build tree binary will pick first any old version of
the foo library installed in the environment instead of the version in the
build tree.
It seems that the Anaconda toolchain is setting the environment lib path
into
the RPATH by its own. It is also set when installing the binaries, even when
CMake is not configured to set the install RPATH:
$ make install
...
Install the project...
-- Install configuration: ""
-- Installing: /home/vagrant/miniconda3/envs/test-conda/lib/libfoo.so
-- Installing: /home/vagrant/miniconda3/envs/test-conda/include/foo.hpp
-- Installing: /home/vagrant/miniconda3/envs/test-conda/bin/bar
-- Set runtime path of
"/home/vagrant/miniconda3/envs/test-conda/bin/bar" to ""
$ bar
Hello World!
$ readelf -d $CONDA_PREFIX/bin/bar
...
0x0000000000000001 (NEEDED) Shared library: [libfoo.so]
0x0000000000000001 (NEEDED) Shared library: [
libxerces-c-3.2.so]
0x0000000000000001 (NEEDED) Shared library:
[libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000f (RPATH) Library rpath:
[/home/vagrant/miniconda3/envs/test-conda/lib]
...
# deactivate the environment
$ conda deactivate
--------------------------------------------------
TL;DR I cannot get CMake and the Anaconda compilers and packages working
correctly.
- On macOS, the Conda environment library path is not added to the build
RPATH.
- On Linux, the Conda environment library path is always added to the RPATH
(in both build and install) independently of CMake.
Any advice or workarounds?
--
Sebastian Mancilla
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://cmake.org/pipermail/cmake/attachments/20180810/999b3788/attachment-0001.html>
More information about the CMake
mailing list