View Issue Details [ Jump to Notes ]

[ Print ]

Project

Category

View Status

Date Submitted

Last Update

0015314

CMake

public

2014-12-17 11:29

2016-06-10 14:31

Reporter

Kelly Thompson

Assigned To

Kitware Robot

Priority

normal

Severity

minor

Reproducibility

always

Status

closed

Resolution

moved

Platform

Linux

any

OS Version

any

Product Version

CMake 3.0.2

Target Version

Fixed in Version

Summary

0015314: FindMPI.cmake can choose incompatible set of libraries for Intel MPI

Description

When using Intel MPI, CMake's FindMPI.cmake can choose the wrong MPI libraries for linking because it does not differentiate between the regular and the thread safe versions of the libraries (this might be related to http://public.kitware.com/Bug/view.php?id=14991 [^]).

The problem can been observed by noting that FindMPI.cmake calls '@mpiicpc -show@' to determine the MPI libraries that should be used during linking (these will not be the thread safe versions). For example:

% mpiicpc -show
icpc -I/projects/opt/mpi/impi/5.0.1.035/intel64/include \
-L/projects/opt/mpi/impi/5.0.1.035/intel64/lib/release \
-L/projects/opt/mpi/impi/5.0.1.035/intel64/lib \
-Xlinker --enable-new-dtags \
-Xlinker -rpath \
-Xlinker /projects/opt/mpi/impi/5.0.1.035/intel64/lib/release \
-Xlinker -rpath -Xlinker /projects/opt/mpi/impi/5.0.1.035/intel64/lib \
-Xlinker -rpath -Xlinker /opt/intel/mpi-rt/5.0/intel64/lib/release \
-Xlinker -rpath -Xlinker /opt/intel/mpi-rt/5.0/intel64/lib \
-lmpicxx -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread

However, if thread safe versions are desired, the command 'mpiicpc -mt_mpi -show' should be used instead. Note that adding '-openmp' (my situation) will also select the thread safe versions. These libraries are:

% mpiicpc -mt_mpi -show
icpc -I/projects/opt/mpi/impi/5.0.1.035/intel64/include \
-L/projects/opt/mpi/impi/5.0.1.035/intel64/lib/release_mt \
-L/projects/opt/mpi/impi/5.0.1.035/intel64/lib \
-Xlinker --enable-new-dtags \
-Xlinker -rpath -Xlinker /projects/opt/mpi/impi/5.0.1.035/intel64/lib/release_mt \
-Xlinker -rpath -Xlinker /projects/opt/mpi/impi/5.0.1.035/intel64/lib \
-Xlinker -rpath -Xlinker /opt/intel/mpi-rt/5.0/intel64/lib/release_mt \
-Xlinker -rpath -Xlinker /opt/intel/mpi-rt/5.0/intel64/lib \
-lmpicxx -lmpifort -lmpi_mt -lmpigi -ldl -lrt -lpthread

The difference is the library folder location: 'release' vs. 'relase_mt'

Currently, when the CMAKE_<LANG>_COMPILER == MPI_<LANG>_COMPILER, FindMPI.cmake will always add the non-thread safe library to the link line via ${MPI_<LANG>_LIBRARIES} so that if the compiler flags include '-mt_mpi' or '-openmp', the link line will include both the thread safe libraries and the normal libraries resulting in:

ld: MPIR_Thread: TLS definition in .../intel64/lib/release_mt/libmpi_mt.so section .tbss mismatches non-TLS definition in .../intel64/lib/libmpi.so section .bss
.../intel64/lib/release_mt/libmpi_mt.so: could not read symbols: Bad value

I think the correct resolution to this issue would be to disable the interrogation of the wrapper when CMAKE_<LANG>_COMPILER == MPI_<LANG>_COMPILER because all of the required include and link information will already be available via the MPI compiler wrapper. In this model, MPI_<LANG>_LIBRARIES would be an empty string if the mpi compiler wrapper is used as the project primary compiler.

A potential fix would be a modification of some sample code that I recently proposed via the cmake-developers mailing list:

--- a/Modules/FindMPI.cmake
+++ b/Modules/FindMPI.cmake
@@ -165,11 +165,28 @@ foreach (lang C CXX Fortran)
     endif()
     unset(_MPI_${id}_${lang}_COMPILER_NAMES) # clean up the namespace here
   endforeach()
+
+ # If cmake_$lang_compiler matches a known mpi compiler wrapper name,
+ # prefer the provided name and prevent interrogation.
+ get_filename_component( compiler_wo_path "${CMAKE_${lang}_COMPILER}" NAME )
+ set( ${lang}_compiler_is_mpiwrapper false )
+ foreach( mpiwrapper ${_MPI_${lang}_COMPILER_NAMES} )
+ if( "${mpiwrapper}" STREQUAL "${compiler_wo_path}" )
+ set( ${lang}_compiler_is_mpiwrapper true )
+ endif()
+ endforeach()
+ if( ${lang}_compiler_is_mpiwrapper )
+ set( MPI_${lang}_COMPILER ${CMAKE_${lang}_COMPILER} )
+ set( MPI_${lang}_NO_INTERROGATE ${CMAKE_${lang}_COMPILER} )
+ endif()
endforeach()

Steps To Reproduce

On Linux with Intel/15.0 and Intel MPI 5.0 (my testing used cmake/3.0.1 and the git source version).

export CXX=`which mpiicpc`
export CC=`which mpiicc`
cmake <dir>

<dir> has 2 files: CMakeLists.txt and hw.cc

# CMakeLists.txt
cmake_minimum_required(VERSION 3.0)
project( hw_mpi CXX )
set( CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -mt_mpi" )
find_package( MPI )
include_directories( ${MPI_INCLUDE_PATH} )
add_executable( hw hw.cc )
target_link_libraries( hw ${MPI_LIBRARIES} )

// hw.cc
#include <mpi.h>
#include <iostream>
int main(int argc, char* argv[])
{
    MPI_Init(&argc, &argv);
    int rank;
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    if (rank == 0)
    {
        int value = 17;
        int result = MPI_Send(&value, 1, MPI_INT, 1, 0, MPI_COMM_WORLD);
        if (result == MPI_SUCCESS)
            std::cout << "Rank 0 OK!" << std::endl;
    }
    else if (rank == 1)
    {
        int value;
        int result = MPI_Recv(&value, 1, MPI_INT, 0, 0, MPI_COMM_WORLD,
                              MPI_STATUS_IGNORE);
        if (result == MPI_SUCCESS && value == 17)
            std::cout << "Rank 1 OK!" << std::endl;
    }
    MPI_Finalize();
    return 0;
}

Additionally, a short test like this should probably be added to the standard suite of CMake regression tests.

Additional Information

This issue is possibly related to http://public.kitware.com/Bug/view.php?id=15182 [^] and http://public.kitware.com/Bug/view.php?id=14991. [^]

Issue History
Date Modified	Username	Field	Change
2014-12-17 11:29	Kelly Thompson	New Issue
2016-06-10 14:29	Kitware Robot	Note Added: 0042690
2016-06-10 14:29	Kitware Robot	Status	new => resolved
2016-06-10 14:29	Kitware Robot	Resolution	open => moved
2016-06-10 14:29	Kitware Robot	Assigned To	=> Kitware Robot
2016-06-10 14:31	Kitware Robot	Status	resolved => closed

Notes
(0042690) Kitware Robot (administrator) 2016-06-10 14:29	Resolving issue as `moved`. This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.

Relationships