View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0015178CMake(No Category)public2014-09-30 06:382015-05-04 09:05
ReporterChristophe Prud'homme 
Assigned ToClinton Stimpson 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformmacOSmavericksOS Version10.9
Product VersionCMake 3.0.2 
Target VersionFixed in Version 
Summary0015178: CMake very slow at generation stage
Descriptionon macosx the generation stage is very slow and it seems that it is the dependencies computation.
On our projet (http://www.feelpp.org [^]) is takes minutes for the generation stage while it took seconds in the previous version.
On Linux no such issues.
It seems that there are a lot of call to otool on macosx and it might be that that slows down thinks
TagsNo tags attached.
Attached Files

 Relationships

  Notes
(0037025)
Christophe Prud'homme (reporter)
2014-10-13 23:39

 could it be that computing the rpath in cmake 3 on macosx takes a long time since MACOSX_RPATH is on by default ? otool seems to be called a lot on our code
(0037027)
Brad King (manager)
2014-10-14 09:17

 > it took seconds in the previous version

What was the previous version you tested?

Also, which generator are you using? Makefile? Xcode? Ninja?
(0037028)
Clinton Stimpson (developer)
2014-10-14 09:57

 The otool calls are made independent of the MACOSX_RPATH setting. Those calls are done on libraries without a known install name.
Also, these same otool calls were made in CMake 2.8.12. How was the speed in that version?
(0037046)
Christophe Prud'homme (reporter)
2014-10-18 15:52

 On linux (cmake 2.x and cmake3) it takes a few seconds to do the generation. On MACOSX with cmake 3 it takes minutes while with cmake 2.x (eg x=2.8 or 2.9) it was similar(about same order) to linux.
I can try to revert back to cmaae 2.8 and check how it was exactely.

I am using the Makefile generator. Our code makes Xcode crash while indexing it (too many templates probably).
(0037474)
Christian Weigel (reporter)
2014-12-17 10:34

 We've experienced the same problem with both 2.8.12 and 3.x. It was much faster in previous versions (e.g. 2.8.9). Could you elaborate a bit more on why those calls to otool are done and if there is any way to turn that off. We found an inofficial solution setting the undocumented variable CMAKE_PLATFORM_HAS_INSTALLNAME to Off. Then the generation process runs fast again but it causes a different problem. We set (also in earlier version)

set( CMAKE_BUILD_WITH_INSTALL_RPATH ON )
set( CMAKE_INSTALL_NAME_DIR "@executable_path" )

in order to insert that information into the dylibs which worked fine for us and what obviously doesn't work anymore then.
(0037476)
Clinton Stimpson (developer)
2014-12-17 14:00

 I'm not seeing slow otool performance.

Consider the below case of running otool on all the *.dylib files in /usr/lib.
I have 494 of them, and it takes 0.026 seconds to do all of them.

$ which otool
/Applications/Xcode-4.6.app/Contents/Developer/usr/bin/otool

$ time otool -D /usr/lib/*.dylib | wc -l
     494

real 0m0.026s
user 0m0.010s
sys 0m0.018s


I just switched to /usr/bin/otool and get longer times. First time, its 3.5 seconds.
$ time /usr/bin/otool -D /usr/lib/*.dylib | wc -l
     494

real 0m3.508s
user 0m0.381s
sys 0m0.317s

Second time I run it, its shorter.
$ time /usr/bin/otool -D /usr/lib/*.dylib | wc -l
     494

real 0m0.089s
user 0m0.012s
sys 0m0.021s

I get shorter times for anything after that.

Here's another one:
$ xcrun -f otool
/Applications/Xcode-4.6.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/otool

$ time xcrun -r otool -D /usr/lib/*.dylib | wc -l
     494

real 0m0.030s
user 0m0.011s
sys 0m0.021s



Christian and Christophe, can you try the different approaches and report your times here?
(0037477)
Clinton Stimpson (developer)
2014-12-17 14:02

 It would also be nice if someone could provide a relatively small example to reproduce the problem.
(0037495)
Christian Weigel (reporter)
2014-12-19 07:30

 Hi Clinton,

thank you for taking this. We made some experiments. Firstly, we did the time meaursuring of /usr/bin/otool and came to the same results as you did. (We have XCode 5.1.1 installed and did not find any otool there). We've also made up a minimal example showing the effect. It seems that if a target depends on some external system libraries (boost and some other in the example) otool is invoked. When doing this on a large project with a lot of targets that do have a lot of dependencies (that, in our case are located at a network share) this leads to very long generation times. When doing set( CMAKE_PLATFORM_HAS_INSTALLNAME OFF) the time decreases. Thus the question is the invokation of otool really required on a target basis or could the check (what is actually checked??) be done once for an external dependency per generation run? Here's the example which takes ~16s (CMAKE_PLATFORM_HAS_INSTALLNAME OFF) vs ~54s.


+++++++++++++ CMakeLists.txt +++++++++++++++++++

cmake_minimum_required(VERSION 2.8.12)

set( CMAKE_BUILD_WITH_INSTALL_RPATH ON )
set( CMAKE_INSTALL_NAME_DIR "@executable_path" )
#set( CMAKE_PLATFORM_HAS_INSTALLNAME OFF )

find_package(Boost COMPONENTS atomic chrono context date_time filesystem iostreams log regex serialization system thread timer unit_test_framework REQUIRED)

if (APPLE)
  find_library( CORE_SERVICES CoreServices )
  find_library( CORE_AUDIO CoreAudio )
  find_library( CORE_FOUNDATION CoreFoundation )
  find_library( CORE_VIDEO CoreVideo )
  find_library( VIDEO_DECODE_ACCELERATION VideoDecodeAcceleration )
  find_library( CARBON_LIBRARY Carbon )
  find_library( QUICKTIME_LIBRARY QuickTime )
  find_library( APP_SERVICES_LIBRARY ApplicationServices )
endif (APPLE)

set(counter 0)
while(counter LESS 500)
  math(EXPR counter "${counter} + 1")
  add_library(foo${counter} SHARED foo.cpp)
  target_link_libraries(foo${counter} PRIVATE
    ${CORE_SERVICES} ${CORE_AUDIO} ${CORE_FOUNDATION} ${CORE_VIDEO} ${VIDEO_DECODE_ACCELERATION} ${CARBON_LIBRARY} ${QUICKTIME_LIBRARY} ${APP_SERVICES_LIBRARY}
    ${Boost_LIBRARIES}
  )
  set_target_properties(foo${counter} PROPERTIES COMPILE_FLAGS -DLIBNAME="foo${counter}")
endwhile()


+++++++++++++ foo.cpp +++++++++++++++++++

#ifndef LIBNAME
#define LIBNAME

#include <iostream>

class LIBNAME
{
public:
  LIBNAME () {
    std::cout << "Constructor of " << LIBNAME << std::endl;
  };
  ~LIBNAME () {};
};

#endif
(0037503)
Clinton Stimpson (developer)
2014-12-19 14:56

 This otool work is done to support @rpath.

There is an alternative I'm working on which I've pushed to the stage repository.
For the tests I have, the configure time was back down to what I'm getting with CMake 2.8.11.
(0037508)
Clinton Stimpson (developer)
2014-12-22 13:39

 Resolved by adding a Mach-O parser similar to the ELF parser CMake already had.

c294a115 Mach-O: Use Mach-O parser to extract install names instead of otool.
e42da307 Mach-O: Add Mach-O parser for OS X and iOS.

From my testing with a few hundred link libraries, the configure times are back to CMake 2.8.11 times.
I did not check with a network share.
(0037509)
Brad King (manager)
2014-12-22 13:41

 Re 0015178:0037508: Thanks. Links to those commits:

 Mach-O: Add Mach-O parser for OS X and iOS.
 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=e42da307 [^]

 Mach-O: Use Mach-O parser to extract install names instead of otool.
 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=c294a115 [^]
(0038705)
Robert Maynard (manager)
2015-05-04 09:05

 Closing resolved issues that have not been updated in more than 4 months.

 Issue History
Date Modified Username Field Change
2014-09-30 06:38 Christophe Prud'homme New Issue
2014-10-13 23:39 Christophe Prud'homme Note Added: 0037025
2014-10-14 09:17 Brad King Note Added: 0037027
2014-10-14 09:57 Clinton Stimpson Note Added: 0037028
2014-10-18 15:52 Christophe Prud'homme Note Added: 0037046
2014-12-17 10:34 Christian Weigel Note Added: 0037474
2014-12-17 14:00 Clinton Stimpson Note Added: 0037476
2014-12-17 14:02 Clinton Stimpson Note Added: 0037477
2014-12-19 07:30 Christian Weigel Note Added: 0037495
2014-12-19 14:56 Clinton Stimpson Note Added: 0037503
2014-12-22 13:39 Clinton Stimpson Note Added: 0037508
2014-12-22 13:39 Clinton Stimpson Status new => resolved
2014-12-22 13:39 Clinton Stimpson Resolution open => fixed
2014-12-22 13:39 Clinton Stimpson Assigned To => Clinton Stimpson
2014-12-22 13:41 Brad King Note Added: 0037509
2015-05-04 09:05 Robert Maynard Note Added: 0038705
2015-05-04 09:05 Robert Maynard Status resolved => closed

Copyright © 2000 - 2018 MantisBT Team