[CMake] Configuring external projects with cmake for a nonexistent install directory which is not configure-time writable

[Michael Wild]

On 03/14/2011 03:37 PM, The Wanderer wrote:
> (I apologize for setting the Reply-To header, but I know of no other way
> to prevent my being sent an additional off-list copy of any reply even
> when there is no specific need to draw my attention to that reply.)
> 
> 
> 
> I am running cmake 2.8.2, installed via Debian testing. No more recent
> version appears to be available via Debian.
> 
> I am attempting to compile a project which has recently switched to
> cmake after a long time spent on autotools.
> 
> The project uses, and includes the source to, at least two other
> projects, which it treats as external projects.
> 
> The project explicitly requires (and enforces) an out-of-tree build.
> 
> I habitually install each new version of this particular project under
> its own brand-new prefix, to more easily retain previous versions for
> fallback if necessary. As a consequence, the target install prefix does
> not exist at the beginning of the build process.
> 
> I configure and compile as an ordinary user, which does not have write
> access to the install location. It has long been my understanding that
> this is ordinarily considered good practice.
> 
> I am attempting to configure with the command line
> 
> ----
> cmake /path/to/source -DDEBUG=1 -DPREFIX=/path/to/nonexistent/install/dir
> ----
> 
> With this command, the configure process fails. The error message is:
> 
> ----
> CMake Error at /usr/share/cmake-2.8/Modules/ExternalProject.cmake:504
> (file):
>   file problem creating directory: /path/to/nonexistent/install/dir
> Call Stack (most recent call first):
>   /usr/share/cmake-2.8/Modules/ExternalProject.cmake:1132
> (_ep_set_directories)
>   dep/tbb/CMakeLists.txt:69 (ExternalProject_Add)
> 
> 
> CMake Error at /usr/share/cmake-2.8/Modules/ExternalProject.cmake:506
> (message):
>   dir '/path/to/nonexistent/install/dir' does not exist after
> file(MAKE_DIRECTORY)
> Call Stack (most recent call first):
>   /usr/share/cmake-2.8/Modules/ExternalProject.cmake:1132
> (_ep_set_directories)
>   dep/tbb/CMakeLists.txt:69 (ExternalProject_Add)
> ----
> 
> Since the user running the cmake command does not have write access to
> the install location, it is of course not surprising that the
> MAKE_DIRECTORY attempt would fail. What is surprising is that cmake
> would even attempt to create the install directory at configure time.
> 
> 
> This attempt is made by the following code snippet, in
> _ep_set_directories from
> /usr/share/cmake-2.8/Modules/ExternalProject.cmake:
> 
> ----
>   # Make the directories at CMake configure time *and* add a custom command
>   # to make them at build time. They need to exist at makefile generation
>   # time for Borland make and wmake so that CMake may generate makefiles
>   # with "cd C:\short\paths\with\no\spaces" commands in them.
>   #
>   # Additionally, the add_custom_command is still used in case somebody
>   # removes one of the necessary directories and tries to rebuild without
>   # re-running cmake.
>   foreach(var ${places})
>     string(TOUPPER "${var}" VAR)
>     get_property(dir TARGET ${name} PROPERTY _EP_${VAR}_DIR)
>     file(MAKE_DIRECTORY "${dir}")
>     if(NOT EXISTS "${dir}")
>       message(FATAL_ERROR "dir '${dir}' does not exist after
> file(MAKE_DIRECTORY)")
>     endif()
>   endforeach()
> ----
> 
> (At a glance, the 'add_custom_command' referenced in the comment does
> not seem to be present, but that's a separate issue.)
> 
> The 'places' variable appears to be set via the line
> 
> ----
>   set(places stamp download source binary install tmp)
> ----
> 
> As a result, cmake unconditionally attempts to create at configure time
> any directories it expects to need for external projects at install
> time, for the sake of (as best I can tell) DOS-based build
> compatibility. Without write access to the install location at configure
> time, this will inherently fail, and the build therefore cannot succeed
> unless the directories already exist.
> 
> I have not been able to find any way to override this behavior. Looking
> at the code snippet involved, I don't see any obvious way for there to
> even currently be one.
> 
> I am not compiling in a DOS-related environment, and I am not using
> either Borland make or wmake, nor do I ever expect to do either for this
> project. (Insofar as I am aware, this project does not even use cmake
> for its Windows port.) I therefore doubt that I will ever need the short
> paths in question.
> 
> 
> I know of only three ways to potentially get this project to build under
> these circumstances:
> 
> * Run the configure process as a user with write access to the install
>   location, such as the user which will run the install. This is a bad
>   idea, for some of the same reasons you don't ordinarily want to
>   compile as root.
> 
> * Manually create the install directory in advance. This is the sort of
>   thing which should be automated, and should not need to be done by
>   hand.
> 
> * Comment out, delete, or otherwise disable the relevant FATAL_ERROR
>   and possibly MAKE_DIRECTORY lines from the system-wide
>   ExternalProject.cmake file, just for the sake of this one project's
>   build. This does not seem appropriate, even though I can't think of a
>   reason why it should ever lead to problems (with any project) in my
>   "GNU make and Linux"-only setup. At a minimum, it would probably have
>   to be re-fixed every time I install an updated version of cmake, since
>   this file would probably get overwritten.
> 
> None of these three approaches seem acceptable.
> 
> 
> Given that I know this directory creation attempt is not necessary for
> the build and will fail, is there a way to override it, so that cmake
> can proceed without manual intervention?
> 
> If so, how?
> 
> If not, I consider that a bug. Is there any chance of getting it fixed,
> so that either the process can succeed fully automatically, or at least
> I can explicitly tell cmake to skip attempting to create directories at
> configure time?
> 


This is a bug in the project, not in CMake. It uses the
ExternalProject.cmake module in a bogus way, probably by setting the
INSTALL_DIR option of ExternalProject_Add() to somewhere in the
CMAKE_INSTALL_PREFIX. This is wrong, it should be somewhere in the build
tree, and then the client project should install the necessary files itself.

Michael