[CMake] Control the build target's external dependencies

[Attila Krasznahorkay]

Just to answer myself: After looking at the source code, I now understand that the regular expression set by include_regular_expression(...) is not compared against the full path name of the included file, but just what's in the source file after #include. So this machinery is non too useful for me at the moment.

For now I patched CMake locally with the following to ignore all out of source files in the dependency generation for me. With:

diff -u -r orig/cmake-3.4.1/Source/cmDepends.cxx cmake-3.4.1/Source/cmDepends.cxx
--- orig/cmake-3.4.1/Source/cmDepends.cxx	2015-12-02 16:43:21.000000000 +0100
+++ cmake-3.4.1/Source/cmDepends.cxx	2015-12-07 13:59:07.039749715 +0100
@@ -18,6 +18,7 @@
 #include "cmFileTimeComparison.h"
 #include <string.h>
 #include <cmsys/FStream.hxx>
+#include <algorithm>
 
 //----------------------------------------------------------------------------
 cmDepends::cmDepends(cmLocalGenerator* lg, const char* targetDir):
@@ -291,6 +292,13 @@
   return okay;
 }
 
+// Helper predicate for removing absolute paths from the include directory
+// list.
+bool IsAbsolutePath( const std::string& path )
+{
+  return cmSystemTools::FileIsFullPath( path );
+}
+
 //----------------------------------------------------------------------------
 void cmDepends::SetIncludePathFromLanguage(const std::string& lang)
 {
@@ -317,4 +325,8 @@
       cmSystemTools::ExpandListArgument(includePath, this->IncludePath);
       }
     }
+  // Remove all absolute paths from the dependency evaluation:
+  auto itr = std::remove_if( this->IncludePath.begin(), this->IncludePath.end(),
+                             IsAbsolutePath );
+  this->IncludePath.erase( itr, this->IncludePath.end() );
 }

(Could be implemented with slightly fewer lines as well.)

Which sped up the generation of these depend.make files immensely. So I wonder if turning on this behaviour could be a new thing in the future. As I imagine that it could be useful to a lot of people if it could be set via some property that only files from the source directory should be considered in the dependency calculation.

Should one open a bug report with this? (Does the CMake bug tracking system have the ability to track "feature requests"?)

Cheers,
           Attila

> On 07 Dec 2015, at 11:55, Attila Krasznahorkay <Attila.Krasznahorkay at gmail.com> wrote:
> 
> Dear All,
> 
> Maybe I should've google-d better. So now I started experimenting with include_regular_expression(...).
> 
> https://cmake.org/cmake/help/v3.0/command/include_regular_expression.html
> 
> But now I'm having a problem with expressing my intent with a regular expression. In principle I'd like to only take headers from the source directory into account in the dependency calculation. So I first tried this:
> 
> include_regular_expression( "^${CMAKE_SOURCE_DIR}.*$" )
> 
> But this was way too restrictive. As headers from the source directory normally show up with relative paths in the depend.make files.
> 
> So okay, I tried this next:
> 
> include_regular_expression( "^(${CMAKE_SOURCE_DIR}.*)|([^/]+.*)$" )
> 
> (To try to make it only accept paths that are either in CMAKE_SOURCE_DIR, or are not starting with "/".)
> 
> But this didn't work either. I'm back to having all the boost files in my dependency list.
> 
> Any suggestion for how I could express my intent in a regular expression that CMake would understand?
> 
> Cheers,
>         Attila
> 
>> On 07 Dec 2015, at 10:57, Attila Krasznahorkay <Attila.Krasznahorkay at gmail.com> wrote:
>> 
>> Dear All,
>> 
>> I'm struggling since a few days with the following issue.
>> 
>> Our development setup is such that we build large software projects in a nightly build system, which puts these projects onto a network drive. The developers then set up these nightly projects, and build their own code against them. (They can even rebuild parts that are in the nightly itself, that also required some clever tricks in CMake...)
>> 
>> This kind of works by now. But the system is incredibly slow. When I try to build some source code against installed releases on the network file system, >90% of the time is seemingly just spent in dependency calculation/evaluation. To demonstrate the extent of the issue, here is an example of the files generated by CMake for one of our "packages". (Sorry, they are pretty large. But that's the point...) Mind you, this is a very simple package that just picks up Boost from the network disk. For high level packages the depend.make file can be up to 2 MB in size. :-(
>> 
>> <depend.make><flags.make>
>> 
>> As you can see, I tried to convince CMake to treat the include directories coming from the network file system (AFS) as system include directories. But still, the dependencies put into depend.make list every single header file that the build targets have any relationship with.
>> 
>> As it turns out, AFS is pretty slow for such operations. Checking the change times of thousands of files. At least this is what I contribute the ridiculously slow build times to. (Not for this example. This package still builds reasonably. It's the higher level packages that break down completely. I just couldn't attach example files from those due to the size limitations of this mailing list.)
>> 
>> So... How could one convince CMake to exclude some directories from the dependency setup? The files that are part of the nightly builds should definitely not need to be checked for changes every time I run a build. As you can see, depend.make even list all the Boost headers at the moment. :-(
>> 
>> What I tried so far was to specify the include directories using SYSTEM in target_include_directories. But it didn't make any difference whether I used SYSTEM or not.
>> 
>> Is there some other mechanism that I could use here?
>> 
>> Cheers,
>>           Attila
>