MantisBT - CMake
View Issue Details
0015769CMakeCMakepublic2015-10-05 10:372016-03-07 09:12
Ruslan Baratov 
Brad King 
normalminoralways
closedfixed 
Apple Mac
CMake 3.3.2 
CMake 3.5CMake 3.5 
0015769: OS X: Filesystem timestamp checks use only 1s resolution
`cmake --build` command doesn't trigger reconfiguration of the project on OS X when CMakeLists.txt changed.

Example:

add_executable(foo foo.cpp) # file foo.cpp exists

cmake -H. -B_builds
cmake --build _builds
# OK

change:
add_executable(foo foo.cpp boo.cpp) # file boo.cpp not exists
cmake --build _builds
# expected error, but no error reported

Ready-to-run example can be found: https://github.com/forexample/cmake-osx-no-reconfigure-bug [^]

Log from OS X machine:
* https://travis-ci.org/forexample/cmake-osx-no-reconfigure-bug/builds/83701171 [^]

Log for similar test on Linux machine:
* https://travis-ci.org/forexample/cmake-osx-no-reconfigure-bug/builds/83702953 [^]

CMake on Linux machine run reconfigure command and report an error:
  cmake -H. -B_builds --check-build-system CMakeFiles/Makefile.cmake 0
  -- Configuring done
  CMake Error at CMakeLists.txt:4 (add_executable):
    Cannot find source file:
      boo.cpp

same error expected on OS X machine
No tags attached.
Issue History
2015-10-05 10:37Ruslan BaratovNew Issue
2015-10-05 14:45Brad KingNote Added: 0039511
2015-10-05 14:52Brad KingNote Added: 0039512
2015-10-05 14:53Brad KingSummaryChange of CMakeLists.txt doesn't trigger reconfigure => OS X: Filesystem timestamp checks use only 1s resolution
2015-10-05 14:54Brad KingNote Added: 0039513
2015-10-06 13:27Brad KingNote Added: 0039518
2015-10-06 14:04Brad KingNote Added: 0039519
2015-10-08 14:01Brad KingNote Added: 0039548
2015-10-08 14:03Brad KingNote Added: 0039549
2015-10-08 14:03Brad KingAssigned To => Brad King
2015-10-08 14:03Brad KingStatusnew => resolved
2015-10-08 14:03Brad KingResolutionopen => fixed
2015-10-08 14:03Brad KingFixed in Version => CMake 3.5
2015-10-08 14:03Brad KingTarget Version => CMake 3.5
2015-10-09 17:41Ruslan BaratovNote Added: 0039558
2015-10-12 15:43Brad KingNote Added: 0039580
2015-10-12 18:18Ruslan BaratovNote Added: 0039582
2015-10-13 13:22Brad KingNote Added: 0039595
2015-10-13 14:48Ruslan BaratovNote Added: 0039596
2015-10-14 08:46Brad KingNote Added: 0039599
2015-10-15 12:33Brad KingNote Added: 0039618
2015-10-15 14:13Ruslan BaratovNote Added: 0039619
2015-10-15 15:01Brad KingNote Added: 0039620
2016-03-07 09:12Robert MaynardNote Added: 0040621
2016-03-07 09:12Robert MaynardStatusresolved => closed

Notes
(0039511)
Brad King   
2015-10-05 14:45   
I can reproduce this when running 'make' directly without 'cmake --build':

 -cmake --build _builds
 +(cd _builds; make)

The problem is that the filesystem and/or make tool seem to have 1s timestamp resolution. If I change the script to do "sleep 1" before the last "cp CMakeBad.txt CMakeLists.txt" then it works.
(0039512)
Brad King   
2015-10-05 14:52   
The issue may be where CMake decides whether it needs to re-run:

 https://cmake.org/gitweb?p=cmake.git;a=blob;f=Source/cmFileTimeComparison.cxx;hb=v3.3.2#l147 [^]

The STAT_HAS_ST_MTIM code path may not be taken on this platform.
(0039513)
Brad King   
2015-10-05 14:54   
Indeed, on OS X "struct stat" has:

 struct timespec st_mtimespec;

instead of

 struct timespec st_mtim;
(0039518)
Brad King   
2015-10-06 13:27   
For reference, I've started a fix for this on the KWSys side here:

 http://review.source.kitware.com/20258/ [^]
(0039519)
Brad King   
2015-10-06 14:04   
It looks like the underlying HFS filesystem only has 1s resolution:

 https://en.wikipedia.org/wiki/HFS_Plus [^]

This may not be possible to fix without a "sleep 1" in the test script.
(0039548)
Brad King   
2015-10-08 14:01   
After the KWSys side learned to use st_mtimespec I've updated CMake too:

 cmFileTimeComparison: Port to OS X nanosecond times
 https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=8d27b407 [^]
(0039549)
Brad King   
2015-10-08 14:03   
Marking as resolved because the proper API for ns resolution is now used. However in practice the underlying filesystem may limit the resolution anyway.
(0039558)
Ruslan Baratov   
2015-10-09 17:41   
I've built the latest version of CMake from branch 'next' (SHA1=5d28a728727caa1eb95e12e3cd99a203e07bcdac) but still experiencing this bug. See log: https://travis-ci.org/forexample/cmake-osx-no-reconfigure-bug/builds/84589416 [^]

I've added system timestamp messages (`stat -f %m CMakeLists.txt`) and they all differs:
  1444426214
  1444426217
  1444426219

> However in practice the underlying filesystem may limit the resolution anyway

From my opinion it would be nice to have some warning message about it. I understand that it's not a CMake flaw but this is a source of very strange behaviour.
(0039580)
Brad King   
2015-10-12 15:43   
Re 0015769:0039558: The times may differ from one another but not necessarily from the files generated.

Under what conditions could CMake possibly know to warn? We've tracked the problem down to the filesystem resolution. This could hit any tool e.g. "make". Now it is time to update your test scripts to sleep for 1s after each build is done before making any changes.
(0039582)
Ruslan Baratov   
2015-10-12 18:18   
> The times may differ from one another but not necessarily from the files generated

Okay, I see. The timestamp of Makefile generated from old CMakeLists.txt is equal to timestamp of new CMakeLists.txt indeed. Thanks for the clarification.

> Under what conditions could CMake possibly know to warn?

I guess it's possible to run a test on generate step. But warning message is quite useless if there is no option to apply a workaround.

> Now it is time to update your test scripts to sleep for 1s after each build is done before making any changes

Well it will not help since actually I hit this problem while doing manual updates. What about CMake/environment variable that will run sleep after the project generation done? Environment variable can be set system-wide to .bashrc and CMake variable can be set in toolchain.
(0039595)
Brad King   
2015-10-13 13:22   
> What about CMake/environment variable that will run sleep after the project generation done

This could be done by a wrapper script or shell alias meant to accommodate this filesystem limitation for your workflow.

I wouldn't expect every tool that can create a file to have an option to sleep after creating it.
(0039596)
Ruslan Baratov   
2015-10-13 14:48   
> This could be done by a wrapper script

This is true. Actually I already have a wrapper script which extends CMake's functionality and apply some workarounds, planning to add this one too. But in general what's the point of having hundreds of front-ends trying to solve the same problems in different ways instead of upgrading CMake? It's nothing special about a test-case (patch file and run build) or my file system (it's regular Apple's HFS and quite the same happens on virtual Travis CI instances).

> I wouldn't expect every tool that can create a file to have an option to sleep after creating it.

There is no need to sleep after each file creation it should be enough to sleep after generate/build step. I.e. in `cmake -H. -B_builds` and `cmake --build _builds` commands, but not in `cmake -E`. I mean I can handle it with Python script but it's not doable in C++? :)
(0039599)
Brad King   
2015-10-14 08:46   
I just remembered that the if() IS_NEWER_THAN test actually returns true if the files have the same time. This was done specifically for cases like 1s file time resolution. We could just do the same thing here fore deciding to re-run CMake. At worst it would re-run unnecessarily when the project configures in less than the filesystem time resolution but that is not too bad since that case is so fast.

Please try this patch:

diff --git a/Source/cmake.cxx b/Source/cmake.cxx
index 6846f1b..3988073 100644
--- a/Source/cmake.cxx
+++ b/Source/cmake.cxx
@@ -2061,7 +2061,7 @@ int cmake::CheckBuildSystem()
   if(!this->FileComparison->FileTimeCompare(out_oldest.c_str(),
                                             dep_newest.c_str(),
                                             &result) ||
- result < 0)
+ result <= 0)
     {
     if(verbose)
       {
(0039618)
Brad King   
2015-10-15 12:33   
Re 0015769:0039599: I tried this patch locally and it causes several failures in the test suite due to CMake re-running when it shouldn't. It also causes confusing behavior like re-running CMake on the first "make" after freshly generating a build tree.

I maintain that this is a problem with the HFS+ filesystem that warrants local workarounds for those users/projects affected by it (or a switch to NTFS). The problem affects every tool. Even build results from "make" are not reliable. Hacking CMake for the specific case discussed here is merely covering up one symptom of the larger problem.
(0039619)
Ruslan Baratov   
2015-10-15 14:13   
> I maintain that this is a problem with the HFS+ filesystem...

I think it's not. 1s resolution just make it more likely to occurs. Here is the result of running 'make' on Linux:

Test: https://github.com/forexample/date-resolution-test [^]
Log: https://travis-ci.org/forexample/date-resolution-test/builds/85021483 [^]

I don't know what file system is on Travis CI machines but I can reproduce it locally on my Linux with ext4. So experiencing such problems on other file systems just a matter of test and harware capabilities which lead to unpredictable/unrepeatable bugs and makes it VERY hard to debug (e.g. to fix first test it's not even enough to just re-run `cmake -H. -B_builds` - you should remove your build directory or touch files one more time after some pause). Again I understand that it's not a problem in CMake but if there is any change to apply some "guarding" functionality it should be used. I'll take a look more deeply next week and will report my thoughts.
(0039620)
Brad King   
2015-10-15 15:01   
Please move discussion over to the cmake developers list for a broader audience.
(0040621)
Robert Maynard   
2016-03-07 09:12   
Closing resolved issues that have not been updated in more than 4 months.