(0039511)
Brad King (manager)
2015-10-05 14:45

I can reproduce this when running 'make' directly without 'cmake --build':

-cmake --build _builds
+(cd _builds; make)

The problem is that the filesystem and/or make tool seem to have 1s timestamp resolution. If I change the script to do "sleep 1" before the last "cp CMakeBad.txt CMakeLists.txt" then it works.

(0039512)
Brad King (manager)
2015-10-05 14:52

The issue may be where CMake decides whether it needs to re-run:

https://cmake.org/gitweb?p=cmake.git;a=blob;f=Source/cmFileTimeComparison.cxx;hb=v3.3.2#l147 [^]

The STAT_HAS_ST_MTIM code path may not be taken on this platform.

(0039513)
Brad King (manager)
2015-10-05 14:54

Indeed, on OS X "struct stat" has:

struct timespec st_mtimespec;

instead of

struct timespec st_mtim;

(0039518)
Brad King (manager)
2015-10-06 13:27

For reference, I've started a fix for this on the KWSys side here:

http://review.source.kitware.com/20258/ [^]

(0039519)
Brad King (manager)
2015-10-06 14:04

It looks like the underlying HFS filesystem only has 1s resolution:

https://en.wikipedia.org/wiki/HFS_Plus [^]

This may not be possible to fix without a "sleep 1" in the test script.

(0039548)
Brad King (manager)
2015-10-08 14:01

After the KWSys side learned to use st_mtimespec I've updated CMake too:

cmFileTimeComparison: Port to OS X nanosecond times
https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=8d27b407 [^]

(0039549)
Brad King (manager)
2015-10-08 14:03

Marking as resolved because the proper API for ns resolution is now used. However in practice the underlying filesystem may limit the resolution anyway.

(0039558)
Ruslan Baratov (reporter)
2015-10-09 17:41

I've built the latest version of CMake from branch 'next' (SHA1=5d28a728727caa1eb95e12e3cd99a203e07bcdac) but still experiencing this bug. See log: https://travis-ci.org/forexample/cmake-osx-no-reconfigure-bug/builds/84589416 [^]

I've added system timestamp messages (`stat -f %m CMakeLists.txt`) and they all differs:
  1444426214
  1444426217
  1444426219

> However in practice the underlying filesystem may limit the resolution anyway

From my opinion it would be nice to have some warning message about it. I understand that it's not a CMake flaw but this is a source of very strange behaviour.

(0039580)
Brad King (manager)
2015-10-12 15:43

Re 0015769:0039558: The times may differ from one another but not necessarily from the files generated.

Under what conditions could CMake possibly know to warn? We've tracked the problem down to the filesystem resolution. This could hit any tool e.g. "make". Now it is time to update your test scripts to sleep for 1s after each build is done before making any changes.

(0039582)
Ruslan Baratov (reporter)
2015-10-12 18:18

> The times may differ from one another but not necessarily from the files generated

Okay, I see. The timestamp of Makefile generated from old CMakeLists.txt is equal to timestamp of new CMakeLists.txt indeed. Thanks for the clarification.

> Under what conditions could CMake possibly know to warn?

I guess it's possible to run a test on generate step. But warning message is quite useless if there is no option to apply a workaround.

> Now it is time to update your test scripts to sleep for 1s after each build is done before making any changes

Well it will not help since actually I hit this problem while doing manual updates. What about CMake/environment variable that will run sleep after the project generation done? Environment variable can be set system-wide to .bashrc and CMake variable can be set in toolchain.

(0039595)
Brad King (manager)
2015-10-13 13:22

> What about CMake/environment variable that will run sleep after the project generation done

This could be done by a wrapper script or shell alias meant to accommodate this filesystem limitation for your workflow.

I wouldn't expect every tool that can create a file to have an option to sleep after creating it.

(0039596)
Ruslan Baratov (reporter)
2015-10-13 14:48

> This could be done by a wrapper script

This is true. Actually I already have a wrapper script which extends CMake's functionality and apply some workarounds, planning to add this one too. But in general what's the point of having hundreds of front-ends trying to solve the same problems in different ways instead of upgrading CMake? It's nothing special about a test-case (patch file and run build) or my file system (it's regular Apple's HFS and quite the same happens on virtual Travis CI instances).

> I wouldn't expect every tool that can create a file to have an option to sleep after creating it.

There is no need to sleep after each file creation it should be enough to sleep after generate/build step. I.e. in `cmake -H. -B_builds` and `cmake --build _builds` commands, but not in `cmake -E`. I mean I can handle it with Python script but it's not doable in C++? :)

(0039599)
Brad King (manager)
2015-10-14 08:46

I just remembered that the if() IS_NEWER_THAN test actually returns true if the files have the same time. This was done specifically for cases like 1s file time resolution. We could just do the same thing here fore deciding to re-run CMake. At worst it would re-run unnecessarily when the project configures in less than the filesystem time resolution but that is not too bad since that case is so fast.

Please try this patch:

diff --git a/Source/cmake.cxx b/Source/cmake.cxx
index 6846f1b..3988073 100644
--- a/Source/cmake.cxx
+++ b/Source/cmake.cxx
@@ -2061,7 +2061,7 @@ int cmake::CheckBuildSystem()
   if(!this->FileComparison->FileTimeCompare(out_oldest.c_str(),
                                             dep_newest.c_str(),
                                             &result) ||
- result < 0)
+ result <= 0)
     {
     if(verbose)
       {

(0039618)
Brad King (manager)
2015-10-15 12:33

Re 0015769:0039599: I tried this patch locally and it causes several failures in the test suite due to CMake re-running when it shouldn't. It also causes confusing behavior like re-running CMake on the first "make" after freshly generating a build tree.

I maintain that this is a problem with the HFS+ filesystem that warrants local workarounds for those users/projects affected by it (or a switch to NTFS). The problem affects every tool. Even build results from "make" are not reliable. Hacking CMake for the specific case discussed here is merely covering up one symptom of the larger problem.

(0039619)
Ruslan Baratov (reporter)
2015-10-15 14:13

> I maintain that this is a problem with the HFS+ filesystem...

I think it's not. 1s resolution just make it more likely to occurs. Here is the result of running 'make' on Linux:

Test: https://github.com/forexample/date-resolution-test [^]
Log: https://travis-ci.org/forexample/date-resolution-test/builds/85021483 [^]

I don't know what file system is on Travis CI machines but I can reproduce it locally on my Linux with ext4. So experiencing such problems on other file systems just a matter of test and harware capabilities which lead to unpredictable/unrepeatable bugs and makes it VERY hard to debug (e.g. to fix first test it's not even enough to just re-run `cmake -H. -B_builds` - you should remove your build directory or touch files one more time after some pause). Again I understand that it's not a problem in CMake but if there is any change to apply some "guarding" functionality it should be used. I'll take a look more deeply next week and will report my thoughts.

(0039620)
Brad King (manager)
2015-10-15 15:01

Please move discussion over to the cmake developers list for a broader audience.

(0040621)
Robert Maynard (manager)
2016-03-07 09:12

Closing resolved issues that have not been updated in more than 4 months.

Notes
(0039511) Brad King (manager) 2015-10-05 14:45	I can reproduce this when running 'make' directly without 'cmake --build': -cmake --build _builds +(cd _builds; make) The problem is that the filesystem and/or make tool seem to have 1s timestamp resolution. If I change the script to do "sleep 1" before the last "cp CMakeBad.txt CMakeLists.txt" then it works.

(0039512) Brad King (manager) 2015-10-05 14:52	The issue may be where CMake decides whether it needs to re-run: https://cmake.org/gitweb?p=cmake.git;a=blob;f=Source/cmFileTimeComparison.cxx;hb=v3.3.2#l147 [^] The STAT_HAS_ST_MTIM code path may not be taken on this platform.

(0039513) Brad King (manager) 2015-10-05 14:54	Indeed, on OS X "struct stat" has: struct timespec st_mtimespec; instead of struct timespec st_mtim;

(0039518) Brad King (manager) 2015-10-06 13:27	For reference, I've started a fix for this on the KWSys side here: http://review.source.kitware.com/20258/ [^]

(0039519) Brad King (manager) 2015-10-06 14:04	It looks like the underlying HFS filesystem only has 1s resolution: https://en.wikipedia.org/wiki/HFS_Plus [^] This may not be possible to fix without a "sleep 1" in the test script.

(0039548) Brad King (manager) 2015-10-08 14:01	After the KWSys side learned to use st_mtimespec I've updated CMake too: cmFileTimeComparison: Port to OS X nanosecond times https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=8d27b407 [^]

(0039549) Brad King (manager) 2015-10-08 14:03	Marking as resolved because the proper API for ns resolution is now used. However in practice the underlying filesystem may limit the resolution anyway.

(0039558) Ruslan Baratov (reporter) 2015-10-09 17:41	I've built the latest version of CMake from branch 'next' (SHA1=5d28a728727caa1eb95e12e3cd99a203e07bcdac) but still experiencing this bug. See log: https://travis-ci.org/forexample/cmake-osx-no-reconfigure-bug/builds/84589416 [^] I've added system timestamp messages (`stat -f %m CMakeLists.txt`) and they all differs: 1444426214 1444426217 1444426219 > However in practice the underlying filesystem may limit the resolution anyway From my opinion it would be nice to have some warning message about it. I understand that it's not a CMake flaw but this is a source of very strange behaviour.

(0039580) Brad King (manager) 2015-10-12 15:43	Re 0015769:0039558: The times may differ from one another but not necessarily from the files generated. Under what conditions could CMake possibly know to warn? We've tracked the problem down to the filesystem resolution. This could hit any tool e.g. "make". Now it is time to update your test scripts to sleep for 1s after each build is done before making any changes.

(0039582) Ruslan Baratov (reporter) 2015-10-12 18:18	> The times may differ from one another but not necessarily from the files generated Okay, I see. The timestamp of Makefile generated from old CMakeLists.txt is equal to timestamp of new CMakeLists.txt indeed. Thanks for the clarification. > Under what conditions could CMake possibly know to warn? I guess it's possible to run a test on generate step. But warning message is quite useless if there is no option to apply a workaround. > Now it is time to update your test scripts to sleep for 1s after each build is done before making any changes Well it will not help since actually I hit this problem while doing manual updates. What about CMake/environment variable that will run sleep after the project generation done? Environment variable can be set system-wide to .bashrc and CMake variable can be set in toolchain.

(0039595) Brad King (manager) 2015-10-13 13:22	> What about CMake/environment variable that will run sleep after the project generation done This could be done by a wrapper script or shell alias meant to accommodate this filesystem limitation for your workflow. I wouldn't expect every tool that can create a file to have an option to sleep after creating it.

(0039596) Ruslan Baratov (reporter) 2015-10-13 14:48	> This could be done by a wrapper script This is true. Actually I already have a wrapper script which extends CMake's functionality and apply some workarounds, planning to add this one too. But in general what's the point of having hundreds of front-ends trying to solve the same problems in different ways instead of upgrading CMake? It's nothing special about a test-case (patch file and run build) or my file system (it's regular Apple's HFS and quite the same happens on virtual Travis CI instances). > I wouldn't expect every tool that can create a file to have an option to sleep after creating it. There is no need to sleep after each file creation it should be enough to sleep after generate/build step. I.e. in `cmake -H. -B_builds` and `cmake --build _builds` commands, but not in `cmake -E`. I mean I can handle it with Python script but it's not doable in C++? :)

(0039599) Brad King (manager) 2015-10-14 08:46	I just remembered that the if() IS_NEWER_THAN test actually returns true if the files have the same time. This was done specifically for cases like 1s file time resolution. We could just do the same thing here fore deciding to re-run CMake. At worst it would re-run unnecessarily when the project configures in less than the filesystem time resolution but that is not too bad since that case is so fast. Please try this patch: diff --git a/Source/cmake.cxx b/Source/cmake.cxx index 6846f1b..3988073 100644 --- a/Source/cmake.cxx +++ b/Source/cmake.cxx @@ -2061,7 +2061,7 @@ int cmake::CheckBuildSystem() if(!this->FileComparison->FileTimeCompare(out_oldest.c_str(), dep_newest.c_str(), &result) \|\| - result < 0) + result <= 0) { if(verbose) {

(0039618) Brad King (manager) 2015-10-15 12:33	Re 0015769:0039599: I tried this patch locally and it causes several failures in the test suite due to CMake re-running when it shouldn't. It also causes confusing behavior like re-running CMake on the first "make" after freshly generating a build tree. I maintain that this is a problem with the HFS+ filesystem that warrants local workarounds for those users/projects affected by it (or a switch to NTFS). The problem affects every tool. Even build results from "make" are not reliable. Hacking CMake for the specific case discussed here is merely covering up one symptom of the larger problem.

(0039619) Ruslan Baratov (reporter) 2015-10-15 14:13	> I maintain that this is a problem with the HFS+ filesystem... I think it's not. 1s resolution just make it more likely to occurs. Here is the result of running 'make' on Linux: Test: https://github.com/forexample/date-resolution-test [^] Log: https://travis-ci.org/forexample/date-resolution-test/builds/85021483 [^] I don't know what file system is on Travis CI machines but I can reproduce it locally on my Linux with ext4. So experiencing such problems on other file systems just a matter of test and harware capabilities which lead to unpredictable/unrepeatable bugs and makes it VERY hard to debug (e.g. to fix first test it's not even enough to just re-run `cmake -H. -B_builds` - you should remove your build directory or touch files one more time after some pause). Again I understand that it's not a problem in CMake but if there is any change to apply some "guarding" functionality it should be used. I'll take a look more deeply next week and will report my thoughts.

(0039620) Brad King (manager) 2015-10-15 15:01	Please move discussion over to the cmake developers list for a broader audience.

(0040621) Robert Maynard (manager) 2016-03-07 09:12	Closing resolved issues that have not been updated in more than 4 months.

Issue History
Date Modified	Username	Field	Change
2015-10-05 10:37	Ruslan Baratov	New Issue
2015-10-05 14:45	Brad King	Note Added: 0039511
2015-10-05 14:52	Brad King	Note Added: 0039512
2015-10-05 14:53	Brad King	Summary	Change of CMakeLists.txt doesn't trigger reconfigure => OS X: Filesystem timestamp checks use only 1s resolution
2015-10-05 14:54	Brad King	Note Added: 0039513
2015-10-06 13:27	Brad King	Note Added: 0039518
2015-10-06 14:04	Brad King	Note Added: 0039519
2015-10-08 14:01	Brad King	Note Added: 0039548
2015-10-08 14:03	Brad King	Note Added: 0039549
2015-10-08 14:03	Brad King	Assigned To	=> Brad King
2015-10-08 14:03	Brad King	Status	new => resolved
2015-10-08 14:03	Brad King	Resolution	open => fixed
2015-10-08 14:03	Brad King	Fixed in Version	=> CMake 3.5
2015-10-08 14:03	Brad King	Target Version	=> CMake 3.5
2015-10-09 17:41	Ruslan Baratov	Note Added: 0039558
2015-10-12 15:43	Brad King	Note Added: 0039580
2015-10-12 18:18	Ruslan Baratov	Note Added: 0039582
2015-10-13 13:22	Brad King	Note Added: 0039595
2015-10-13 14:48	Ruslan Baratov	Note Added: 0039596
2015-10-14 08:46	Brad King	Note Added: 0039599
2015-10-15 12:33	Brad King	Note Added: 0039618
2015-10-15 14:13	Ruslan Baratov	Note Added: 0039619
2015-10-15 15:01	Brad King	Note Added: 0039620
2016-03-07 09:12	Robert Maynard	Note Added: 0040621
2016-03-07 09:12	Robert Maynard	Status	resolved => closed

Relationships