MantisBT - CMake
View Issue Details
0005939CMakeCMakepublic2007-10-23 20:592016-06-10 14:30
Brandon Van Every 
Bill Hoffman 
normalmajoralways
closedmoved 
 
 
0005939: TO_NATIVE_PATH incorrect on MinGW
A CMakeLists.txt containing:

FILE(TO_NATIVE_PATH ${CMAKE_CURRENT_SOURCE_DIR} native_srcdir)
MESSAGE("${native_srcdir}")

produces the following output for a "Visual Studio 8 2005" generator:

ERROR
C:\devel\src\bugs\native
(Press Cancel to suppress further error messages.)

which is the correct native path we'd expect. However a "MinGW Makefiles" generator produces:

ERROR
C:/devel/src/bugs/native
(Press Cancel to suppress further error messages.)

This is wrong. The "MinGW Makefiles" generator is intended to be used by mingw32-make.exe under a Windows Command Prompt. This can only imply that "native" paths are backslashed Windows paths, not forward slashed CMake / Unix paths.
No tags attached.
related to 0015509closed Kitware Robot CMake needs a generator expression equivalent to TO_NATIVE_PATH. 
Issue History
2007-10-23 20:59Brandon Van EveryNew Issue
2007-12-17 17:06Bill HoffmanStatusnew => assigned
2007-12-17 17:06Bill HoffmanAssigned To => Bill Hoffman
2010-07-13 03:14jorgenysNote Added: 0021372
2010-09-29 15:38EricGNote Added: 0022385
2011-09-14 07:11Thomas ThorsenNote Added: 0027415
2012-04-13 18:56Daniel FrankeNote Added: 0029155
2014-07-09 08:45Johannes S. Mueller-RoemerNote Added: 0036330
2014-07-22 11:14Bill HoffmanNote Added: 0036427
2014-07-22 13:34Brad KingNote Added: 0036428
2014-07-22 13:34Brad KingNote Edited: 0036428bug_revision_view_page.php?bugnote_id=36428#r1530
2014-07-22 13:37Brad KingNote Added: 0036429
2014-07-23 02:48Johannes S. Mueller-RoemerNote Added: 0036433
2014-07-23 10:09Brad KingNote Added: 0036437
2014-07-23 11:07Johannes S. Mueller-RoemerNote Added: 0036439
2014-07-23 13:13Brad KingNote Added: 0036440
2015-04-10 16:24Brad KingRelationship addedrelated to 0015509
2016-06-10 14:27Kitware RobotNote Added: 0041396
2016-06-10 14:27Kitware RobotStatusassigned => resolved
2016-06-10 14:27Kitware RobotResolutionopen => moved
2016-06-10 14:30Kitware RobotStatusresolved => closed

Notes
(0021372)
jorgenys   
2010-07-13 03:14   
Is this really not resolved, yet?

I'm currently trying to use this to convert a path during a MinGW configuration and the resulting path is not with appropriate backslashes.

What is worse, TO_NATIVE_PATH does not seem to alter any slashes at all when using MinGW generator (when containing a mix of both, the resulting path retains the different slashes).
(0022385)
EricG   
2010-09-29 15:38   
I'm also seeing this issue.
(0027415)
Thomas Thorsen   
2011-09-14 07:11   
I have run into this issue as well on the MSYS Makefile generator. It is almost 4 years since this was reported, can we get some kind of indication when this will be looked at?

This facility is essential to support add_custom_commands that invoke native tools that do not understand forward slashes but also to avoid the POSIX path translation applied by MSYS when arguments are passed to a non-MSYS executable (such as cmake -E).

Here's an example invoking coverity in a custom command:

file(TO_NATIVE_PATH "${CMAKE_CURRENT_BINARY_DIR}/myfile.covout" _out_native)
file(TO_NATIVE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/myfile" _in_native)
file(TO_NATIVE_PATH "${PROJECT_BINARY_DIR}/coverity" _dir_native)

add_custom_command(
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/myfile.covout"
COMMAND "${COVERITY_TRANSLATE_EXECUTABLE}"
--verbose 0 --force
--dir ${_dir_native}
--redirect "stdout,${_out_native}"
${CMAKE_C_COMPILER}
${_coverity_parameters}
${_in_native}
DEPENDS myfile
COMMENT "Coverity translate ${_in_native}"
VERBATIM)

The problem here is that the absolute paths are converted to the format "C:/bla/bla" instead of "C:\bla\bla". This is first evident in the printout of the comment:

[ 0%] Coverity translate s;C:\MinGW\msys\1.0\sourcedirectory\myfile

While this is not fatal, the other arguments to coverity causes the MSYS POSIX path translation to mangle the path so the tool execution fails.

Experimenting with "cmake -E echo" reveals what is going on (as cmake is considered external to MSYS the parameters passed to it are subject to POSIX Path conversion, just like the custom commands in the generated build system):

> cmake -E echo "s:/sourcedirectory/myfile"
s:/sourcedirectory/myfile

Not touched by conversion!

> cmake -E echo "Coverity translate s:/sourcedirectory/myfile"
Coverity translate s;C:\MinGW\msys\1.0\sourcedirectory\myfile

Adding the extra comment text and the conversion destroys the path! Same is happening for the --redirect argument:

> cmake -E echo "stdout,s:/sourcedirectory/myfile.covout"
stdout,s;C:\MinGW\msys\1.0\sourcedirectory\myfile.covout

Making sure the path is really native (i.e. using backslash instead of forward slash), the conversion does not destroy the path:

> cmake -E echo "Coverity translate s:\\sourcedirectory\\myfile"
Coverity translate s:\sourcedirectory\myfile

> cmake -E echo "stdout,s:\\sourcedirectory\\myfile.covout"
stdout,s:\sourcedirectory\myfile.covout

Consequently, the TO_NATIVE_PATH on MSYS (and possible MinGW, haven't played around with that, as it only runs in a cmd.exe which is useless for real work), should always emit paths of the form "C:\bla\bla", as that is understood by all MSYS and non-MSYS tools.

Let me know if we can do anything to help this along.
(0029155)
Daniel Franke   
2012-04-13 18:56   
Another fun problem on MinGW (cmake-2.8.6): if one has a file to be included by the NSIS installer, one needs to pass its location along. A straightforward approach ...

set (CPACK_NSIS_DEFINES "!include ${CMAKE_CURRENT_LIST_DIR}/FileAssociation.nsh")

... fails with an NSIS error that the file was not found. If one passes the same location with '\' instead of '/', it works. So NSIS expects native names. But unfortunately

file(TO_NATIVE_PATH ${CMAKE_CURRENT_LIST_DIR}/FileAssociation.nsh FILE_ASSOCIATION_NSH)
set (CPACK_NSIS_DEFINES "!include ${FILE_ASSOCIATION_NSH}")

does not work do to the "native path" coming with '/' as well. Meh!
(0036330)
Johannes S. Mueller-Roemer   
2014-07-09 08:45   
Now, almost 7 years later, I encountered this bug in the LLVM project, where type (equivalent to cat) is used which does not support forward slashes in paths.

Is this bug of such low priority that it will never be fixed? Is the assignee dead? 7 years is a long time.
(0036427)
Bill Hoffman   
2014-07-22 11:14   
I am not quite dead yet... :)


The trouble comes in here:

cmGlobalMinGWMakefileGenerator::cmGlobalMinGWMakefileGenerator()
{
  this->FindMakeProgramFile = "CMakeMinGWFindMake.cmake";
  this->ForceUnixPaths = true;
  this->ToolSupportsColor = true;
  this->UseLinkScript = true;
}

Can someone try taking out this->ForceUnixPaths = true and see if it breaks anything else with the MinGW Makefiles?
(0036428)
Brad King   
2014-07-22 13:34   
Re 0005939:0036427: That breaks the generator because Makefile syntax wants \-escaped spaces instead of quoting.

file(TO_NATIVE_PATH) has a few problems. It uses ConvertToOutputPath which is meant to create an escaped/quoted path for writing to Makefile rules, not shells. On Windows file(TO_NATIVE_PATH) removes the double-quotes returned by ConvertToOutputPath, but on UNIX it leaves escaped spaces. This means there is no well-defined way to use the resulting value on a command-line.

file(TO_NATIVE_PATH) also outputs path lists with ;-separation even on UNIX where environment paths expect :-separation.

The behavior of the command should be redesigned and fixed with a policy.

(0036429)
Brad King   
2014-07-22 13:37   
Re 0005939:0036428: It may be tricky to define file(TO_NATIVE_PATH) correctly because the meaning of "native" depends on context even within one platform. On MSYS, for example, CMake is still a Windows application, and forward-slash paths are only "native" to the MSYS shell.

Perhaps it would be better to disallow the command completely through a policy. Project code can use string(REPLACE) to fix paths the way it needs
(0036433)
Johannes S. Mueller-Roemer   
2014-07-23 02:48   
Re 0005939:0036428: I don't know if the list separator is really an issue, as a path list isn't really a path itself.

Re 0005939:0036429: MSYS-Makefiles and MinGW-Makefiles are two different generators though. I am talking about the second, not the first.
(0036437)
Brad King   
2014-07-23 10:09   
Re 0005939:0036433: I understand the difference between MSYS Makefiles and MinGW Makefiles, but I'm saying that the whole notion of a "native" path is not well-defined in general because we do not know whether the path is to be used in the environment, on the command line, or elsewhere. The MSYS case is one example of ambiguity.

IIRC the file(TO_CMAKE_PATH) command was created as a way to evaluate environment variables like PATH safely. Then file(TO_NATIVE_PATH) was created as the inverse of that for symmetry, but was not done correctly. Perhaps we should create a new pair of commands meant specifically for dealing with paths in environment variables, e.g. list(FROM_NATIVE_ENV) and list(TO_NATIVE_ENV). Then we can deprecate the broken file(TO_NATIVE_PATH).

In your use case I suggest using string(REPLACE) always and dropping use of file(TO_NATIVE_PATH).
(0036439)
Johannes S. Mueller-Roemer   
2014-07-23 11:07   
IMO, "ConvertToOutputPath" is the wrong approach, as Mingw-Makefiles work just fine with backslashes inside a path, as long as they are escaped properly. Therefore wouldn't the right solution (on the CMake side of things) be to split TO_NATIVE_PATH and Makefile escaping into two separate functions?
(0036440)
Brad King   
2014-07-23 13:13   
Re 0005939:0036439: Yes, but I'm also arguing that TO_NATIVE_PATH is impossible to implement correctly. Also we can't change its behavior now without a policy, so we might as well deprecate it and define new interfaces that are better.

On the MinGW Makefiles generation side, ConvertToOutputPath is a bit of a mess left from very early days when we did not distinguish between paths meant for a Makefile rule and paths meant for a shell command within such a rule. It is not trivial to remove though because there are many distinct call paths that go through ConvertToOutputPath and they each need to be refactored properly. I've made some incremental progress on that over the years but have not had time to make a full sweep.
(0041396)
Kitware Robot   
2016-06-10 14:27   
Resolving issue as `moved`.

This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.