[CMake] WIN32 set even when cross compiling = build configuration error

[Miroslav Keš]

Hi!

I'm using CMake to cross-compile C/C++ code for VxWorks targets. I have a platform file for VxWorks and toolchain files for the target boards.
The setup works fine with CMake 3.5 and older but it didn't work with 3.6 or newer. I was getting errors like

CMake Error at C:/Program Files/CMake/share/cmake-3.10/Modules/CMakeDetermineCompilerId.cmake:569 (string):
  string begin index: 1 is out of range 0 - 0
Call Stack (most recent call first):
  C:/Program Files/CMake/share/cmake-3.10/Modules/CMakeDetermineCompilerId.cmake:37 (CMAKE_DETERMINE_COMPILER_ID_CHECK)
  C:/Program Files/CMake/share/cmake-3.10/Modules/CMakeDetermineCCompiler.cmake:112 (CMAKE_DETERMINE_COMPILER_ID)
  CMakeLists.txt:33 (project)

Now I wanted to upgrade to a newer version so I spent some time debugging the problem ...

My toolchain file defines:
 
set( CMAKE_SYSTEM_NAME VxWorks )
set( CMAKE_SYSTEM_VERSION 6.9 )
set( CMAKE_SYSTEM_PROCESSOR arm )

which should be sufficient to tell CMake that the target platform is not Windows (where the CMake is run).
But the place in CMakeDetermineCompilerId.cmake where the error occurs looks like:

    # Detect the exact architecture from the PE header.
    if(WIN32)
      # The offset to the PE signature is stored at 0x3c.
      file(READ ${file} peoffsethex LIMIT 1 OFFSET 60 HEX)
      string(SUBSTRING "${peoffsethex}" 0 1 peoffsethex1)
      string(SUBSTRING "${peoffsethex}" 1 1 peoffsethex2)   <- this is where the error above occurs
     
      ...
    endif()

If I add a debug print of WIN32 before the project() command and run the CMake command line for my build with the --debug-output argument, I can see that:
1. the WIN32 is set to true before the project() command
2. the WIN32 is still set to true when it is tested in the CMakeDetermineCompilerId.cmake causing the error
3. and later finally unset in CMakeSystemSpecificInformation.cmake (line 17) but it is too late.

The reason why it worked in 3.5 and earlier is not in the CMakeDetermineCompilerId.cmake but in (more tolerant) checking of  string(SUBSTRING ...) range arguments.

I was able to work around that problem explicitly resetting the WIN32 variable in my toolchain file.

But I think it is a bug in CMake and the WIN32 variable should be unset when the CMAKE_SYSTEM_NAME is redefined for cross compilation.

Regards.

Mira