[CMake] CMake, armcc and Ninja workarounds and problems
Olsson Gerhard
gerhard.olsson at volvo.com
Mon Oct 27 10:01:56 EDT 2014
A summary of limitations with (mostly) armcc and Cmake, primarly using Ninja generator.
(Note that this is referring to two emails and add a new problem.)
The workarounds where applicable are listed, as well as possible corrections/enhancements in CMake. I will probably open issues later, maybe patches too, but there may be other solutions.
Setup:
CMake 3.0.2 with Ninja 1.5.1 on Windows. Unix Makefiles was previously used in Windows and is still used in Linux (Linux is used for speedup compilation).
WindRiver Diab (DCC) and ARM DS-5 armcc (RVDS) cross compiling.
1. No append to log file for armcc
(see original email)
Workaround is to patch generated files, replacing a token with a temp file that is later concatenated.
A proper solution could be custom variables.
For example (set in user part of configuration)
ADD_DEFINITIONS( --errors= logs/<SOURCE_FILE_NAME>.log)
<SOURCE_FILE_NAME> would be a new expand variable, i.e. "SourceFileName.c" in SOURCE path/to/SourceFileName.c
The configuratione
2. Cannot control name of dependfile set in Ninja
Ninja generator sets the name to objectpath.d
cmGlobalNinjaGenerator::EncodeDepfileSpace(objectFileName + ".d");
WindRiver sets the dep file name to objectDir/SourceName.d, so the ".obj" in "DEP_FILE = path/BaseName.c.obj.d" must be removed
Armcc just uses base name in the same directory as the command runs, "BaseName.d". -depend_dir= can be used to control the directory for C but not asm.
The workaround is to patch the generated build.ninja
A proper solution is using new variables similar to above. (Just examples, not figured out <ARCH> etc)
WindRiver (not added to CMake):
SET (CMAKE_<LANG>_<COMPILER_ID>_DEPFILE_PATTERN <OBJECT_DIR>/<SOURCE_FILE_NAME>.d )
Armcc (RVDS patch exists for CMake):
SET (CMAKE_DEPFILE_FLAGS_C -depend_dir=<OBJECT_FULL_DIR> --depend_format=unix)
SET (CMAKE_C_ARMCC_DEPFILE_PATTERN <OBJECT_FULL_DIR>/<SOURCE_BASE_NAME>.d )
SET (CMAKE_DEPFILE_FLAGS_ASM --depend_format=unix)
SET (CMAKE_ASM_ARMCC_DEPFILE_PATTERN <SOURCE_BASE_NAME>.d )
Note: <OBJECT> is replaced in CMAKE_C_ARMCC_DEPFILE_PATTERN (but adding a semicolon), but not <OBJECT_DIR> (as that is evaluated after FLAGS/DEFINES) so it is a little more than just add new variables.
The workaround for armcc could therefore not use CMAKE_DEPFILE_FLAGS_C, the -depend_dir flag was added to normal FLAGS
3. CMake adds CMD /C for both LINKER and prelink/postbuild
PRE_LINK and POST_BUILD is executed as part of linking, so extra CMD is an error (at least the quotes)
CMake adds CMD /C for linker, but also for PRE_LINK and POST_BUILD.CMD /C is added twice.
rule C_EXECUTABLE_LINKER_RSP_FILE
command = cmd.exe /C "$PRE_LINK && C:\PROGRA~1\DS-5\bin\armcc.exe $FLAGS $LINK_FLAGS --via=$RSP_FILE -o $out && $POST_BUILD"
The first workaround was to patch build.ninja. Not really needed after the second workaround that removed the need for PRE_LINK/POST_BUILD, stll included in the patches below.
Setup separate targets for a few ASM files (not compiling in PRE_LINK), move out POST_BUILD to separate targets included in ALL.
This works fine here (better if full armcc/WindRiver support was added of course)
I believe the solution is to remove CMD /C for PRE_LINK/POSTBUILD. There are command length complications though.
4. Commands can be longer than 8000 bytes in Windows
a. The Ninja generator may need to use RSP_FILE also when compiling.
b. PRE_LINK/POSTBUILD can give long lines
The workaround for a. is to decrease the include paths, for b. to not use PRE_LINK.
The solution for a. is to evaluate using RSP_FILE if needed in Ninja. I believe the Unix Makefiles generator handles this better.
For b. I have no good proposal.
/Gerhard
--
Rough workaround
Build is invoked from make (CMAKE_MAKEFILE is set to path to build.ninja):
${CMAKE_MAKEFILE}: ${CMAKELISTS} ${CMAKE_TOOLCHAIN} | ${CMAKE_BUILD_DIR}
@${ECHO}
$(CMD_V)cd ${CMAKE_BUILD_DIR} && ${CMAKE_MPC5516E} $(CMAKE_FLAGS_MPC5516E) ${CMAKELISTS_OSDIR}
ifeq ($(CMAKE_GENERATOR),${CMAKE_NINJA})
@#CMake Ninja uses cmd, quotes incorrectly
@perl -pi -e 's%^(\s*(POST_BUILD|PRE_LINK)\s*=\s*)cmd.exe\s(/C)?\s*"(.*)"%$$1$$4%i; ' $@
ifneq ($(COMPILE_DEPEND_DIR),)
@#no path for armcc, use --depend_dir=depend to avoid have all in root (asm go there)
@perl -pi -e 'BEGIN{$$c="$(COMPILE_DEPEND_DIR)";} s%^(\s*DEP_FILE\s*=\s*).*/(.*).c\.obj\.d\s*$$%$$1$$c/$$2.d\n%;' $@
@$(MKDIR) -p $(dir ${CMAKE_MAKEFILE})depend
else
@#unexpected suffix (probably other ways to do this)
@perl -pi -e 's%^(\s*DEP_FILE\s*=\s*.*)\.obj\.d\s*$$%$$1.d\n%;' $@
endif
ifneq ($(COMPILE_LOG_NO_APPEND),)
@perl -pi -e 'BEGIN{$$c="$(COMPILE_LOG_NO_APPEND)"; $$n="unknown"; $$i=1;}if(m%^build .*?([\w\.]+):%){$$n=$$1;} if (s%(--errors)=$$c%$$1=logs/$$n.$$i%){$$i++; $$n="unknown";}' $@
@$(MKDIR) -p $(dir ${CMAKE_MAKEFILE})logs
endif
compile_target: ${CMAKE_MAKEFILE} | $(INFO_BUILD_DIR) ${BUILD_RESULT_DIR}
@${ECHO}
ifneq ($(COMPILE_LOG_NO_APPEND),)
@$(ECHO) " Note: If compile fails: \"make copy_log\""
endif
$(CMAKE_MAKE) install
ifneq ($(COMPILE_LOG_NO_APPEND),)
@$(MAKE) copy_logs
endif
copy_logs:
ifneq ($(COMPILE_LOG_NO_APPEND),)
$(CMD_V)$(CAT) $(CMAKE_BUILD_DIR)/logs/* > ${INFO_BUILD_DIR}/error.txt
$(CMD_V)$(RM) $(CMAKE_BUILD_DIR)/logs/*
endif
From: Olsson Gerhard
Sent: den 8 september 2014 7:46
To: 'cmake at cmake.org'
Subject: Compilation logs: Workaround for no appendfile
ARM DS-5 compiler (armcc compiler 5.04)
Platform: Windows and Linux
Armcc (as well as arasm, armlink etc) has option '-errors=file 'to log compilation messages (warnings and errors) to a logfile instead of stdout.
There is no possibility from what I have found out to append to a log file, all output overwrites the log.
Background:
We cannot change some 3rd party code and cannot suppress all compilation warnings. The workaround is to parse the logs.
If printed to the console, the messages are mixed up with progress and license messages.
Other compilers has the possibility to append to logs, but not armcc what I can find.
A workaround would be to CMake to save stdout to a logfile. May be possible, but I am not sure that is a good solution.
Another is to log to unique logname, include the compiled file in the logname. I am not sure how that can be done in CMake.
Any hints how to do this?
/Gerhard
From: Olsson Gerhard
Sent: den 1 oktober 2014 7:46
To: 'cmake at cmake.org'
Subject: Ninja: CMD /C, quoting added for PRE_LINK/POST_BUILD
The commands for PRE_LINK/POST_BUILD is not correctly quoted for Ninja in Windows. 'CMD /C "<command>"' is added to the command, just as Ninja itself does.
(this was maybe different in earlier Ninja versions, but has been around for a couple of years)
(The quoting in the working command is strange, but works.)
A workaround that is not so bad, is to patch build.ninja
Is there a better way?
Note: The lines LINK_FLAGS ,POST_BUILD , PRE_LINK , TARGET_PDB are duplicated in build.ninja. Does not make a difference.
ARM DS-5 compiler (armcc compiler 5.04)
Platform: Windows (Linux used too, but with Unix Makefiles)
CMake 3.0.2
Ninja 1.5.1
/Gerhard
CmakeLists.txt excerpt:
ADD_CUSTOM_COMMAND(
TARGET ECU
PRE_LINK
COMMAND "C:/Program Files/DS-5/bin/armasm.exe" --cpu=Cortex-A9 -g --cpreproc --apcs=interwork --arm --no_unaligned_access --pd "__EVAL SETA 1" --pd "__MICROLIB SETA 1" -Ic:/Temp/includes -o ${CMAKE_CURRENT_BINARY_DIR}/osekasm.o c:/Os/osekasm.s VERBATIM
)
#Generate hex, copy to view
add_custom_command(TARGET ECU POST_BUILD
COMMAND "C:/Program Files/DS-5/bin/fromelf.exe" --m32combined --output=${CMAKE_CURRENT_BINARY_DIR}/ECU.hex ${CMAKE_CURRENT_BINARY_DIR}/ECU.elf VERBATIM
COMMAND "C:/cygwin/bin/objcopy" -I elf32-big -O srec --srec-len=32 --srec-forceS3 -j .securityLevel ${CMAKE_CURRENT_BINARY_DIR}/ECU.elf ${CMAKE_CURRENT_BINARY_DIR}/parameter_section.hex VERBATIM
)
Generated in build.ninja:
LINK_FLAGS = --cpu=Cortex-A9 c:/Temp/includes/SecModLib.a -L--list=c:/Temp/build/ECU.map -L--scatter=c:/Zynq7000/scatter.txt -L--map -L--no_remove -L--symbols -L--info=common,summarysizes,summarystack,sizes,totals,unused,unusedsymbols,veneers C:/Temp/build/osekasm.o
POST_BUILD = cmd.exe /C "cd /D C:\Temp\build && "C:\Program Files\DS-5\bin\fromelf.exe" --m32combined --output=C:/Temp/build/ECU.hex C:/Temp/build/ECU.elf && C:\cygwin\bin\objcopy -I elf32-big -O srec --srec-len=32 --srec-forceS3 -j .securityLevel C:/Temp/build/ECU.elf C:/Temp/build/parameter_section.hex"
PRE_LINK = cmd.exe /C "cd /D C:\Temp\build && "C:\Program Files\DS-5\bin\armasm.exe" --cpu=Cortex-A9 -g --cpreproc --apcs=interwork --arm --no_unaligned_access --pd "__EVAL SETA 1" --pd "__MICROLIB SETA 1" -Ic:/Temp/includes -o C:/Temp/build/osekasm.o c:/Temp/Os/osekasm.s && cd C:\Temp\build"
TARGET_PDB = ECU.elf.dbg
Console output:
[106/107] Linking C executable ECU.elf
FAILED: cmd.exe /C "cmd.exe /C "cd /D C:\Temp\build && "C:\Program Files\DS-5\bin\armasm.exe" --cpu=Cortex-A9 -g --cpreproc --apcs=interwork --arm --no_unaligned_access --pd "__EVAL SETA 1" --pd "__MICROLIB SETA 1" -Ic:/Temp/includes -o C:/Temp/build/osekasm.o c:/Temp/Os/osekasm.s && cd C:\Temp\build" && C:\PROGRA~1\DS-5\bin\armcc.exe --cpu=Cortex-A9 c:/Temp/includes/SecModLib.a -L--list=c:/Temp/build/ECU.map -L--scatter=c:/Zynq7000/scatter.txt -L--map -L--no_remove -L--symbols -L--info=common,summarysizes,summarystack,sizes,totals,unused,unusedsymbols,veneers C:/Temp/build/osekasm.o --via=CMakeFiles/ECU.rsp -o ECU.elf && cmd.exe /C "cd /D C:\Temp\build && "C:\Program Files\DS-5\bin\fromelf.exe" --m32combined --output=C:/Temp/build/ECU.hex C:/Temp/build/ECU.elf && C:\cygwin\bin\objcopy -I elf32-big -O srec --srec-len=32 --srec-forceS3 -j .securityLevel C:/Temp/build/ECU.elf C:/Temp/build/parameter_section.hex""
'cmd.exe' is not recognized as an internal or external command,
operable program or batch file.
ninja: build stopped: subcommand failed.
Patched (in Makefile):
${CMAKE_MAKEFILE}: ${CMAKELISTS} ${CMAKE_TOOLCHAIN} | ${CMAKE_BUILD_DIR}
@#CMake Ninja uses cmd incorrectly
@perl -pi -e 's%^(\s*(POST_BUILD|PRE_LINK)\s*=\s*)cmd.exe\s(/C)?\s*"(.*)"%$$1$$4%i; ' $@
Rules patched OK:
LINK_FLAGS = --cpu=Cortex-A9 c:/Temp/includes/SecModLib.a -L--list=c:/Temp/build/ECU.map -L--scatter=c:/Zynq7000/scatter.txt -L--map -L--no_remove -L--symbols -L--info=common,summarysizes,summarystack,sizes,totals,unused,unusedsymbols,veneers C:/Temp/build/osekasm.o
POST_BUILD = cd /D C:\Temp\build && "C:\Program Files\DS-5\bin\fromelf.exe" --m32combined --output=C:/Temp/build/ECU.hex C:/Temp/build/ECU.elf && C:\cygwin\bin\objcopy -I elf32-big -O srec --srec-len=32 --srec-forceS3 -j .securityLevel C:/Temp/build/ECU.elf C:/Temp/build/parameter_section.hex
PRE_LINK = cd /D C:\Temp\build && "C:\Program Files\DS-5\bin\armasm.exe" --cpu=Cortex-A9 -g --cpreproc --apcs=interwork --arm --no_unaligned_access --pd "__EVAL SETA 1" --pd "__MICROLIB SETA 1" -Ic:/Temp/includes -o C:/Temp/build/osekasm.o c:/Temp/Os/osekasm.s && cd C:\Temp\build
TARGET_PDB = ECU.elf.dbg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cmake/attachments/20141027/dd16c3a9/attachment-0001.html>
More information about the CMake
mailing list