[CMake] CMake, armcc and Ninja workarounds and problems
fungos
fungos at gmail.com
Tue Oct 28 11:39:57 EDT 2014
I agree with your review and I agree that there are plenty of problems to
be able to get everything working correctly.
I would like to warn you and point to an issue about the use of
--depend_format=unix:
https://github.com/martine/ninja/issues/794
Also, I've got around the .obj.d problem patching CMake generator adding a
flag: set(CMAKE_NINJA_DEPFILE_REMOVE_OBJECT_EXT TRUE)
I currently use this:
set(CMAKE_DEPFILE_FLAGS_C "--md --no_depend_system_headers
--depend_format=unix --depend_single_line --depend=<OBJECT>.d")
The sad is that these are simple things to solve but require patches and
maintaining a "fork".
On Tue, Oct 28, 2014 at 10:55 AM, Olsson Gerhard <gerhard.olsson at volvo.com>
wrote:
> Update:
>
>
>
> 1. Append to logfile
>
> Just resolving <OBJECT> would be an improvement, but that may require
> changes to all generators, no global resolution what I see.
>
> Should be changed in armcc, dropping for CMake.
>
>
>
> 2. Dependfile
>
> This can be done in armcc:
>
> SET( CMAKE_DEPFILE_FLAGS_C "--depend=<OBJECT>.d --depend_format=unix" )
>
> Similar for ARM, but untested.
>
>
>
> WindRiver:
>
> SET( CMAKE_DEPFILE_FLAGS_C "-Xmake-dependency=6
> -Xmake-dependency-savefile=<OBJECT>.d" )
>
> (not possible for asm)
>
>
>
> So this should be working as intended
>
>
>
>
>
> 3. Ninja PRE_LINK/POST_BUILD adds cmd.exe
>
> Need to be changed in cmNormalNinjaTargetGenerator.cxx to not add twice.
>
>
>
> vars["PRE_LINK"] =
>
> this->GetLocalGenerator()->BuildCommandLine(preLinkCmdLines);
>
> (cmLocalNinjaGenerator::BuildCommandLine)
>
>
>
> /Gerhard
>
>
>
> *From:* Olsson Gerhard
> *Sent:* den 27 oktober 2014 3:02
> *To:* 'cmake at cmake.org'
> *Subject:* CMake, armcc and Ninja workarounds and problems
>
>
>
> A summary of limitations with (mostly) armcc and Cmake, primarly using
> Ninja generator.
>
> (Note that this is referring to two emails and add a new problem.)
>
> The workarounds where applicable are listed, as well as possible
> corrections/enhancements in CMake. I will probably open issues later, maybe
> patches too, but there may be other solutions.
>
>
>
>
>
> Setup:
>
> CMake 3.0.2 with Ninja 1.5.1 on Windows. Unix Makefiles was previously
> used in Windows and is still used in Linux (Linux is used for speedup
> compilation).
>
> WindRiver Diab (DCC) and ARM DS-5 armcc (RVDS) cross compiling.
>
>
>
>
>
> 1. No append to log file for armcc
>
> (see original email)
>
> Workaround is to patch generated files, replacing a token with a temp file
> that is later concatenated.
>
>
>
> A proper solution could be custom variables.
>
> For example (set in user part of configuration)
>
> ADD_DEFINITIONS( --errors= logs/<SOURCE_FILE_NAME>.log)
>
> <SOURCE_FILE_NAME> would be a new expand variable, i.e. “SourceFileName.c”
> in SOURCE path/to/SourceFileName.c
>
> The configuratione
>
>
>
>
>
> 2. Cannot control name of dependfile set in Ninja
>
>
>
> Ninja generator sets the name to objectpath.d
>
> cmGlobalNinjaGenerator::EncodeDepfileSpace(objectFileName +
> ".d");
>
>
>
> WindRiver sets the dep file name to objectDir/SourceName.d, so the “.obj”
> in “DEP_FILE = path/BaseName.c.obj.d” must be removed
>
> Armcc just uses base name in the same directory as the command runs,
> “BaseName.d”. –depend_dir= can be used to control the directory for C but
> not asm.
>
>
>
> The workaround is to patch the generated build.ninja
>
>
>
> A proper solution is using new variables similar to above. (Just examples,
> not figured out <ARCH> etc)
>
>
>
> WindRiver (not added to CMake):
>
> SET (CMAKE_<LANG>_<COMPILER_ID>_DEPFILE_PATTERN
> <OBJECT_DIR>/<SOURCE_FILE_NAME>.d )
>
>
>
> Armcc (RVDS patch exists for CMake):
>
> SET (CMAKE_DEPFILE_FLAGS_C –depend_dir=<OBJECT_FULL_DIR>
> --depend_format=unix)
>
> SET (CMAKE_C_ARMCC_DEPFILE_PATTERN <OBJECT_FULL_DIR>/<SOURCE_BASE_NAME>.d )
>
> SET (CMAKE_DEPFILE_FLAGS_ASM --depend_format=unix)
>
> SET (CMAKE_ASM_ARMCC_DEPFILE_PATTERN <SOURCE_BASE_NAME>.d )
>
>
>
> Note: <OBJECT> is replaced in CMAKE_C_ARMCC_DEPFILE_PATTERN (but adding a
> semicolon), but not <OBJECT_DIR> (as that is evaluated after FLAGS/DEFINES)
> so it is a little more than just add new variables.
>
> The workaround for armcc could therefore not use CMAKE_DEPFILE_FLAGS_C,
> the –depend_dir flag was added to normal FLAGS
>
>
>
>
>
> 3. CMake adds CMD /C for both LINKER and prelink/postbuild
>
> PRE_LINK and POST_BUILD is executed as part of linking, so extra CMD is an
> error (at least the quotes)
>
> CMake adds CMD /C for linker, but also for PRE_LINK and POST_BUILD.CMD /C
> is added twice.
>
>
>
> rule C_EXECUTABLE_LINKER_RSP_FILE
>
> command = cmd.exe /C "$PRE_LINK && C:\PROGRA~1\DS-5\bin\armcc.exe
> $FLAGS $LINK_FLAGS --via=$RSP_FILE -o $out && $POST_BUILD"
>
>
>
> The first workaround was to patch build.ninja. Not really needed after the
> second workaround that removed the need for PRE_LINK/POST_BUILD, stll
> included in the patches below.
>
> Setup separate targets for a few ASM files (not compiling in PRE_LINK),
> move out POST_BUILD to separate targets included in ALL.
>
> This works fine here (better if full armcc/WindRiver support was added of
> course)
>
>
>
> I believe the solution is to remove CMD /C for PRE_LINK/POSTBUILD. There
> are command length complications though.
>
>
>
>
>
> 4. Commands can be longer than 8000 bytes in Windows
>
> a. The Ninja generator may need to use RSP_FILE also when compiling.
>
> b. PRE_LINK/POSTBUILD can give long lines
>
>
>
> The workaround for a. is to decrease the include paths, for b. to not use
> PRE_LINK.
>
>
>
> The solution for a. is to evaluate using RSP_FILE if needed in Ninja. I
> believe the Unix Makefiles generator handles this better.
>
> For b. I have no good proposal.
>
>
>
>
>
> /Gerhard
>
>
>
> --
>
>
>
> Rough workaround
>
> Build is invoked from make (CMAKE_MAKEFILE is set to path to build.ninja):
>
>
>
> ${CMAKE_MAKEFILE}: ${CMAKELISTS} ${CMAKE_TOOLCHAIN} | ${CMAKE_BUILD_DIR}
>
> @${ECHO}
>
> $(CMD_V)cd ${CMAKE_BUILD_DIR} &&
> ${CMAKE_MPC5516E} $(CMAKE_FLAGS_MPC5516E) ${CMAKELISTS_OSDIR}
>
>
>
> ifeq ($(CMAKE_GENERATOR),${CMAKE_NINJA})
>
> @#CMake Ninja uses cmd, quotes incorrectly
>
> @perl -pi -e
> 's%^(\s*(POST_BUILD|PRE_LINK)\s*=\s*)cmd.exe\s(/C)?\s*"(.*)"%$$1$$4%i; ' $@
>
> ifneq ($(COMPILE_DEPEND_DIR),)
>
> @#no path for armcc, use --depend_dir=depend
> to avoid have all in root (asm go there)
>
> @perl -pi -e
> 'BEGIN{$$c="$(COMPILE_DEPEND_DIR)";}
> s%^(\s*DEP_FILE\s*=\s*).*/(.*).c\.obj\.d\s*$$%$$1$$c/$$2.d\n%;' $@
>
> @$(MKDIR) -p $(dir ${CMAKE_MAKEFILE})depend
>
> else
>
> @#unexpected suffix (probably other ways to
> do this)
>
> @perl -pi -e
> 's%^(\s*DEP_FILE\s*=\s*.*)\.obj\.d\s*$$%$$1.d\n%;' $@
>
> endif
>
> ifneq ($(COMPILE_LOG_NO_APPEND),)
>
> @perl -pi -e
> 'BEGIN{$$c="$(COMPILE_LOG_NO_APPEND)"; $$n="unknown"; $$i=1;}if(m%^build
> .*?([\w\.]+):%){$$n=$$1;} if (s%(--errors)=$$c%$$1=logs/$$n.$$i%){$$i++;
> $$n="unknown";}' $@
>
> @$(MKDIR) -p $(dir ${CMAKE_MAKEFILE})logs
>
> endif
>
>
>
> compile_target: ${CMAKE_MAKEFILE} | $(INFO_BUILD_DIR) ${BUILD_RESULT_DIR}
>
> @${ECHO}
>
> ifneq ($(COMPILE_LOG_NO_APPEND),)
>
> @$(ECHO) " Note: If compile fails: \"make
> copy_log\""
>
> endif
>
> $(CMAKE_MAKE) install
>
> ifneq ($(COMPILE_LOG_NO_APPEND),)
>
> @$(MAKE) copy_logs
>
> endif
>
>
>
> copy_logs:
>
> ifneq ($(COMPILE_LOG_NO_APPEND),)
>
> $(CMD_V)$(CAT) $(CMAKE_BUILD_DIR)/logs/* >
> ${INFO_BUILD_DIR}/error.txt
>
> $(CMD_V)$(RM) $(CMAKE_BUILD_DIR)/logs/*
>
> endif
>
>
>
> *From:* Olsson Gerhard
> *Sent:* den 8 september 2014 7:46
> *To:* 'cmake at cmake.org'
> *Subject:* Compilation logs: Workaround for no appendfile
>
>
>
> ARM DS-5 compiler (armcc compiler 5.04)
>
> Platform: Windows and Linux
>
>
>
> Armcc (as well as arasm, armlink etc) has option ‘—errors=file ’to log
> compilation messages (warnings and errors) to a logfile instead of stdout.
>
> There is no possibility from what I have found out to append to a log
> file, all output overwrites the log.
>
>
>
> Background:
>
> We cannot change some 3rd party code and cannot suppress all compilation
> warnings. The workaround is to parse the logs.
>
> If printed to the console, the messages are mixed up with progress and
> license messages.
>
> Other compilers has the possibility to append to logs, but not armcc what
> I can find.
>
>
>
> A workaround would be to CMake to save stdout to a logfile. May be
> possible, but I am not sure that is a good solution.
>
> Another is to log to unique logname, include the compiled file in the
> logname. I am not sure how that can be done in CMake.
>
>
>
> Any hints how to do this?
>
>
>
> /Gerhard
>
>
>
>
>
>
>
> *From:* Olsson Gerhard
> *Sent:* den 1 oktober 2014 7:46
> *To:* 'cmake at cmake.org'
> *Subject:* Ninja: CMD /C, quoting added for PRE_LINK/POST_BUILD
>
>
>
> The commands for PRE_LINK/POST_BUILD is not correctly quoted for Ninja in
> Windows. ‘CMD /C “<command>”’ is added to the command, just as Ninja itself
> does.
>
> (this was maybe different in earlier Ninja versions, but has been around
> for a couple of years)
>
> (The quoting in the working command is strange, but works.)
>
>
>
> A workaround that is not so bad, is to patch build.ninja
>
> Is there a better way?
>
>
>
> Note: The lines LINK_FLAGS ,POST_BUILD , PRE_LINK , TARGET_PDB are
> duplicated in build.ninja. Does not make a difference.
>
>
>
> ARM DS-5 compiler (armcc compiler 5.04)
>
> Platform: Windows (Linux used too, but with Unix Makefiles)
>
> CMake 3.0.2
>
> Ninja 1.5.1
>
>
>
> /Gerhard
>
>
>
>
>
> CmakeLists.txt excerpt:
>
>
>
> ADD_CUSTOM_COMMAND(
>
> TARGET ECU
>
> PRE_LINK
>
> COMMAND "C:/Program Files/DS-5/bin/armasm.exe" --cpu=Cortex-A9 -g
> --cpreproc --apcs=interwork --arm --no_unaligned_access --pd "__EVAL SETA
> 1" --pd "__MICROLIB SETA 1" -Ic:/Temp/includes -o
> ${CMAKE_CURRENT_BINARY_DIR}/osekasm.o c:/Os/osekasm.s VERBATIM
>
> )
>
>
>
> #Generate hex, copy to view
>
> add_custom_command(TARGET ECU POST_BUILD
>
> COMMAND "C:/Program Files/DS-5/bin/fromelf.exe" --m32combined
> --output=${CMAKE_CURRENT_BINARY_DIR}/ECU.hex
> ${CMAKE_CURRENT_BINARY_DIR}/ECU.elf VERBATIM
>
> COMMAND "C:/cygwin/bin/objcopy" -I elf32-big -O srec --srec-len=32
> --srec-forceS3 -j .securityLevel ${CMAKE_CURRENT_BINARY_DIR}/ECU.elf
> ${CMAKE_CURRENT_BINARY_DIR}/parameter_section.hex VERBATIM
>
> )
>
>
>
>
>
> Generated in build.ninja:
>
>
>
> LINK_FLAGS = --cpu=Cortex-A9 c:/Temp/includes/SecModLib.a
> -L--list=c:/Temp/build/ECU.map -L--scatter=c:/Zynq7000/scatter.txt -L--map
> -L--no_remove -L--symbols
> -L--info=common,summarysizes,summarystack,sizes,totals,unused,unusedsymbols,veneers
> C:/Temp/build/osekasm.o
>
> POST_BUILD = cmd.exe /C "cd /D C:\Temp\build && "C:\Program
> Files\DS-5\bin\fromelf.exe" --m32combined --output=C:/Temp/build/ECU.hex
> C:/Temp/build/ECU.elf && C:\cygwin\bin\objcopy -I elf32-big -O srec
> --srec-len=32 --srec-forceS3 -j .securityLevel C:/Temp/build/ECU.elf
> C:/Temp/build/parameter_section.hex"
>
> PRE_LINK = cmd.exe /C "cd /D C:\Temp\build && "C:\Program
> Files\DS-5\bin\armasm.exe" --cpu=Cortex-A9 -g --cpreproc --apcs=interwork
> --arm --no_unaligned_access --pd "__EVAL SETA 1" --pd "__MICROLIB SETA 1"
> -Ic:/Temp/includes -o C:/Temp/build/osekasm.o c:/Temp/Os/osekasm.s && cd
> C:\Temp\build"
>
> TARGET_PDB = ECU.elf.dbg
>
>
>
>
>
> Console output:
>
>
>
> [106/107] Linking C executable ECU.elf
>
> FAILED: cmd.exe /C "cmd.exe /C "cd /D C:\Temp\build && "C:\Program
> Files\DS-5\bin\armasm.exe" --cpu=Cortex-A9 -g --cpreproc --apcs=interwork
> --arm --no_unaligned_access --pd "__EVAL SETA 1" --pd "__MICROLIB SETA 1"
> -Ic:/Temp/includes -o C:/Temp/build/osekasm.o c:/Temp/Os/osekasm.s && cd
> C:\Temp\build" && C:\PROGRA~1\DS-5\bin\armcc.exe --cpu=Cortex-A9
> c:/Temp/includes/SecModLib.a -L--list=c:/Temp/build/ECU.map
> -L--scatter=c:/Zynq7000/scatter.txt -L--map -L--no_remove -L--symbols
> -L--info=common,summarysizes,summarystack,sizes,totals,unused,unusedsymbols,veneers
> C:/Temp/build/osekasm.o --via=CMakeFiles/ECU.rsp -o ECU.elf && cmd.exe /C
> "cd /D C:\Temp\build && "C:\Program Files\DS-5\bin\fromelf.exe"
> --m32combined --output=C:/Temp/build/ECU.hex C:/Temp/build/ECU.elf &&
> C:\cygwin\bin\objcopy -I elf32-big -O srec --srec-len=32 --srec-forceS3 -j
> .securityLevel C:/Temp/build/ECU.elf C:/Temp/build/parameter_section.hex""
>
> 'cmd.exe' is not recognized as an internal or external command,
>
> operable program or batch file.
>
> ninja: build stopped: subcommand failed.
>
>
>
>
>
> Patched (in Makefile):
>
>
>
> ${CMAKE_MAKEFILE}: ${CMAKELISTS} ${CMAKE_TOOLCHAIN} | ${CMAKE_BUILD_DIR}
>
> @#CMake Ninja uses cmd incorrectly
>
> @perl -pi -e
> 's%^(\s*(POST_BUILD|PRE_LINK)\s*=\s*)cmd.exe\s(/C)?\s*"(.*)"%$$1$$4%i; ' $@
>
>
>
>
>
>
>
> Rules patched OK:
>
>
>
> LINK_FLAGS = --cpu=Cortex-A9 c:/Temp/includes/SecModLib.a
> -L--list=c:/Temp/build/ECU.map -L--scatter=c:/Zynq7000/scatter.txt -L--map
> -L--no_remove -L--symbols
> -L--info=common,summarysizes,summarystack,sizes,totals,unused,unusedsymbols,veneers
> C:/Temp/build/osekasm.o
>
> POST_BUILD = cd /D C:\Temp\build && "C:\Program
> Files\DS-5\bin\fromelf.exe" --m32combined --output=C:/Temp/build/ECU.hex
> C:/Temp/build/ECU.elf && C:\cygwin\bin\objcopy -I elf32-big -O srec
> --srec-len=32 --srec-forceS3 -j .securityLevel C:/Temp/build/ECU.elf
> C:/Temp/build/parameter_section.hex
>
> PRE_LINK = cd /D C:\Temp\build && "C:\Program Files\DS-5\bin\armasm.exe"
> --cpu=Cortex-A9 -g --cpreproc --apcs=interwork --arm --no_unaligned_access
> --pd "__EVAL SETA 1" --pd "__MICROLIB SETA 1" -Ic:/Temp/includes -o
> C:/Temp/build/osekasm.o c:/Temp/Os/osekasm.s && cd C:\Temp\build
>
> TARGET_PDB = ECU.elf.dbg
>
>
>
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake
>
--
Animal Liberation Front
http://www.animal-liberation.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cmake/attachments/20141028/c561d39a/attachment-0001.html>
More information about the CMake
mailing list