View Issue Details Jump to Notes ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0013385CMakeCMakepublic2012-07-09 06:032013-01-09 10:57
ReporterZaheer Chothia 
Assigned ToPeter Kuemmel 
PrioritynormalSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformMicrosoftOSWindows 7, 64-bitOS VersionSP1
Product VersionCMake 2.8.8 
Target VersionCMake 2.8.9Fixed in Version 
Summary0013385: Ninja: LINK_LIBRARIES not spilled to response file
Description* Command line length is limited on Windows.
* To alleviate this, CMake places object files into a response file, but the
  same is not done for libraries.
* This can pose a problem when the link line becomes too long.
Steps To ReproduceSee attached testcase generator.
Additional InformationI am primarily interested in a solution for the Ninja generator, although this
issue affects other generators too (e.g. NMake, MinGW Makefiles). For further discussion see the mailing list: http://www.cmake.org/pipermail/cmake/2012-June/051065.html [^]
TagsNo tags attached.
Attached Files? file icon testcase.sh [^] (997 bytes) 2012-07-09 06:03
bz2 file icon cmake_testcase_many_libraries_rspfile.tar.bz2 [^] (5,625 bytes) 2012-07-09 07:44
? file icon testcase-fixed.sh [^] (972 bytes) 2012-07-09 07:47
patch file icon NinjaRspFileIncludes_LINK_LIBRARIES.patch [^] (2,039 bytes) 2012-07-09 12:34 [Show Content]
zip file icon issue13385_build_ninja_mingw.zip [^] (50,990 bytes) 2012-07-10 14:17

 Relationships
related to 0013461closedPeter Kuemmel Ninja: linker response file gets expanded, command to long 

  Notes
(0029994)
Peter Kuemmel (developer)
2012-07-09 07:41

I could have a look at it but your test script doesn't work,
neither in msys nor ubuntu: "invalid option" and after
removing -e it simply copies the loop code.
(0029995)
Zaheer Chothia (reporter)
2012-07-09 07:47

Hello Peter, thanks for the quick response. I've uploaded a sample project. I think the script didn't work due to trailing whitespace when I copied from the mailing list post -- it should work fine now.
(0029996)
Peter Kuemmel (developer)
2012-07-09 07:53
edited on: 2012-07-10 05:06

Thanks, script works now, and I can reproduce the error.

Until I've pushed a patch you can fix rules.ninja:
move '$LINK_LIBRARIES' to 'rspfile_content = $in $LINK_LIBRARIES'

(0030006)
ClausKlein (reporter)
2012-07-09 12:37

I have tested this patch only under Darwin!
Please check why the command line length is checked to use response files.
(0030011)
Peter Kuemmel (developer)
2012-07-10 05:05

I pushed a patch to next:
http://public.kitware.com/pipermail/cmake-commits/2012-July/013022.html [^]

It enables rspfile support now also on Linux.
For Darwin we need to figure out how to get the max command line length at runtime.
(0030026)
Zaheer Chothia (reporter)
2012-07-10 14:17

The patch you pushed works fine on the testcase when using Microsoft's compilers,
although unfortunately it doesn't work correctly with MinGW.

The issue there is the following:
- The corresponding build rule (C_EXECUTABLE_LINKER_RSPFILE) first invokes 'ar'
  and passes LINK_LIBRARIES in the response file.
- LINK_LIBRARIES contains a mixture of library files (e.g. libxyz.a) as well as
  implicit dependencies (e.g. -lkernel32).
- As a result 'ar' fails with "invalid option" since it doesn't understand the
  implicit library options. What should be done instead, is to pass the first
  options to 'ar' and the latter only when calling 'gcc'.
- Attached are the relevant files from the build directory (issue13385_build_ninja_mingw.zip).
  The build log may be best to observe what I am referring to.

For reference these are the details of what I am using:
$ cmake --version
cmake version 2.8.8.20120710-ge5abdb
$ gcc --version
gcc (rubenvb-4.6.3) 4.6.3
(0030033)
Peter Kuemmel (developer)
2012-07-11 03:08

Explicit usage of ar.exe is a work around for older mingw versions.
This work around is not needed for gcc >4.4, so I've disabled it.
I don't know which mingw versions <4.4 will need this work around,
but I assume you can't use 3.4 with ninja now. Is this a problem?
(0030034)
Peter Kuemmel (developer)
2012-07-11 04:44

Also on Mac sysconf() can be used.
FYI, on Mac the maximum command line length is 262144.
(0030035)
Zaheer Chothia (reporter)
2012-07-11 05:48

Thanks Peter - the testcase works as expected for me now (MSVC + GCC)! :-)

I would personally be fine with removing the 'ar' workaround. There may be
others, though who experience require this indirection, even when using a fairly
recent version of GCC. This is due to a configuration issue with MinGW-w64
builds which was only fixed in May this year:
http://sourceforge.net/mailarchive/forum.php?thread_name=2766613.iHjfUlaVMm%40speed&forum_name=mingw-w64-public [^]
(0030036)
Peter Kuemmel (developer)
2012-07-11 05:58

AH, I haven't tested -w64.
Thanks for the link, seems it is only a configure problem, --with-gnu-ld must be used.

Isn't MinGW-w64 more experimental than mingw32?
So most -w64 users are forced to update anyway?
(0030037)
Zaheer Chothia (reporter)
2012-07-11 06:21

I agree that it is reasonable to assume an up-to-date toolset, especially since
it's quite easy to upgrade (unpack and add to PATH). In case someone runs into
this issue I would suggest trying either of these MinGW-w64 builds, which I have
found to be very stable:
http://sourceforge.net/projects/mingwbuilds/ [^]
http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/rubenvb/release/ [^]
(0032048)
Robert Maynard (manager)
2013-01-09 10:57

Closing resolved issues that have not been updated in more than 4 months.

 Issue History
Date Modified Username Field Change
2012-07-09 06:03 Zaheer Chothia New Issue
2012-07-09 06:03 Zaheer Chothia File Added: testcase.sh
2012-07-09 07:41 Peter Kuemmel Note Added: 0029994
2012-07-09 07:44 Zaheer Chothia File Added: cmake_testcase_many_libraries_rspfile.tar.bz2
2012-07-09 07:47 Zaheer Chothia Note Added: 0029995
2012-07-09 07:47 Zaheer Chothia File Added: testcase-fixed.sh
2012-07-09 07:53 Peter Kuemmel Note Added: 0029996
2012-07-09 07:53 Peter Kuemmel Status new => confirmed
2012-07-09 09:13 Peter Kuemmel Note Edited: 0029996
2012-07-09 12:34 ClausKlein File Added: NinjaRspFileIncludes_LINK_LIBRARIES.patch
2012-07-09 12:37 ClausKlein Note Added: 0030006
2012-07-10 05:05 Peter Kuemmel Note Added: 0030011
2012-07-10 05:06 Peter Kuemmel Note Edited: 0029996
2012-07-10 14:17 Zaheer Chothia Note Added: 0030026
2012-07-10 14:17 Zaheer Chothia File Added: issue13385_build_ninja_mingw.zip
2012-07-11 03:08 Peter Kuemmel Note Added: 0030033
2012-07-11 04:44 Peter Kuemmel Note Added: 0030034
2012-07-11 04:45 Peter Kuemmel Assigned To => Peter Kuemmel
2012-07-11 05:48 Zaheer Chothia Note Added: 0030035
2012-07-11 05:58 Peter Kuemmel Note Added: 0030036
2012-07-11 06:21 Zaheer Chothia Note Added: 0030037
2012-08-12 06:16 Peter Kuemmel Relationship added related to 0013461
2012-08-12 06:26 Peter Kuemmel Status confirmed => assigned
2012-08-13 08:34 Peter Kuemmel Status assigned => resolved
2012-08-13 08:34 Peter Kuemmel Resolution open => fixed
2012-08-13 08:34 Peter Kuemmel Target Version => CMake 2.8.9
2013-01-09 10:57 Robert Maynard Note Added: 0032048
2013-01-09 10:57 Robert Maynard Status resolved => closed


Copyright © 2000 - 2018 MantisBT Team