MantisBT - CMake
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0014892CMakeCMakepublic2014-04-20 06:462015-06-01 08:38
ReporterМаксим Прохоренко 
Assigned ToBen Boeckel 
PrioritynormalSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformLinuxOSFedoraOS Version20
Product VersionCMake 2.8.12.2 
Target VersionCMake 3.2Fixed in VersionCMake 3.2 
Summary0014892: Ninja: wrong check for link with RSP_FILE
Descriptionninja use sh -c 'cmd' for all commands, limit for 'cmd' less than _SC_ARG_MAX

MAX_ARG_STRLEN or sysconf(_SC_PAGESIZE) * 32
Steps To Reproduceuse CMakeLists.txt (attached)

cmake -GNinja ..

ninja-build

Argument list too long
ninja: build stopped: subcommand failed.
Additional Information
TagsNo tags attached.
Relationships
Attached Filestxt CMakeLists.txt (911) 2014-04-20 06:46
https://public.kitware.com/Bug/file/5135/CMakeLists.txt
patch 0001-Ninja-fix-length-limit-for-link-command-fix-for-v3.0.patch (1,206) 2014-04-20 06:46
https://public.kitware.com/Bug/file/5136/0001-Ninja-fix-length-limit-for-link-command-fix-for-v3.0.patch
patch 0001-Ninja-fix-length-limit-for-rsp-file-generation.patch (1,144) 2015-01-14 12:57
https://public.kitware.com/Bug/file/5346/0001-Ninja-fix-length-limit-for-rsp-file-generation.patch
Issue History
Date ModifiedUsernameFieldChange
2014-04-20 06:46Максим ПрохоренкоNew Issue
2014-04-20 06:46Максим ПрохоренкоFile Added: CMakeLists.txt
2014-04-20 06:46Максим ПрохоренкоFile Added: 0001-Ninja-fix-length-limit-for-link-command-fix-for-v3.0.patch
2014-04-21 08:52Brad KingNote Added: 0035748
2014-04-21 10:10Максим ПрохоренкоNote Added: 0035750
2014-04-21 10:14Максим ПрохоренкоNote Edited: 0035750bug_revision_view_page.php?bugnote_id=35750#r1451
2014-04-21 10:33Brad KingNote Added: 0035751
2014-04-21 13:23Максим ПрохоренкоNote Added: 0035753
2014-04-21 13:51Peter KuemmelNote Added: 0035754
2014-04-21 14:07Peter KuemmelNote Added: 0035756
2014-04-21 14:09Brad KingNote Added: 0035757
2015-01-14 03:40Максим ПрохоренкоNote Added: 0037682
2015-01-14 08:35Brad KingAssigned To => Ben Boeckel
2015-01-14 08:35Brad KingStatusnew => assigned
2015-01-14 08:35Brad KingTarget Version => CMake 3.2
2015-01-14 12:57Ben BoeckelFile Added: 0001-Ninja-fix-length-limit-for-rsp-file-generation.patch
2015-01-26 09:52Brad KingNote Added: 0037809
2015-01-26 09:52Brad KingStatusassigned => resolved
2015-01-26 09:52Brad KingResolutionopen => fixed
2015-01-26 09:52Brad KingFixed in Version => CMake 3.2
2015-06-01 08:38Robert MaynardNote Added: 0038859
2015-06-01 08:38Robert MaynardStatusresolved => closed

Notes
(0035748)
Brad King   
2014-04-21 08:52   
Thanks. Can you please provide a link to documentation describing the PAGE_SIZE*32 limit so it can be included in the commit message?
(0035750)
Максим Прохоренко   
2014-04-21 10:10   
(edited on: 2014-04-21 10:14)
/usr/include/linux/binfmts.h


/*
 * These are the maximum length and maximum number of strings passed to the
 * execve() system call. MAX_ARG_STRLEN is essentially random but serves to
 * prevent the kernel from being unduly impacted by misaddressed pointers.
 * MAX_ARG_STRINGS is chosen to fit in a signed 32-bit integer.
 */
#define MAX_ARG_STRLEN (PAGE_SIZE * 32)
#define MAX_ARG_STRINGS 0x7FFFFFFF
...

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=b6a2fea39318e43fee84fa7b0b90d68bed92d2ba [^]

http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/include/linux/binfmts.h?id=HEAD [^]

http://unix.stackexchange.com/questions/120642/what-defines-the-maximum-size-for-a-command-single-argument [^]
(0035751)
Brad King   
2014-04-21 10:33   
Re 0014892:0035750: MAX_ARG_STRLEN appears to be about the longest allowed single argv[i] entry, not the overall command line length limit. They may happen to be the same in some cases.

For reference, the current logic came from here:

 Ninja: also write link libraries to rsp file
 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=7f647cf5 [^]

According to the stackexchange link the value returned by sysconf(_SC_ARG_MAX) may depend on the current process stack size rather than being a system-wide limit of the OS. The size in the cmake process may be different than in the ninja process that actually launches the command line.

Perhaps ninja needs a way to ask for the max command line length it can run without a response file.
(0035753)
Максим Прохоренко   
2014-04-21 13:23   
Вefore that I was trying to fix the ninja too

https://github.com/gendalph/ninja/compare/martine:master...master [^]
https://github.com/martine/ninja/pull/520 [^]

But only for fast fix. This is not the true way.
I checked that the gnu make is working properly and
discussed this problem with Martin.

ninja is a simple utility not gnu make.
if use an implementation from the gnu make will be greatly complicate the code.
(0035754)
Peter Kuemmel   
2014-04-21 13:51   
http://www.in-ulm.de/~mascheck/various/argmax/ [^] :

And as additional limit since 2.6.23, one argument must not be longer than MAX_ARG_STRLEN (131072).
This might become relevant if you generate a long call like "sh -c 'generated with long arguments'".


So I think not using ARG_MAX is the best way.
(0035756)
Peter Kuemmel   
2014-04-21 14:07   
http://cmake.org/gitweb?p=stage/cmake.git;a=commitdiff;h=2270580a9cf33e1e54899963bcb9548e653ca875 [^]

Is this OK? Then I could merge.
(0035757)
Brad King   
2014-04-21 14:09   
Re 0014892:0035756: Please extend the commit message to explain the rationale.

Perhaps the code should check both limits and take the smaller.
(0037682)
Максим Прохоренко   
2015-01-14 03:40   
"sysconf(_SC_PAGESIZE) * 32" solve my problem

Can you add to 3.1?
(0037809)
Brad King   
2015-01-26 09:52   
This should fix it:

 ninja: use the minimum of all command line length limits
 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=097e26f4 [^]
(0038859)
Robert Maynard   
2015-06-01 08:38   
Closing resolved issues that have not been updated in more than 4 months.