[CMake] LINK_LIBRARIES not spilled to response file

Zaheer Chothia zaheer.chothia at gmail.com
Mon Jul 9 06:13:59 EDT 2012 

Done: http://public.kitware.com/Bug/view.php?id=13385

--Zaheer

On Mon, Jul 9, 2012 at 8:15 AM, Brett Delle Grazie
<brett.dellegrazie at gmail.com> wrote:
>
> On Jul 8, 2012 10:59 PM, "Zaheer Chothia" <zaheer.chothia at gmail.com> wrote:
>>
>> Hello,
>>
>> I posted a mail here [1] but have yet to receive any replies.  I think my
>> last
>> message was too long and detailed so let me summarize:
>> * Command line length is limited on Windows.
>> * To alleviate this, CMake places object files into a response file, but
>> the
>>   same is not done for libraries.
>> * This can pose a problem when the link line becomes too long.
>>
>> I am primarily interested in a solution for the Ninja generator, although
>> this
>> issue affects other generators too.  My previous mail contains a testcase
>> and
>> proposed solution.  In the interim another issue [2] was posted, but that
>> is
>> orthogonal and does not solve what is discussed here.
>
> I'd suggest you raise a defect. Kitware are pretty good at fixing things or
> suggesting temporary workarounds until the underlying issue is fixed.
>
>>
>> Kind regards,
>>
>> --Zaheer
>>
>> [1]: http://www.cmake.org/pipermail/cmake/2012-June/051065.html
>> [2]: http://public.kitware.com/Bug/view.php?id=13366
>>
>> On Thu, Jun 28, 2012 at 3:50 PM, Zaheer Chothia <zaheer.chothia at gmail.com>
>> wrote:
>>>
>>> Hello,
>>>
>>> I encountered an issue while building a CMake project where one target is
>>> linked
>>> against a large number of libraries.  Unlike object files, libraries are
>>> not
>>> placed into a response file, which can lead to build commands which
>>> exceed the
>>> length limits on Windows.  For reference, I am using the CMake 2.8.9-rc1
>>> and
>>> Ninja generator with Microsoft compilers.
>>>
>>> Following this mail is a testcase generator [1] to demonstrate this issue
>>> (sample project attached for convenience).  The build fails with this
>>> error (for
>>> readibility I replaced a long sequence of libraries with <...>):
>>>
>>>     FAILED: cmd.exe /c cd. && "C:\Program Files
>>> (x86)\CMake\bin\cmake.exe" -E vs_link_exe
>>> C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe  /nologo @hello.exe.rsp  /DWIN32
>>> /D_WINDOWS /W3 /Zm1000 /D_DEBUG /MDd /Zi  /Ob0 /Od /RTC1 /Fehello.exe
>>> /Fdhello.pdb -link /implib:hello.lib /version:0.0  /STACK:10000000
>>> /machine:X86  /debug /INCREMENTAL /subsystem:console
>>> src\abcdefghijklmnopqrstuvwxyz0123456789\library1.lib
>>> src\abcdefghijklmnopqrstuvwxyz0123456789\library2.lib <...>
>>> kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib
>>> oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd.
>>>     The command line is too long.
>>>
>>>     ninja: build stopped: subcommand failed.
>>>
>>> Although this example may seem artificial, with the use case I refer to
>>> (i)
>>> libraries are are specified by absolute paths so they are indeed
>>> reasonably long
>>> and (ii) since there are third-party libraries involved I would not be
>>> able to
>>> simply combine source files into one large library as is possible here.
>>> I
>>> should also mention that this issue does not affect the Visual Studio
>>> generators, however it is present with the following: Ninja, MinGW
>>> Makefiles, NMake Makefiles, MSYS Makefiles.  For Ninja I suspect that the
>>> indirection via cmd.exe imposes a maximum command length of 8192 KB,
>>> whereas for
>>> the others this will likely be 32 KB (CreateProcess).
>>>
>>> I would be quite content if this is fixed for the Ninja generator.  A
>>> simple fix
>>> would be to adapt the build rules by moving $LINK_LIBRARIES from
>>> 'command' to
>>> 'rspfile_content':
>>>
>>>     --- rules.ninja.bak 2012-06-28 15:23:35 +0100
>>>     +++ rules.ninja     2012-06-28 15:38:09 +0100
>>>     @@ -40,10 +40,10 @@
>>>      # Rule for linking C executable.
>>>
>>>      rule C_EXECUTABLE_LINKER_RSPFILE
>>>     -  command = cmd.exe /c $PRE_LINK && "C:\Program Files
>>> (x86)\CMake\bin\cmake.exe" -E vs_link_exe
>>> C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe  /nologo @$out.rsp  $FLAGS
>>> /Fe$out /Fd$TARGET_PDB -link /implib:$TARGET_IMPLIB /version:0.0
>>> $LINK_FLAGS $LINK_LIBRARIES && $POST_BUILD
>>>     +  command = cmd.exe /c $PRE_LINK && "C:\Program Files
>>> (x86)\CMake\bin\cmake.exe" -E vs_link_exe
>>> C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe  /nologo @$out.rsp  $FLAGS
>>> /Fe$out /Fd$TARGET_PDB -link /implib:$TARGET_IMPLIB /version:0.0
>>> $LINK_FLAGS && $POST_BUILD
>>>        description = Linking C executable $out
>>>        rspfile = $out.rsp
>>>     -  rspfile_content = $in
>>>     +  rspfile_content = $in $LINK_LIBRARIES
>>>
>>> Best,
>>>
>>> --Zaheer
>>>
>>> ---- [1]: BEGIN: testcase.sh
>>> ---------------------------------------------------
>>> #!/bin/bash -e
>>>
>>> NUM_LIBRARIES=500
>>> # Use a long path to quickly exhaust the command-line length limit.
>>> SRC_DIR=src/abcdefghijklmnopqrstuvwxyz0123456789
>>>
>>> # Root directory: application and CMakeLists.txt
>>> echo "int main() { return 0; }" > hello.c
>>>
>>> cat << EOF > CMakeLists.txt
>>> cmake_minimum_required(VERSION 2.8)
>>>
>>> project(Hello)
>>>
>>> add_subdirectory($SRC_DIR)
>>>
>>> add_executable(hello hello.c)
>>> target_link_libraries(hello
>>> EOF
>>>
>>> for ((i = 1; i <= $NUM_LIBRARIES; i++)); do
>>>     echo "    library$i" >> "CMakeLists.txt"
>>> done
>>>
>>> echo ")" >> "CMakeLists.txt"
>>>
>>> # Libraries: sources and CMakeLists.txt
>>> mkdir -p "$SRC_DIR"
>>> [[ -f "$SRC_DIR/CMakeLists.txt" ]] && rm "$SRC_DIR/CMakeLists.txt"
>>> for ((i = 1; i <= $NUM_LIBRARIES; i++)); do
>>>     echo "int function$i() { return $i; }" > "$SRC_DIR/function$i.c"
>>>     echo "add_library(library$i function$i.c)" >>
>>> "$SRC_DIR/CMakeLists.txt"
>>> done
>>>
>>> echo "Testcase has been setup: now build with CMake and Ninja generator."
>>> ---- [1]: END: testcase.sh
>>> ---------------------------------------------------
>>
>>
>>
>> --
>>
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Please keep messages on-topic and check the CMake FAQ at:
>> http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.cmake.org/mailman/listinfo/cmake

More information about the CMake mailing list