MantisBT - CMake
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0006284CMakeCMakepublic2008-01-29 06:262014-04-14 14:02
ReporterMathieu MARACHE 
Assigned ToBrad King 
PrioritynormalSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version 
Summary0006284: Linking static libraries with long list of objects files with Makefile generators fail
DescriptionDepending on the OS the length of a command line is limited. Creating a static library from a huge number of files (I have 720 for example) crashes the build both under linux and windows with mingw.

Steps To Reproduce
Additional InformationThe command is in fact written in a 'link.txt' file that is called with a 'cmake -E cmake_link_script' command

ar supports having to much arguments from the comand line by using a file to give it's arguments. The syntax is : 'ar @file'

breaking the offending link.txt file into two separate files :
 - leaving the commands in link.txt :
    ar cr ../../libname.a @CMakeFiles/name.dir/files_to_link.txt
    ranlib ../../libname.a
 - and creating a file called file_to_link.txt (obviously :) that contains all the objects files

This works fine. However modifying CMake to fullfil this scheme has proven too hard for my partial knowledge of CMake.

From what I've understood is that one needs to modify the way that the CMAKE_CXX_CREATE_STATIC_LIBRARY expands to create the appropriate.

Maybe someone with better knowledge would be able to point me out the way to work it out or better provide another solution ???
TagsNo tags attached.
Relationships
related to 0014874closed Brad King Static library can miss some object files 
Attached Filespatch CMake.patch (2,288) 2008-01-29 06:48
https://public.kitware.com/Bug/file/1295/CMake.patch
Issue History
Date ModifiedUsernameFieldChange
2008-01-29 06:26Mathieu MARACHENew Issue
2008-01-29 06:48Mathieu MARACHEFile Added: CMake.patch
2008-01-29 06:50Mathieu MARACHENote Added: 0010322
2008-01-29 08:57Bill HoffmanStatusnew => assigned
2008-01-29 08:57Bill HoffmanAssigned To => Brad King
2008-01-29 09:07Brad KingNote Added: 0010324
2008-01-29 09:16Mathieu MARACHENote Added: 0010325
2008-01-29 10:05Brad KingNote Added: 0010327
2008-01-29 10:27Brad KingNote Added: 0010328
2008-01-29 10:37Mathieu MARACHENote Added: 0010329
2008-01-29 10:48Brad KingNote Added: 0010330
2008-01-29 20:48Brad KingStatusassigned => closed
2008-01-29 20:48Brad KingNote Added: 0010337
2008-01-29 20:48Brad KingResolutionopen => fixed
2014-04-14 14:02Brad KingRelationship addedrelated to 0014874

Notes
(0010322)
Mathieu MARACHE   
2008-01-29 06:50   
The patch submitted here works around this issue this is on today CVS version.
It creates a file called object_files_list.txt at the same place than the link.txt

I needed to change the CMAKE_CXX_CREATE_STATIC_LIBRARY to point to this file

HTH
(0010324)
Brad King   
2008-01-29 09:07   
The @ syntax is not available on all ar commands. The whole point of the link.txt script is to support long link lines because make tools cannot handle them.

I just created a simple project that compiles 1000 source files with really long names. The link.txt script has one line that is over 120000 characters. It linked without a problem on Linux. What crashes for you on Linux?

On Windows the command line is limited to 32K, so this is not surprising. We'll have to update the Modules/Platform/*.cmake files appropriately to enable support for the @ syntax where it is available.
(0010325)
Mathieu MARACHE   
2008-01-29 09:16   
Hi brad,
I'm confused... it works well under linux, you are right the problem is only for the Windows/MinGW couple...

My link.txt is /only/ 43kb

My apologies for leading you the wrong way, and thanks for looking into it :-)
(0010327)
Brad King   
2008-01-29 10:05   
It looks like there is no @ syntax on MinGW's archiver. The only way to do this is to invoke the archiver multiple times:

  ar cr libfoo.a src0001.obj src0002.obj ...
  ar r libfoo.a src1000.obj src1001.obj ...
(0010328)
Brad King   
2008-01-29 10:27   
We're close to the 2.6 release and this is pretty obscure. I'm probably not going to get to it for a while. I suggest refactoring your software to have more than one library.
(0010329)
Mathieu MARACHE   
2008-01-29 10:37   
Well, I tried this road before reporting anything. To cut a long story short I can't, these files are generated and interdependent...

my MinGW has the @<file> option :

C:\>c:\MinGW\bin\ar.exe
Usage: c:\MinGW\bin\ar.exe [emulation options] [-]{dmpqrstx}[abcfilNoPsSuvV] [member-name] [count] archive-file file...
       c:\MinGW\bin\ar.exe -M [<mri-script]
 commands:
  d - delete file(s) from the archive
  m[ab] - move file(s) in the archive
  p - print file(s) found in the archive
  q[f] - quick append file(s) to the archive
  r[ab][f][u] - replace existing or insert new file(s) into the archive
  t - display contents of archive
  x[o] - extract file(s) from the archive
 command specific modifiers:
  [a] - put file(s) after [member-name]
  [b] - put file(s) before [member-name] (same as [i])
  [N] - use instance [count] of name
  [f] - truncate inserted file names
  [P] - use full path names when matching
  [o] - preserve original dates
  [u] - only replace files that are newer than current archive contents
 generic modifiers:
  [c] - do not warn if the library had to be created
  [s] - create an archive index (cf. ranlib)
  [S] - do not build a symbol table
  [v] - be verbose
  [V] - display the version number
  @<file> - read options from <file>
 emulation options:
  No emulation specific options
c:\MinGW\bin\ar.exe: supported targets: pe-i386 pei-i386 elf32-i386 elf32-little elf32-big srec symbolsrec tekhex binary ihex
(0010330)
Brad King   
2008-01-29 10:48   
Well, my mingw does not have the option, so we cannot assume it is present.

CMake does support cyclic dependencies among static libraries:

add_library(foo1 STATIC src1.c src2.c ...)
add_library(foo2 STATIC src3.c src4.c ...)
target_link_libraries(foo1 foo2)
target_link_libraries(foo2 foo1)
(0010337)
Brad King   
2008-01-29 20:48   
Okay, so I got to this earlier than I thought I would.

I've created new rules for archive creation and enabled them on MinGW.
Object file lists are split into groups that do not exceed 32K command line lengths and the archive is built incrementally.

/cvsroot/CMake/CMake/Modules/Platform/Windows-gcc.cmake,v <-- Windows-gcc.cmake
new revision: 1.20; previous revision: 1.19
/cvsroot/CMake/CMake/Source/cmDocumentVariables.cxx,v <-- cmDocumentVariables.cxx
new revision: 1.13; previous revision: 1.12
/cvsroot/CMake/CMake/Source/cmMakefileLibraryTargetGenerator.cxx,v <-- cmMakefileLibraryTargetGenerator.cxx
new revision: 1.52; previous revision: 1.51
/cvsroot/CMake/CMake/Source/cmMakefileTargetGenerator.cxx,v <-- cmMakefileTargetGenerator.cxx
new revision: 1.88; previous revision: 1.87
/cvsroot/CMake/CMake/Source/cmMakefileTargetGenerator.h,v <-- cmMakefileTargetGenerator.h
new revision: 1.20; previous revision: 1.19