MantisBT - CMake |
View Issue Details |
|
ID | Project | Category | View Status | Date Submitted | Last Update |
0005430 | CMake | CPack | public | 2007-07-28 15:28 | 2011-01-18 16:33 |
|
Reporter | irwin | |
Assigned To | Eric NOULARD | |
Priority | normal | Severity | minor | Reproducibility | always |
Status | closed | Resolution | fixed | |
Platform | | OS | | OS Version | |
Product Version | | |
Target Version | | Fixed in Version | CMake 2.8.3 | |
|
Summary | 0005430: cpack does not preserve symbolic links in the source distribution tarball |
Description | I have some data directories in one of my source trees which are so large that they dominate the size of the source tree. For convenience they are symlinked from a number of different directories in the source tree. However, the source distribution tarball generated by
make package_source
is extremely large (five times the size which it ought to be for this particular case) because the symlinks are not preserved and instead are replaced with copies of the large data directories.
Symlinks are normally put into the source tree for good reasons. (For example, the cvs replacement subversion deals properly with symlinks for exactly this reason.) Thus, cpack should always preserve symlinks when creating a tarball. |
Steps To Reproduce | |
Additional Information | |
Tags | No tags attached. |
Relationships | related to | 0010162 | closed | Eric NOULARD | CPack Cannot create binary zip archive containing shared libs | related to | 0010096 | closed | Bill Hoffman | cpack does not preserve symlinks in CPACK_INSTALLED_DIRECTORIES |
|
Attached Files | cpack-duplicate-shared-libs.tgz (21,421) 2010-07-29 06:52 https://public.kitware.com/Bug/file/3279/*
0001-CPack-handle-symlinks-in-CPACK_INSTALLED_DIRECTORIES.patch (4,080) 2010-07-29 08:37 https://public.kitware.com/Bug/file/3280/0001-CPack-handle-symlinks-in-CPACK_INSTALLED_DIRECTORIES.patch |
|
Issue History |
Date Modified | Username | Field | Change |
2007-08-31 11:27 | Alex Neundorf | Assigned To | System Admin => Bill Hoffman |
2009-12-02 15:43 | engels | Note Added: 0018659 | |
2009-12-02 15:59 | engels | Note Edited: 0018659 | |
2009-12-02 15:59 | Bill Hoffman | Note Added: 0018660 | |
2009-12-02 16:04 | engels | Note Added: 0018661 | |
2009-12-02 16:17 | engels | Note Edited: 0018661 | |
2009-12-02 16:18 | Bill Hoffman | Note Added: 0018662 | |
2009-12-02 16:20 | engels | Note Added: 0018663 | |
2009-12-02 16:56 | engels | Note Edited: 0018663 | |
2010-02-23 09:15 | engels | Note Added: 0019594 | |
2010-06-18 12:50 | David Cole | Summary | cpack does not preserve symbolic links in the source
distribution tarball => cpack does not preserve symbolic links in the source distribution tarball |
2010-07-26 07:29 | Eric NOULARD | Relationship added | related to 0010162 |
2010-07-26 07:34 | Eric NOULARD | Note Added: 0021480 | |
2010-07-26 07:35 | Eric NOULARD | Relationship added | related to 0010096 |
2010-07-26 07:36 | Eric NOULARD | Note Added: 0021481 | |
2010-07-27 09:57 | Bill Hoffman | Note Added: 0021491 | |
2010-07-27 11:35 | Eric NOULARD | Note Added: 0021493 | |
2010-07-29 06:52 | Eric NOULARD | File Added: cpack-duplicate-shared-libs.tgz | |
2010-07-29 08:37 | Eric NOULARD | File Added: 0001-CPack-handle-symlinks-in-CPACK_INSTALLED_DIRECTORIES.patch | |
2010-07-29 08:43 | Eric NOULARD | Note Added: 0021538 | |
2010-07-29 13:12 | engels | Note Added: 0021555 | |
2010-07-29 13:21 | Eric NOULARD | Note Added: 0021556 | |
2010-08-28 10:51 | Eric NOULARD | Assigned To | Bill Hoffman => Eric NOULARD |
2010-08-28 10:53 | Eric NOULARD | Note Added: 0021993 | |
2010-08-28 10:53 | Eric NOULARD | Status | assigned => closed |
2010-08-28 10:53 | Eric NOULARD | Resolution | open => fixed |
2010-08-28 10:53 | Eric NOULARD | Fixed in Version | => CMake-2-8 |
2010-08-31 16:32 | Eric NOULARD | Note Added: 0022033 | |
2010-08-31 16:32 | Eric NOULARD | Status | closed => feedback |
2010-08-31 16:32 | Eric NOULARD | Resolution | fixed => reopened |
2010-10-06 14:24 | David Cole | Fixed in Version | CMake-2-8 => |
2011-01-18 11:18 | David Cole | Note Added: 0024862 | |
2011-01-18 12:40 | engels | Note Added: 0024883 | |
2011-01-18 12:51 | engels | Note Edited: 0024883 | bug_revision_view_page.php?bugnote_id=24883#r139 |
2011-01-18 14:11 | Alan W. Irwin | Note Added: 0024891 | |
2011-01-18 14:17 | David Cole | Note Added: 0024892 | |
2011-01-18 15:35 | Eric NOULARD | Note Added: 0024894 | |
2011-01-18 16:33 | Eric NOULARD | Note Added: 0024898 | |
2011-01-18 16:33 | Eric NOULARD | Status | feedback => closed |
2011-01-18 16:33 | Eric NOULARD | Resolution | reopened => fixed |
2011-01-18 16:33 | Eric NOULARD | Fixed in Version | => CMake 2.8.3 |
Notes |
|
(0018659)
|
engels
|
2009-12-02 15:43
(edited on: 2009-12-02 15:59) |
|
Hi everyone,
I also think symbolic links should always be preserved (not only in source distribution tarballs). When packing shared libraries this can quickly become a major problem, since each library typically has two symbolic links for it's version.
If possible could someone please raise the priority of this issue?
Thanks, Jan.
|
|
|
(0018660)
|
Bill Hoffman
|
2009-12-02 15:59
|
|
Is this still a problem? Jan do have a case where shrared libraries are duplicated in cpack? |
|
|
(0018661)
|
engels
|
2009-12-02 16:04
(edited on: 2009-12-02 16:17) |
|
Hi Bill,
sorry, I did not test properly. this problem does not occur when packaging a cmake project. only when using cpack without cmake.
|
|
|
(0018662)
|
Bill Hoffman
|
2009-12-02 16:18
|
|
|
|
(0018663)
|
engels
|
2009-12-02 16:20
(edited on: 2009-12-02 16:56) |
|
|
|
(0019594)
|
engels
|
2010-02-23 09:15
|
|
Hi Bill, are there any news on this issue? |
|
|
(0021480)
|
Eric NOULARD
|
2010-07-26 07:34
|
|
I did add a link to a "related bug" concerning symlinks in ZIP archive.
Just to note that for CPackTGZ (binary) package the symlinks are preserved
if CMake+CPack is used.
However symlinks are not preserved if:
package_source
or
direct cpack usage as in the example from engels |
|
|
(0021481)
|
Eric NOULARD
|
2010-07-26 07:36
|
|
Adding one more relationship. |
|
|
(0021491)
|
Bill Hoffman
|
2010-07-27 09:57
|
|
If someone wants to create a patch for this that would be great. I think it is just figuring out what the arguments to libarchive calls need to be for this to work. |
|
|
(0021493)
|
Eric NOULARD
|
2010-07-27 11:35
|
|
Hi Bill,
I've just has a deeper look into this.
It appears that libarchive is not the culprit.
The trouble comes from the usage of "CPACK_INSTALLED_DIRECTORIES"
which is copied by cmake from "INSTALLED DIRS" to
CPack own "CPACK_TEMPORARY_INSTALL_DIRECTORY"
have a look at: int cmCPackGenerator::InstallProjectViaInstalledDirectories.
which ends up in a:
cmSystemTools::CopyFileIfDifferent
which clearly does not handle symbolic links.
so the loss of symbolic links has nothing to do with libarchive.
libarchive seems to handle symbolic links just fine for TAR archives,
for ZIP it does not (and there is no option for that) see bug 10162.
I don't really know is it would be a desirable behavior for
cmSystemTools::CopyFileIfDifferent to handle symbolic links?
And this is not that easy to do, because:
- one should check if the target file is part of the files to be package
or you'll end-up with a dangling link
- does every usage of cmSystemTools::CopyFileIfDifferent implies
we want to replicate the symlink or do we sometimes want to copy
the target...
May be the good way to go would be to filter out symlinks
InstallProjectViaInstalledDirectories
and treat them after "regular" files have been copied. |
|
|
(0021538)
|
Eric NOULARD
|
2010-07-29 08:43
|
|
Hi Bill,
You'll find a patch proposal attach.
The patch does modify the behavior of "InstallProjectViaInstalledDirectories":
- if a file is a symlink we do not "CopyFileIfDifferent"
we remember the name of the link (and the target)
- at the end of the processing of the current INSTALLED dir
we recreate the links.
The patch is OK on engels example.
It should work if the link are "local" to each directory of INSTALLED dir.
(We may do more to support "cross-directory" links but I'm not sure
it's worth the extra complexity).
I cannot test how it behaves on platform not supporting symlinks.
However it should be harmless since in this case
FileIsSymlink should always answer "false".
I did re-check that the problem does NOT occurs for in-source links
and "make package_source". |
|
|
(0021555)
|
engels
|
2010-07-29 13:12
|
|
Hi Eric,
thank you very much for the patch. I did not yet try it but I will asap and hopefully I will then be able to use cpack to also package some external dependencies from our main project.
I tried to also look into fixing this problem, but as I started digging into the code I found out that it was not as trivial as I thought and that I do not yet have enough experience with the cmake source code.
So, once again.. thanks a lot!
Jan |
|
|
(0021556)
|
Eric NOULARD
|
2010-07-29 13:21
|
|
Hi Jan,
No problem.
Tell us if it works for you, this way the patch may find its way to
next CMake release.
And don't be afraid by code digging :-) |
|
|
(0021993)
|
Eric NOULARD
|
2010-08-28 10:53
|
|
Pushed to next:
commit 013e0039eeb7e8bf213c3851161ce9a18f7b940d
Author: Eric NOULARD <eric.noulard@gmail.com>
Date: Sat Aug 28 16:50:06 2010 +0200
CPack handle symlinks in CPACK_INSTALLED_DIRECTORIES fix for bug5430
The proposed solution is to avoid to CopyIfDifferent any links
found in CPACK_INSTALLED_DIRECTORIES but memorize them instead
then at the end of the processing for each dir do re-create the
link in the installed tree.
Current patch should work if the link are "local" to the directory. |
|
|
(0022033)
|
Eric NOULARD
|
2010-08-31 16:32
|
|
> The proposed solution is to avoid to CopyIfDifferent any links
> found in CPACK_INSTALLED_DIRECTORIES but memorize them instead
> then at the end of the processing for each dir do re-create the
> link in the installed tree.
> Current patch should work if the link are "local" to the directory.
This brings up another place for refactoring. It looks like the code
this patch modifies does the same thing that the file(COPY) and
file(INSTALL) commands do. In cmFileCommand.cxx there is a class
cmFileCopier and subclass cmFileInstaller used to implement those
commands. These could be factored out of this file and used in
CPack to install the directories into the package more robustly. |
|
|
(0024862)
|
David Cole
|
2011-01-18 11:18
|
|
Looking at "older" "feedback" status bugs today... is this still an issue that we want to address moving forward? If so, please remove the "feedback" status and just set it to "assigned"... If not, please resolve it.
Thanks. |
|
|
(0024883)
|
engels
|
2011-01-18 12:40
(edited on: 2011-01-18 12:51) |
|
From my side this issue can be closed. The problem was fixed in version 2.8.3. Sorry, reading back previous posts I just realized that I forgot to give feedback to Eric regarding the patch. Sorry
|
|
|
(0024891)
|
Alan W. Irwin
|
2011-01-18 14:11
|
|
I am the original reporter of this bug, but I am currently not in a good
position to test the fix. However, from the above notes it does appear to be
fixed in 2.8.3 so please go ahead and close it. I will open another bug
if it turns out there are still some issues left once I can test it
again. |
|
|
(0024892)
|
David Cole
|
2011-01-18 14:17
|
|
Eric,
Was there a reason you wanted to leave this one open still, despite the original reporters saying the issue is fixed...?
Something about what's in the notes trail?
If not, please go ahead and resolve it as fixed in 2.8.3 or 2.8.4, whichever is appropriate.
Thanks,
David |
|
|
(0024894)
|
Eric NOULARD
|
2011-01-18 15:35
|
|
I'll check later tonight.
As far as I remember my patch didn't reached master
(not merged from next) because the
patch was doing something similar to other CMake code which
was more up-to-date.
I'll check the status of current master (later tonight)
and close it if it was finally merged. |
|
|
(0024898)
|
Eric NOULARD
|
2011-01-18 16:33
|
|
|