MantisBT - CMake
View Issue Details
0011958CMakeCMakepublic2011-03-10 06:422011-11-11 10:45
simonh 
Brad King 
normalminoralways
closedfixed 
CMake 2.8.4 
CMake 2.8.5CMake 2.8.5 
0011958: libarchive still produces PAX tar archives which are unreadable on older versions of tar
As per issue 0011176, which I'd re-open if I could, and as per my comment 0011176:0021986, the current libarchive will generate extended header keywords such as "LIBARCHIVE.xattr.security.selinux" which cause some tar programs to error rather than silently ignore (such as http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/7/Fedora/source/SRPMS/tar-1.15.1-26.fc7.src.rpm [^])

I know this is really a tar problem (shouldn't error), but it means I have to patch CMake to use "ustar" instead of "pax".
No tags attached.
related to 0011176closed Brad King "cmake -E tar" encodes long paths such that Fedora GNU tar 1.15.1 reports errors 
related to 0011020closed Brad King CPack produces uninstallable DEB files with 2.8.2, worked with 2.8.1 
Issue History
2011-03-10 06:42simonhNew Issue
2011-03-10 06:43simonhNote Added: 0025727
2011-03-10 07:29Brad KingRelationship addedrelated to 0011176
2011-03-10 08:50Brad KingRelationship addedrelated to 0011020
2011-03-10 08:55Brad KingNote Added: 0025728
2011-03-10 08:55Brad KingAssigned To => Brad King
2011-03-10 08:55Brad KingStatusnew => assigned
2011-03-10 09:34Brad KingNote Added: 0025730
2011-03-29 19:29Todd HarringtonNote Added: 0025960
2011-03-29 19:34Todd HarringtonNote Edited: 0025960bug_revision_view_page.php?bugnote_id=25960#r263
2011-03-30 13:35Brad KingNote Added: 0025973
2011-03-31 15:15Todd HarringtonNote Added: 0025994
2011-03-31 16:20Brad KingNote Added: 0025995
2011-03-31 16:21Brad KingNote Added: 0025996
2011-03-31 16:28Todd HarringtonNote Added: 0025997
2011-03-31 16:36Brad KingNote Added: 0025998
2011-03-31 17:06Brad KingNote Added: 0026000
2011-03-31 17:08Brad KingNote Edited: 0026000bug_revision_view_page.php?bugnote_id=26000#r275
2011-04-01 08:45simonhNote Added: 0026006
2011-04-01 09:34Brad KingNote Added: 0026008
2011-04-04 04:25simonhNote Added: 0026031
2011-04-07 09:02Brad KingNote Added: 0026134
2011-04-12 16:23Brad KingStatusassigned => resolved
2011-04-12 16:23Brad KingResolutionopen => fixed
2011-06-17 18:37David ColeFixed in Version => CMake 2.8.5
2011-06-17 18:37David ColeTarget Version => CMake 2.8.5
2011-11-11 10:45David ColeNote Added: 0027770
2011-11-11 10:45David ColeStatusresolved => closed

Notes
(0025727)
simonh   
2011-03-10 06:43   
An example of the current output:
DEBUG util.py:256:  tar: Ignoring unknown extended header keyword `LIBARCHIVE.xattr.security.selinux'

DEBUG util.py:256:  tar: Ignoring unknown extended header keyword `LIBARCHIVE.xattr.security.selinux'

DEBUG util.py:256:  tar: Error exit delayed from previous errors
DEBUG util.py:256:  RPM build errors:
DEBUG util.py:256:  error: Bad exit status from /var/tmp/rpm-tmp.34269 (%prep)
DEBUG util.py:256:      Bad exit status from /var/tmp/rpm-tmp.34269 (%prep)
DEBUG util.py:330:  Child returncode was: 1
INFO util.py:283:  EXCEPTION: Command failed. See logs for output.
(0025728)
Brad King   
2011-03-10 08:55   
The code has been refactored due to the related issues since your original patch. What patch to CMake do you now propose?
(0025730)
Brad King   
2011-03-10 09:34   
I cannot reproduce this. I tested with an "input/tmp" directory that contains 38 files one of which is at a path over 900 characters deep.

  $ cmake --version
  cmake version 2.8.4
  $ tar --version
  tar (GNU tar) 1.15.1
  $ (cd input; cmake -E tar c ../tmp.tar tmp)
  $ tar xf tmp.tar
  $ diff -r input/tmp tmp
  (no differences)


The tar was created as described in 0011176:0021989 and has all the patches from the RPM:

patch -p1 -b < ../tar-1.14-nolibrt.patch
patch -p1 -b < ../tar-1.14-loneZeroWarning.patch
patch -p1 -b < ../tar-1.15.1-makeCheck.patch
patch -p1 -b < ../tar-1.15.1-gcc4.patch
patch -p1 -b < ../tar-1.15.1-lseek.patch
patch -p1 -b < ../tar-1.15.1-sparseTotals.patch
patch -p1 -b < ../tar-1.15.1-newerOption.patch
patch -p1 -b < ../tar-1.15.1-padCorrectly.patch
patch -p1 -b < ../tar-1.15.1-vfatTruncate.patch
patch -p0 -b < ../tar-1.15.1-heapOverflow.patch
patch -p1 -b < ../tar-1.15.1-hugeSparse.patch
patch -p1 -b < ../tar-1.15.1-optionsOrder.patch
patch -p1 -b < ../tar-1.15.1-permissions.patch
patch -p1 -b < ../tar-1.15.1-incompatibilities.patch
patch -p1 -b < ../tar-1.15.1-xattrs.patch
patch -p1 -b < ../tar-1.15.1-mangling.patch
(0025960)
Todd Harrington   
2011-03-29 19:29   
(edited on: 2011-03-29 19:34)
I am having the exact same issue:

tar: Ignoring unknown extended header keyword `LIBARCHIVE.xattr.security.selinux'
tar: Ignoring unknown extended header keyword `LIBARCHIVE.xattr.security.selinux'
tar: Error exit delayed from previous errors

However, I am using the STGZ CPack target. Building the package succeeds. However, running the generated installer script outputs hundreds of the above messages, and ends with the error message.

The result seems fine, but I don't want to deliver an installer that claims to have failed. Any help would be appreciated.

  $ cmake --version
  cmake version 2.8.4
  $ tar --version
  tar (GNU tar) 1.15.1

CMake/CPack was run on CentOS 5.2.
The installer was run on CentSO 5.2 and 5.4 with the same result.

(0025973)
Brad King   
2011-03-30 13:35   
I still cannot reproduce this. Where did you get CMake? How was it built?

The text "selinux" does not appear in libarchive's entire history or in CMake's source tree. Here is the function that adds the headers:

  http://cmake.org/gitweb?p=cmake.git;a=blob;f=Utilities/cmlibarchive/libarchive/archive_write_set_format_pax.c;hb=v2.8.4#l399 [^]

I see nothing about security or selinux.

I run "make package" in CMake's own source tree. Then I can extract it perfectly using the tar 1.15.1 in 0011958:0025730.

Please attach a sample tarball produced by CMake that shows the problem.
(0025994)
Todd Harrington   
2011-03-31 15:15   
I also searched for "selinux" and could not find it in the source. I have no idea where its coming from.

I tried building cmake against the system libarchive, but that did not change anything.

I ran "make package" for cmake-2.8.4 and the result *.tar.gz file has the same issue. However, I was unable to attach the file as it is 9MB and the upload limit is 5MB. Is there another way for me to get the file to you?
(0025995)
Brad King   
2011-03-31 16:20   
The file you sent me via email extracts with the same warnings even on modern GNU tar 1.23:

  tar: Ignoring unknown extended header keyword `LIBARCHIVE.xattr.security.selinux'
(0025996)
Brad King   
2011-03-31 16:21   
I think this is reproducible only on filesystems with selinux security attributes enabled. I think libarchive is copying the attributes out of the filesystem. That is why the strings do not appear anywhere in the libarchive source.

Here is a similar report in upstream libarchive's issue tracker:

  http://code.google.com/p/libarchive/issues/detail?id=131 [^]
(0025997)
Todd Harrington   
2011-03-31 16:28   
Is there a way for CMake/CPack to tell libarchive not to include any xattr values? Those attributes seem more geared for backups and don't seem relevant to installation packages. Not to mention that they are not supported by GNU tar anyway.
(0025998)
Brad King   
2011-03-31 16:36   
Re 0011958:0025997: As noted in the original description of this issue we can set the type to "ustar" to avoid storing any xattr headers. However, then the format cannot handle long file names. In the same test directory I describe in 0011958:0025730 I try to create a ustar archive but get:

  archive_write_header: Pathname too long
(0026000)
Brad King   
2011-03-31 17:06   
(edited on: 2011-03-31 17:08)
According to

  http://code.google.com/p/libarchive/issues/detail?id=131#c1 [^]

a "gnutar" output format is available in newer versions of libarchive that does not store xattr attributes.

I committed one fix for building against an outside libarchive in a non-standard location:

  http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=9c78ef13 [^]

Then I built libarchive from svn trunk rev 3140.

I built CMake against that version of libarchive plus the following patch:

diff --git a/Source/cmArchiveWrite.cxx b/Source/cmArchiveWrite.cxx
index d9e4742..a39bbf7 100644
--- a/Source/cmArchiveWrite.cxx
+++ b/Source/cmArchiveWrite.cxx
@@ -123,9 +123,9 @@ cmArchiveWrite::cmArchiveWrite(std::ostream& os, Compress c, Type t):
         }
       break;
     case TypeTAR:
-      if(archive_write_set_format_pax_restricted(this->Archive) != ARCHIVE_OK)
+      if(archive_write_set_format_gnutar(this->Archive) != ARCHIVE_OK)
         {
-        this->Error = "archive_write_set_format_pax_restricted: ";
+        this->Error = "archive_write_set_format_gnutar: ";
         this->Error += archive_error_string(this->Archive);
         return;
         }


Now I can create a tarball that handles long names. I do not know if this tar format is compatible with non-gnu tar versions any less than pax_restricted is.

(0026006)
simonh   
2011-04-01 08:45   
I can confirm that this arose on an SELinux-enabled box.

Isn't gnutar likely to be the most widely supported? Could archive_write_set_format_ be different on the various platforms for which CPack is compiled?
(0026008)
Brad King   
2011-04-01 09:34   
Re 0011958:0026006: Great. That explains why I couldn't reproduce this.

Yes, I think gnutar is the best choice. It is not available in the version of libarchive currently distributed with CMake. We'll have to look at updating our snapshot. However, the gnutar format is not available even in the most recent release of libarchive (coincidentally also version 2.8.4).

We could consider making the format a *runtime* switch since support for multiple formats can be compiled in. Then those that want to avoid xattr headers and do not have long path names can use 'ustar' without recompiling CMake.
(0026031)
simonh   
2011-04-04 04:25   
A runtime switch would make a lot of sense.

Whilst you have some control over the source/build tree and its path names, you often can't affect the unpacking software (i.e. don't know whether it's GNU tar, BSD tar, etc.). Therefore, if trying to maximise the portability of the archive, the CPack user should be able to choose something like ustar; those with longer path names may have to sacrifice some portability for the additional long filename support.
(0026134)
Brad King   
2011-04-07 09:02   
There is a much simpler solution to this issue, suggested by Tim from upstream librachive. We can stick with pax_restricted but strip xattr and acl fields from each entry after reading it from disk but before adding it to the archive:

  http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=e8558efa [^]
(0027770)
David Cole   
2011-11-11 10:45   
Closing resolved issues that have not been updated in more than 4 months.