MantisBT - CMake
View Issue Details
0013251CMakeCMakepublic2012-05-27 07:372014-10-06 10:32
Brad King 
CMake 2.8.8 
CMake 3.0CMake 3.0 
0013251: cmake -E tar <file> fails to extract some files correctly
On certain tar.gz files cmake fails to extract them properly. I suspect this has to do with tar.gz files that have unix symlinks and are extracted on windows.

One such file can be obtained from: [^]

When I try: cmake -E tar xf openssl-1.0.1c.tar.gz it fails on some files:

CMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:openssl-1.0.1c/apps/md4.c
CMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:openssl-1.0.1c/include/openssl/aes.h
CMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:openssl-1.0.1c/include/openssl/asn1.h
CMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:openssl-1.0.1c/include/openssl/asn1_mac.h
CMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:openssl-1.0.1c/include/openssl/asn1t.h
CMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:openssl-1.0.1c/include/openssl/bio.h
CMake Error: Problem with archive_write_header(): Can't create '\'

Using other extraction tools, e.g. cygwin or 7zip, this works without any problems.

No tags attached.
related to 0011176closed Brad King "cmake -E tar" encodes long paths such that Fedora GNU tar 1.15.1 reports errors 
related to 0010162closed Eric NOULARD CPack Cannot create binary zip archive containing shared libs 
related to 0012376closed Kitware Robot Add symlink support 
parent of 0011766closed Kitware Robot CPack using native TAR with adjustable command line options. 
has duplicate 0014805closed  Can not extract any famous library package on windows platform, include openssl and libbson 
gz cmake-tar-test.tar.gz (51,092) 2012-05-27 10:48
patch 0001-cmSystemTools-Re-order-extract_tar-logic.patch (2,707) 2013-07-03 09:25
patch 0002-cmake-Avoid-E-tar-failure-to-extract-symlinks-Window.patch (1,263) 2013-07-03 09:25
Issue History
2012-05-27 07:37sigmaNew Issue
2012-05-27 07:52Eric NOULARDRelationship addedrelated to 0011766
2012-05-27 07:52Eric NOULARDRelationship addedrelated to 0011176
2012-05-27 07:52Eric NOULARDRelationship deletedrelated to 0011766
2012-05-27 07:53Eric NOULARDRelationship addedparent of 0011766
2012-05-27 07:56Eric NOULARDNote Added: 0029571
2012-05-27 07:58Eric NOULARDNote Added: 0029572
2012-05-27 10:48sigmaFile Added: cmake-tar-test.tar.gz
2012-05-27 10:48sigmaNote Added: 0029573
2012-05-28 09:49Brad KingNote Added: 0029577
2012-05-28 09:51Brad KingNote Added: 0029578
2012-05-31 18:47sigmaNote Added: 0029592
2012-05-31 20:55sigmaNote Added: 0029593
2012-06-01 02:39Eric NOULARDRelationship addedrelated to 0010162
2012-06-01 02:56Eric NOULARDNote Added: 0029594
2012-06-01 02:57Eric NOULARDNote Edited: 0029594bug_revision_view_page.php?bugnote_id=29594#r662
2012-08-11 21:42David ColeStatusnew => backlog
2012-08-11 21:42David ColeNote Added: 0030431
2012-12-07 09:15Kevin BurgeNote Added: 0031848
2012-12-07 11:11David ColeAssigned To => David Cole
2012-12-07 11:11David ColeStatusbacklog => assigned
2012-12-07 11:12David ColeAssigned ToDavid Cole =>
2012-12-07 11:12David ColeStatusassigned => new
2012-12-07 11:13David ColeTarget Version => CMake 2.8.11
2013-01-29 06:52frank watson songNote Added: 0032172
2013-05-17 09:33Robert MaynardTarget VersionCMake 2.8.11 => CMake 2.8.12
2013-05-17 14:59MitmischerNote Added: 0033092
2013-07-03 06:08Pau Garcia i QuilesNote Added: 0033459
2013-07-03 08:07Brad KingNote Added: 0033460
2013-07-03 08:22Pau Garcia i QuilesNote Added: 0033461
2013-07-03 09:25Brad KingFile Added: 0001-cmSystemTools-Re-order-extract_tar-logic.patch
2013-07-03 09:25Brad KingFile Added: 0002-cmake-Avoid-E-tar-failure-to-extract-symlinks-Window.patch
2013-07-03 09:26Brad KingNote Added: 0033464
2013-09-15 15:58Eric NOULARDRelationship addedrelated to 0012376
2013-11-02 10:52Brad KingNote Added: 0034326
2013-11-02 10:53Brad KingAssigned To => Brad King
2013-11-02 10:53Brad KingStatusnew => resolved
2013-11-02 10:53Brad KingResolutionopen => fixed
2013-11-02 10:53Brad KingFixed in Version => CMake 3.0
2013-11-02 10:53Brad KingTarget VersionCMake 2.8.12 => CMake 3.0
2014-03-14 08:30Brad KingRelationship addedhas duplicate 0014805
2014-10-06 10:32Robert MaynardNote Added: 0036941
2014-10-06 10:32Robert MaynardStatusresolved => closed

2012-05-27 07:56   
I can extract this archive with CMake 2.8.8
on Linux (Debian Squeeze) without trouble...
2012-05-27 07:58   
You are right those files are symlinks.
How do other tools like 7-zip handles those symlink on Windows?
Do they create plain files which are a copy of the symlink target?

Do they handle symlink in some way?
2012-05-27 10:48   
I created a test tar.gz with various file types to see what happens.
Compress it on a linux box with tar (GNU tar) 1.26:

tar -cpvzf cmake-tar-test.tar.gz cmake-tar-test/

The archive contains:

total 12
drwxr-xr-x 2 root root 4096 May 27 16:44 bin
drwxr-xr-x 2 root root 4096 May 27 17:23 links
drwxr-xr-x 2 root root 4096 May 27 16:44 src

total 100
-rwxr--r-- 1 root root 99328 May 27 16:43 main.exe

total 0
lrwxrwxrwx 1 root root 6 May 27 16:46 dir-link -> ../src
lrwxrwxrwx 1 root root 15 May 27 16:45 main.cpp -> ../src/main.cpp
lrwxrwxrwx 1 root root 15 May 27 16:45 main.exe -> ../bin/main.exe
lrwxrwxrwx 1 root root 13 May 27 16:45 main.h -> ../src/main.h

total 8
-rwxr--r-- 1 root root 127 May 27 16:42 main.cpp
-rwxr--r-- 1 root root 19 May 27 16:42 main.h

So there is a mixture of symlinks to various filetypes and directories. main.exe was compiled on win7 and is a valid executable.

I moved the files back to the Win7 machine to see how the different programs (7-zip, cygwin, cmake) behave. cmake log is here:

cmake -E tar xvfz cmake-tar-test.tar.gz

x cmake-tar-test/
x cmake-tar-test/links/
x cmake-tar-test/links/main.cppCMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:cmake-tar-test/links/main.cpp

x cmake-tar-test/links/main.exeCMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:cmake-tar-test/links/main.exe

x cmake-tar-test/links/main.hCMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:cmake-tar-test/links/main.h

x cmake-tar-test/links/dir-linkCMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:cmake-tar-test/links/dir-link

x cmake-tar-test/src/
x cmake-tar-test/src/main.cpp
x cmake-tar-test/src/main.h
x cmake-tar-test/bin/
x cmake-tar-test/bin/main.exe

Here is what each program does:

  original | 7zip
links/main.cpp | just contains the text ../src/main.cpp (NOT the contents of)
links/main.exe | just contains the text ../bin/main.exe (NOT the contents of)
links/main.h | just contains the text ../src/main.h (NOT the contents of)
links/dir-link | just contains the text ../src

  original | cygwin
links/main.cpp | binary - contains "!<symlink>ÿþ....." (most likely a posix link?)
links/main.exe | binary - contains "!<symlink>ÿþ....."
links/main.h | binary - contains "!<symlink>ÿþ....."
links/dir-link | binary - contains "!<symlink>ÿþ....."

  original | cmake
links/main.cpp | link missing
links/main.exe | link missing
links/main.h | link missing
links/dir-link | link missing

I think there is no simple answer, since the output of all is typically not useful. I didn't look closely, but the openssl package does compile and test fine both if extracted with cygwin and 7zip, but it doesn't work with cmake, because it doesn't extract the links at all.
Brad King   
2012-05-28 09:49   
Possibly related upstream libarchive issue: [^]
Brad King   
2012-05-28 09:51   
A quick experiment tells me that GNU tar on Windows (MinGW/MSYS) extracts a copy of the content from the linked file, at least if the symlink points to another entry in the tarball.

In my understanding the "symlink" generated by GNU tar under Cygwin is just a file that Cygwin's runtime library recognizes and treats as a symlink for the purposes of its POSIX emulation.
2012-05-31 18:47   
Actually, just copying the contents of the pointed file would most likely enable source packages to function properly most of the times afterwards, at the expense of space. It is better than failing, like it does now.

Right now, I use ExternalProject to download and build openssl on windows as a dependency for another project, this fails of course, as EP uses the internal tar by default. I haven't done it yet, but I remember there is a way to specify the command to use to extract it, so one can skip the internal tar. I need to look into it, but it would be greatly appreciated if you could make it copy the files.
2012-05-31 20:55   
I spend some time debugging cmake and it seems this is purely because of the failure of libarchive to handle symlinks.
2012-06-01 02:56   
(edited on: 2012-06-01 02:57)

Have a look at bug 0010162 you'll find that libarchive *is* handling symlinks
but only on supported platforms and/or archive format.
See the libarchive isssue reference pointed by Brad as well.

In fact we do some tuning/verification when creating archive with libarchive
see cmArchiveWrite.[h|cxx]
but we do minimal error checking when extracting
see cmSystemTools.cxx

A) We could (theoretically) check, when extracting, whether if the entry is a symlink and we are on Windows platform, in order to create a plain file
out of the symlink found in libarchive.

This could be seen as workaround of the inability of libarchive to do it
but it could be done.

B) Another option would be to error out when symlink is found in archive when extracting on Windows.

C) Or put some work in order to make this fixed upstream.

I personnally think B) is not satisfactory, so that we should do A) and
possibly suppress it when C) is ready.

My 2 cents, I won't have time to work on this anyway.

David Cole   
2012-08-11 21:42   
Sending old, never assigned issues to the backlog.

(The age of the bug, plus the fact that it's never been assigned to anyone means that nobody is actively working on it...)

If an issue you care about is sent to the backlog when you feel it should have been addressed in a different manner, please bring it up on the CMake mailing list for discussion. Sign up for the mailing list here, if you're not already on it: [^]

It's easy to re-activate a bug here if you can find a CMake developer who has the bandwidth to take it on, and ferry a fix through to our 'next' branch for dashboard testing.
Kevin Burge   
2012-12-07 09:15   
I am seeing this same problem with the same package (openssl) with I agree that making the extract copy of the file is better than failing. My options are: extract and repackage the gz with these links either removed altogether or manually replaced with the original file. It seems that is even worse than 2.8.8. At least it allowed you to continue building. Now it fails and stops.

Please address this.
frank watson song   
2013-01-29 06:52   
I met the same problem when I use ExternalProject_Add work with openssl on windows platform.
I use the blew script to build openssl automaticly on windows platform:

    URL [^]
    URL_MD5 ae412727c8c15b67880aef7bd2999b2e
    CONFIGURE_COMMAND perl Configure VC-WIN32 no-asm --prefix=${CMAKE_INSTALL_PREFIX} & ms\\do_ms & "${VCVARSALL_BATCH}"
    BUILD_COMMAND "${NMAKE_EXECUTABLE}" -f ms\\ntdll.mak
    INSTALL_COMMAND "${NMAKE_EXECUTABLE}" -f ms\\ntdll.mak install

And I met the logs:
2> Creating directories for 'openssl'
2> Performing download step (verify and extract) for 'openssl'
2> -- verifying file...
2> file='I:/gotest/arkWorkspace/thirdparty/project/openssl/openssl-1.0.1c.tar.gz'
2> -- verifying file... done
2> -- extracting...
2> src='I:/gotest/arkWorkspace/thirdparty/project/openssl/openssl-1.0.1c.tar.gz'
2> dst='I:/gotest/arkWorkspace/thirdparty/MSVC-17.0.50727.1-build/openssl/openssl-prefix/src/openssl'
2> -- extracting... [tar xfz]
2>CUSTOMBUILD : CMake error : Problem with archive_write_header(): Can't create '\'
2>CUSTOMBUILD : CMake error : Current file:openssl-1.0.1c/apps/md4.c
2>CUSTOMBUILD : CMake error : Problem extracting tar: I:/gotest/arkWorkspace/thirdparty/project/openssl/openssl-1.0.1c.tar.gz
2> -- extracting... [error clean up]
2> CMake Error at openssl-stamp/extract-openssl.cmake:33 (message):
2>CUSTOMBUILD : error : extract of
2> 'I:/gotest/arkWorkspace/thirdparty/project/openssl/openssl-1.0.1c.tar.gz'
2> failed

I hope this bug can be fixed, it can appear easyly. I use cmake command line tool to decompression openssl package met the same problem.
2013-05-17 14:59   
I can confirm this bug with the NMake-generator.

       URL [^]

`nmake SFML` fails like this:

Scanning dependencies of target SFML
[ 2%] Creating directories for 'SFML'
[ 4%] Performing download step (download, verify and extract) for 'SFML'
-- downloading...
     src='' [^]
-- [download 0% complete]
-- [download 100% complete]
-- [download 0% complete]
-- [download 2% complete]
-- [download 5% complete]
-- [download 11% complete]
-- [download 17% complete]
-- [download 23% complete]
-- [download 30% complete]
-- [download 39% complete]
-- [download 50% complete]
-- [download 64% complete]
-- [download 79% complete]
-- [download 90% complete]
-- [download 100% complete]
-- downloading... done
-- verifying file...
-- verifying file... warning: did not verify file - no URL_HASH specified?
-- extracting...
-- extracting... [tar xfz]
CMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:SFML-2.0/extlibs/libs-osx/Frameworks/freetype.framewor
CMake Error: Problem extracting tar: C:/Windows/Temp/src/2.0.tar.gz
-- extracting... [error clean up]
CMake Error at SFML-stamp/extract-SFML.cmake:33 (message):
  error: extract of 'C:/Windows/Temp/src/2.0.tar.gz' failed

NMAKE : fatal error U1077: "echo": Rückgabe-Code "0x1"
NMAKE : fatal error U1077: ""C:\Program Files (x86)\Microsoft Visual Studio 11.0
\VC\BIN\nmake.exe"": Rückgabe-Code "0x2"
NMAKE : fatal error U1077: ""C:\Program Files (x86)\Microsoft Visual Studio 11.0
\VC\BIN\nmake.exe"": Rückgabe-Code "0x2"
NMAKE : fatal error U1077: ""C:\Program Files (x86)\Microsoft Visual Studio 11.0
\VC\BIN\nmake.exe"": Rückgabe-Code "0x2"
Pau Garcia i Quiles   
2013-07-03 06:08   
Is this going to be fixed any time soon? It's been over a year with this issue and it's breaking ExternalProject badly :-(
Brad King   
2013-07-03 08:07   
Re 0013251:0033459: AFAIK the status is as follows:

* No one is actively working on this issue.
* The work around is to publish one's own openssl tarball with the symlinks replaced as desired.
* It has been raised in libarchive upstream: [^]
* Upstream libarchive developers raised concerns about defining semantics for both creating and extracting archives with symlinks on Windows
* No one has investigated whether it is possible to handle such archive entries in client code rather than libarchive itself

IMO cross-platform source tarballs should not be released with symbolic links in them because any tool extracting them on Windows can at best guess what semantics the source tree wants (copy, text link, NTFS symlink requiring admin privileges, etc.).
Pau Garcia i Quiles   
2013-07-03 08:22   

My workaround for now is downloading OpenSSL from GitHub: [^]

but this is far from optimal.

Granted, cross-platform source tarballs should not be released with symbolic links. But a solution is needed and that solution is not going to be telling every project out there to not tarball with symlinks.

IMHO 7-zip's is good enough: it works with no privileges.
Brad King   
2013-07-03 09:26   
Try attached patches


on top of 2.8.11.
Brad King   
2013-11-02 10:52   
Re 0013251:0033464: Well I didn't get any feedback from the patches but they work for me so I've applied them:

cmSystemTools: Re-order extract_tar logic;a=commitdiff;h=0d8552c4 [^]

cmake: Avoid '-E tar' failure to extract symlinks on Windows;a=commitdiff;h=4c383b52 [^]
Robert Maynard   
2014-10-06 10:32   
Closing resolved issues that have not been updated in more than 4 months.