View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0014807CMakeCMakepublic2014-03-14 05:052014-10-06 10:32
ReporterJohn Szakmeister 
Assigned ToBrad King 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionCMake 3.1Fixed in VersionCMake 3.1 
Summary0014807: file(DOWNLOAD) progress goes wild
DescriptionWe're using ExternalProject_Add to help build dependencies, but it can lead to nearly 3,000 lines of progress output as seen here: https://travis-ci.org/neovim/neovim/jobs/20601268#L826 [^]

It'd be nice if CMake provided a way to disable that output. Plus, the output is bogus. The first output was 100%, then 0%, then 739%, and ending at 3069066%.

Finally, this issue would've been easy to work around had the download, verify, and extract steps been exposed separately. Since they're not, it a substantial amount of work, just to avoid seeing 3,000 lines of progress. :-(
TagsNo tags attached.
Attached Files

 Relationships

  Notes
(0035402)
Brad King (manager)
2014-03-14 08:59

 Those messages come from the file(DOWNLOAD) command. See the UpdatePercentage method here:

 http://cmake.org/gitweb?p=cmake.git;a=blob;f=Source/cmFileCommand.cxx;hb=v3.0.0-rc1#l2535 [^]

and the call to it here:

 http://cmake.org/gitweb?p=cmake.git;a=blob;f=Source/cmFileCommand.cxx;hb=v3.0.0-rc1#l2569 [^]

The callbacks and numbers come from curl. Normally it only calls back a few times and gets to 100% so it is not supposed to be so verbose. I would be interested in knowing why it is saying more data have been downloaded than the total. That is likely the underlying problem.
(0035405)
John Szakmeister (reporter)
2014-03-14 09:36

 Perhaps the bundled curl needs to be updated? The release notes for 7.22.0 (http://curl.haxx.se/changes.html#7_22_0 [^]) have an interesting entry:

    * progress: reset flags at transfer start to avoid wrong CURLINFO_CONTENT_LENGTH_DOWNLOAD

It looks like 7.20.0 fixed some issues too:

    * progress meter percentage and transfer time estimates fixes
    * progress callback called repeatedly during slow connects
(0035406)
Brad King (manager)
2014-03-14 09:39

 Re 0014807:0035405: We support building against a system-installed curl. Can you try doing that with a newer curl to see if it resolves the issue? If so then I'll look at updating the builtin curl too.
(0035412)
John Szakmeister (reporter)
2014-03-14 15:20

 Brad, I just checked and the Ubuntu 12.04 CMake is being built against the system curl, which is at 7.22.0 (plus a couple other patches). So, it's not likely either one of the fixes I mention above--though there are other fixes in later releases of curl for progress-related bits.

It's worth noting that the frequency of updates changed in the latest curl, and probably not for the better when using Travis. So having an option to disable the progress updates with ExternalProject_Add is still desirable.
(0035419)
Brad King (manager)
2014-03-17 11:10

 Re 0014807:0035412: It looks like we already round progress to the nearest integer percentage and report only when that changes. That should limit progress to at most 101 lines. However, when progress goes beyond 100% then there is no limit. Here is a fix for that:

 file: Avoid runaway DOWNLOAD/UPLOAD progress reports
 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=413282c0 [^]
(0035420)
Brad King (manager)
2014-03-17 11:31

 I've added a DOWNLOAD_NO_PROGRESS option to ExternalProject_Add:

 ExternalProject: Add option to disable download progress
 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=7d35b550 [^]
(0035421)
John Szakmeister (reporter)
2014-03-17 11:34

 Thank you Brad!
(0035422)
Brad King (manager)
2014-03-17 11:38

 Re 0014807:0035421: The remaining problem is to figure out why Curl provides the wrong numbers in the first place. I tried file(DOWNLOAD) on https://github.com/joyent/libuv/archive/v0.11.21.tar.gz [^] locally and it gets expected progress reports. This is with a system-installed curl 7.34.0. Perhaps it is a bug that has been fixed in Curl since 7.22.0?
(0036947)
Robert Maynard (manager)
2014-10-06 10:32

 Closing resolved issues that have not been updated in more than 4 months.

 Issue History
Date Modified Username Field Change
2014-03-14 05:05 John Szakmeister New Issue
2014-03-14 08:59 Brad King Note Added: 0035402
2014-03-14 08:59 Brad King Summary ExternalProject_Add needs a way to silence progress output (and progress output is bogus) => file(DOWNLOAD) progress goes wild
2014-03-14 09:36 John Szakmeister Note Added: 0035405
2014-03-14 09:39 Brad King Note Added: 0035406
2014-03-14 15:20 John Szakmeister Note Added: 0035412
2014-03-17 11:10 Brad King Note Added: 0035419
2014-03-17 11:31 Brad King Note Added: 0035420
2014-03-17 11:31 Brad King Assigned To => Brad King
2014-03-17 11:31 Brad King Status new => assigned
2014-03-17 11:31 Brad King Target Version => CMake 3.1
2014-03-17 11:34 John Szakmeister Note Added: 0035421
2014-03-17 11:38 Brad King Note Added: 0035422
2014-03-20 09:27 Brad King Status assigned => resolved
2014-03-20 09:27 Brad King Resolution open => fixed
2014-03-20 09:27 Brad King Fixed in Version => CMake 3.1
2014-10-06 10:32 Robert Maynard Note Added: 0036947
2014-10-06 10:32 Robert Maynard Status resolved => closed

Copyright © 2000 - 2018 MantisBT Team