[Insight-developers] [ITK Community] More info on download step when failed

[Matt McCormick]

Hi Brad L.,

Here is a CMake patch that will retry the download if verification fails:

  http://cmake.org/gitweb?p=stage/cmake.git;a=shortlog;h=refs/heads/robust-ep-download-verify

Please give it a try.

Thanks,
Matt

On Wed, Jan 15, 2014 at 9:07 AM, Bradley Lowekamp
<blowekamp at mail.nih.gov> wrote:
> Cory,
>
> I believe that the night build scripts don't clear out the MD5 test data cache directories which also get downloaded from midas.
>
> Many of the failures occurred on systems with just 2-4 core. It's also not practical to change my build script to be serial. It'd be a waste of resources and things would just not get done.
>
> Additionally the download error was also occurring from source forge too:
> http://open.cdash.org/viewBuildError.php?buildid=3176009
>
> Brad
>
> On Jan 15, 2014, at 9:00 AM, Cory Quammen <cory.quammen at kitware.com> wrote:
>
>> Hi Brad,
>>
>> I've recently seen a similar problem with downloading files during a
>> parallel superbuild in another project.
>>
>> My wild guess was that my parallel build issued too many requests on
>> the server during a short amount of time, perhaps causing some error
>> in the server or perhaps triggering a denial of service attack
>> detection algorithm that shut down some of the downloads. Doing a
>> serial build seemed to solve the problem.
>>
>> Hope that helps,
>> Cory
>>
>> On Wed, Jan 15, 2014 at 8:39 AM, Bradley Lowekamp
>> <blowekamp at mail.nih.gov> wrote:
>>> Hello,
>>>
>>> I have gotten a large number of download failures on my SimpleITK and ITK
>>> builds from midas3.kitware.com in the base 48 hours. It appears to be just
>>> me.
>>>
>>> http://open.cdash.org/index.php?project=SimpleITK&date=2014-01-14
>>>
>>> What appears in the build log is as follows:
>>>
>>> CMake Error at PCRE-stamp/download-PCRE.cmake:9 (file):
>>>
>>>  file DOWNLOAD HASH mismatch
>>>
>>>    for file:
>>> [/.../SimpleITK-build/PCRE-prefix/src/method=midas.bitstream.download&checksum=fa69e4c5d8971544acd71d1f10d59193&name=pcre-8.12.tar.gz]
>>>      expected hash: [fa69e4c5d8971544acd71d1f10d59193]
>>>        actual hash: [d41d8cd98f00b204e9800998ecf8427e]
>>>
>>> where the actual hash it for that of an empty file. All the build logs I
>>> have check have had this empty file hash. Last night while the errors were
>>> occurring, I log onto a fail build and was able to perform to build the
>>> target with the failed download after deleting the empty file.
>>>
>>> I don't have any information to go on... Was it a time out? Was there an
>>> error? I need some more information to go on! Any know if there is some
>>> place to look? or some option I can turn on someplace?
>>>
>>> Thanks,
>>> Brad
>>>
>>> _______________________________________________
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>>
>>> Kitware offers ITK Training Courses, for more information visit:
>>> http://kitware.com/products/protraining.php
>>>
>>> Please keep messages on-topic and check the ITK FAQ at:
>>> http://www.itk.org/Wiki/ITK_FAQ
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://www.itk.org/mailman/listinfo/insight-developers
>>>
>>> _______________________________________________
>>> Community mailing list
>>> Community at itk.org
>>> http://public.kitware.com/cgi-bin/mailman/listinfo/community
>>>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.php
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.itk.org/mailman/listinfo/insight-developers
> _______________________________________________
> Community mailing list
> Community at itk.org
> http://public.kitware.com/cgi-bin/mailman/listinfo/community