[ITK] [ITK-dev] Refresh cmake ITKData

Matt McCormick matt.mccormick at kitware.com
Thu Jun 25 12:41:38 EDT 2015 

Hi Pablo,

CMake will look in the directories listed in the
"ExternalData_OBJECT_STORES" CMake cache variable (note the required
underscore). It also looks in the ".ExternalData" directory of the source
tree [1].

To clear all the cache data, "make clean" will not do it -- instead:

1) delete the "MD5" directory in these locations
2) then delete the "ExternalData" directory in the build tree
3) "make ITKData" to regenerate the data

Hope this helps,
Matt



[1]
http://itk.org/gitweb?p=ITK.git;a=blob;f=CMake/ITKExternalData.cmake;h=0a55a5bf1274be63cffdfddbf6acf036f94fdc46;hb=HEAD#l21

On Thu, Jun 25, 2015 at 2:52 AM, Pablo Hernández <
pablo.hernandez.cerdan at outlook.com> wrote:

> I set the ExternalData OBJECT_STORES to a local folder outside the build.
> But nothing there, nothing in the build ExternalData folder...
>
>
>
> On 25/06/2015, at 17:02, Pablo Hernández <
> pablo.hernandez.cerdan at outlook.com> wrote:
>
> The fetching/download part seems all right though, where is cmake storing
> those files? Maybe I should clean that temporary folder, to get rid off the
> wrong hash...
> I have no much idea about the ExternalData process
>
> ------------------------------
> From: pablo.hernandez.cerdan at outlook.com
> To: brad at lowekamp.net; matt.mccormick at kitware.com
> Date: Thu, 25 Jun 2015 06:59:19 +0200
> CC: insight-developers at itk.org
> Subject: Re: [ITK-dev] Refresh cmake ITKData
>
> Wget is now downloading properly, but the wrong hash is still there...
>
> ------------------------------
> From: pablo.hernandez.cerdan at outlook.com
> To: brad at lowekamp.net; matt.mccormick at kitware.com
> Date: Thu, 25 Jun 2015 05:08:02 +0200
> CC: insight-developers at itk.org
> Subject: Re: [ITK-dev] Refresh cmake ITKData
>
> Ups...
> Wget is successful, but of course the downloaded page says:
>
> Your access has been blocked by firewall policy 100967.
>
> ...
> I will ask the proxy guys...
> Sorry for your extra time
>
>
> ------------------------------
> From: pablo.hernandez.cerdan at outlook.com
> To: brad at lowekamp.net; matt.mccormick at kitware.com
> Date: Thu, 25 Jun 2015 05:03:28 +0200
> CC: insight-developers at itk.org
> Subject: Re: [ITK-dev] Refresh cmake ITKData
>
> I have upgraded to cmake 3.3. I hadn't have the option of OPENSSL, so I
> turned it on.
> But still having the same hash problem with existing build.
> I am behind a university proxy.
>
> ------------------------------
> From: brad at lowekamp.net
> Subject: Re: [ITK-dev] Refresh cmake ITKData
> Date: Wed, 24 Jun 2015 22:34:46 -0400
> CC: pablo.hernandez.cerdan at outlook.com; insight-developers at itk.org
> To: matt.mccormick at kitware.com
>
> Maybe it's a firewall or proxy issue. What is the size of the files? Are
> the contents of gave file an html error code?
>
> On Jun 24, 2015, at 10:18 PM, Matt McCormick <matt.mccormick at kitware.com>
> wrote:
>
> Was CMake built with OpenSSL support? Check for
>
>   CMAKE_USE_OPENSSL
>
> It is required to make https connections.
>
> On Wed, Jun 24, 2015 at 9:39 PM, Pablo Hernández <
> pablo.hernandez.cerdan at outlook.com> wrote:
>
> It's pretty new...
>
> [phc at phc build]$ cmake --version
> cmake version 3.2.20150216-gf724a
>
> ------------------------------
> Date: Wed, 24 Jun 2015 21:33:13 -0400
>
> Subject: Re: [ITK-dev] Refresh cmake ITKData
> From: matt.mccormick at kitware.com
> To: pablo.hernandez.cerdan at outlook.com
> CC: insight-developers at itk.org
>
> What version of CMake? A newer version of CMake may at least be more
> informative on the source of the error.
>
> HTH,
> Matt
>
> On Wed, Jun 24, 2015 at 9:20 PM, Pablo Hernández <
> pablo.hernandez.cerdan at outlook.com> wrote:
>
> If I do make -j4, I receive the same errors but with different objects,
> and the (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73) is always the
> same...
>
> Where is this hash coming from?
>
> -- Fetching "
> http://itkexternaldata.blob.core.windows.net/md5/8aa41348437f595b21e08eb46726002a
> "
> [  6%] Built target gdcmjpeg12
> -- [download 100% complete]
> -- Fetching "
> http://itkexternaldata.blob.core.windows.net/md5/5970c467ac9b44e55829ccf0a1296fd1
> "
> [  6%] Generating
> /home/phc/Software/ITK-dev/build/ExternalData/Modules/Filtering/AntiAlias/test/Baseline/itkAntiAliasBinaryImageFilterTestBaseline.mha
> -- [download 100% complete]
> -- Fetching "
> http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5
> "
> -- [download 100% complete]
> -- Fetching "
> http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=5970c467ac9b44e55829ccf0a1296fd1&algorithm=MD5
> "
> -- [download 100% complete]
> -- Fetching "
> http://www.itk.org/files/ExternalData/MD5/8aa41348437f595b21e08eb46726002a
> "
> -- [download 100% complete]
> -- Fetching "
> http://www.itk.org/files/ExternalData/MD5/5970c467ac9b44e55829ccf0a1296fd1
> "
> -- Fetching "
> http://itkexternaldata.blob.core.windows.net/md5/8063914920e15a3958953fc8e4e4f6ff
> "
> -- [download 100% complete]
> -- [download 100% complete]
> -- Fetching "
> http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5
> "
> -- [download 100% complete]
> -- Fetching "
> http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=5970c467ac9b44e55829ccf0a1296fd1&algorithm=MD5
> "
> -- Fetching "
> http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8063914920e15a3958953fc8e4e4f6ff&algorithm=MD5
> "
> -- [download 100% complete]
> -- [download 100% complete]
> CMake Error at /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1012
> (message):
>     Object MD5=8aa41348437f595b21e08eb46726002a not found at:
>
>
> http://itkexternaldata.blob.core.windows.net/md5/8aa41348437f595b21e08eb46726002a
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
>
> http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
> http://www.itk.org/files/ExternalData/MD5/8aa41348437f595b21e08eb46726002a
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
>
> http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
> Call Stack (most recent call first):
>     /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1034
> (_ExternalData_download_object)
>
>
>      make[2]: ***
> [ExternalData/Testing/Data/Baseline/BasicFilters/GradientAnisotropicDiffusionImageFilterTest2.png.md5-stamp]
> Error 1
>      make[2]: *** Waiting for unfinished jobs....
>      CMake Error at
> /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1012 (message):
>          Object MD5=5970c467ac9b44e55829ccf0a1296fd1 not found at:
>
>
> http://itkexternaldata.blob.core.windows.net/md5/5970c467ac9b44e55829ccf0a1296fd1
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
>
> http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=5970c467ac9b44e55829ccf0a1296fd1&algorithm=MD5
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
> http://www.itk.org/files/ExternalData/MD5/5970c467ac9b44e55829ccf0a1296fd1
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
>
> http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=5970c467ac9b44e55829ccf0a1296fd1&algorithm=MD5
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
> Call Stack (most recent call first):
>     /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1034
> (_ExternalData_download_object)
>
>      -- Fetching "
> http://www.itk.org/files/ExternalData/MD5/8063914920e15a3958953fc8e4e4f6ff
> "
>
>      make[2]: ***
> [ExternalData/Testing/Data/Input/cake_easy.png.md5-stamp] Error 1
>      -- [download 100% complete]
>      -- Fetching "
> http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8063914920e15a3958953fc8e4e4f6ff&algorithm=MD5
> "
>      -- [download 100% complete]
>      CMake Error at
> /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1012 (message):
>          Object MD5=8063914920e15a3958953fc8e4e4f6ff not found at:
>
>
> http://itkexternaldata.blob.core.windows.net/md5/8063914920e15a3958953fc8e4e4f6ff
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
>
> http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8063914920e15a3958953fc8e4e4f6ff&algorithm=MD5
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
> http://www.itk.org/files/ExternalData/MD5/8063914920e15a3958953fc8e4e4f6ff
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
>
> http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8063914920e15a3958953fc8e4e4f6ff&algorithm=MD5
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
> Call Stack (most recent call first):
>     /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1034
> (_ExternalData_download_object)
>
>
>      [  6%] Built target ITKznz
>      make[2]: ***
> [ExternalData/Modules/Filtering/AntiAlias/test/Baseline/itkAntiAliasBinaryImageFilterTestBaseline.mha.md5-stamp]
> Error 1
>      make[1]: *** [CMakeFiles/ITKData.dir/all] Error 2
>      make[1]: *** Waiting for unfinished jobs....
>
>
>
> ------------------------------
> From: pablo.hernandez.cerdan at outlook.com
> To: matt.mccormick at kitware.com
> Date: Thu, 25 Jun 2015 03:12:26 +0200
> CC: insight-developers at itk.org
>
> Subject: Re: [ITK-dev] Refresh cmake ITKData
>
> Yes, it works.
>
> ------------------------------
> Date: Wed, 24 Jun 2015 21:08:57 -0400
> Subject: Re: [ITK-dev] Refresh cmake ITKData
> From: matt.mccormick at kitware.com
> To: pablo.hernandez.cerdan at outlook.com
> CC: insight-developers at itk.org
>
> Does the following work?
>
>   wget
> http://itkexternaldata.blob.core.windows.net/md5/8aa41348437f595b21e08eb46726002a
>
> It is working from my location.
>
> On Wed, Jun 24, 2015 at 8:59 PM, Pablo Hernández <
> pablo.hernandez.cerdan at outlook.com> wrote:
>
> After "make" in build directory: There are quite a lot of similar errors.
>
> [  0%] Generating
> /home/phc/Software/ITK-dev/build/ExternalData/Testing/Data/Baseline/BasicFilters/GradientAnisotropicDiffusionImageFilterTest2.png
> -- Fetching "
> http://itkexternaldata.blob.core.windows.net/md5/8aa41348437f595b21e08eb46726002a
> "
> -- [download 100% complete]
> -- Fetching "
> http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5
> "
> -- [download 100% complete]
> -- Fetching "
> http://www.itk.org/files/ExternalData/MD5/8aa41348437f595b21e08eb46726002a
> "
> -- [download 100% complete]
> -- Fetching "
> http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5
> "
> -- [download 100% complete]
> CMake Error at /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1012
> (message):
>   Object MD5=8aa41348437f595b21e08eb46726002a not found at:
>
>
> http://itkexternaldata.blob.core.windows.net/md5/8aa41348437f595b21e08eb46726002a
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
>
> http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
> http://www.itk.org/files/ExternalData/MD5/8aa41348437f595b21e08eb46726002a
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
>
> http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
> Call Stack (most recent call first):
>     /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1034
> (_ExternalData_download_object)
>
>
>      make[2]: ***
> [ExternalData/Testing/Data/Baseline/BasicFilters/GradientAnisotropicDiffusionImageFilterTest2.png.md5-stamp]
> Error 1
>      make[1]: *** [CMakeFiles/ITKData.dir/all] Error 2
>      make: *** [all] Error 2
>
> ------------------------------
> From: pablo.hernandez.cerdan at outlook.com
> To: matt.mccormick at kitware.com
> Subject: RE: [ITK-dev] Refresh cmake ITKData
> Date: Thu, 25 Jun 2015 02:58:57 +0200
>
>
> After "make" in build directory: There are quite a lot of similar errors.
>
> [  0%] Generating
> /home/phc/Software/ITK-dev/build/ExternalData/Testing/Data/Baseline/BasicFilters/GradientAnisotropicDiffusionImageFilterTest2.png
> -- Fetching "
> http://itkexternaldata.blob.core.windows.net/md5/8aa41348437f595b21e08eb46726002a
> "
> -- [download 100% complete]
> -- Fetching "
> http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5
> "
> -- [download 100% complete]
> -- Fetching "
> http://www.itk.org/files/ExternalData/MD5/8aa41348437f595b21e08eb46726002a
> "
> -- [download 100% complete]
> -- Fetching "
> http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5
> "
> -- [download 100% complete]
> CMake Error at /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1012
> (message):
>   Object MD5=8aa41348437f595b21e08eb46726002a not found at:
>
>
> http://itkexternaldata.blob.core.windows.net/md5/8aa41348437f595b21e08eb46726002a
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
>
> http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
> http://www.itk.org/files/ExternalData/MD5/8aa41348437f595b21e08eb46726002a
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
>
> http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5
> (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
> Call Stack (most recent call first):
>     /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1034
> (_ExternalData_download_object)
>
>
>      make[2]: ***
> [ExternalData/Testing/Data/Baseline/BasicFilters/GradientAnisotropicDiffusionImageFilterTest2.png.md5-stamp]
> Error 1
>      make[1]: *** [CMakeFiles/ITKData.dir/all] Error 2
>      make: *** [all] Error 2
>
> ------------------------------
> Date: Wed, 24 Jun 2015 20:43:50 -0400
> Subject: Re: [ITK-dev] Refresh cmake ITKData
> From: matt.mccormick at kitware.com
> To: pablo.hernandez.cerdan at outlook.com
> CC: insight-developers at itk.org
>
> Hi Pablo,
>
> Could you please share the full text of an error?
>
> Thanks,
> Matt
>
> On Wed, Jun 24, 2015 at 8:20 PM, Pablo Hernández <
> pablo.hernandez.cerdan at outlook.com> wrote:
>
> Hi, I am receiving a lot of wrong hash errors when building ITKData. I am
> guessing that is because I am updating an old build, but not sure.
> Is there a way to force cmake to rerun the scripts related with ITKData to
> refresh all thoses hashes? I mean, the CMakesLists.txt from all the tests
> for example.
>
> make clean ITKData, rebuilds everything but the hash stuff is handled by
> cmake, I am keeping seeing errors downloading ExternalData.
>
> Thanks in advance.
> Pablo
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.php
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/insight-developers
>
>
>
>
> _______________________________________________ Powered by www.kitware.com
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html Kitware offers ITK
> Training Courses, for more information visit:
> http://kitware.com/products/protraining.php Please keep messages on-topic
> and check the ITK FAQ at: http://www.itk.org/Wiki/ITK_FAQ Follow this
> link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/insight-developers
>
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.php
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/insight-developers
>
>
> _______________________________________________ Powered by www.kitware.com
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html Kitware offers ITK
> Training Courses, for more information visit:
> http://kitware.com/products/protraining.php Please keep messages on-topic
> and check the ITK FAQ at: http://www.itk.org/Wiki/ITK_FAQ Follow this
> link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/insight-developers
>
> _______________________________________________ Powered by www.kitware.com
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html Kitware offers ITK
> Training Courses, for more information visit:
> http://kitware.com/products/protraining.php Please keep messages on-topic
> and check the ITK FAQ at: http://www.itk.org/Wiki/ITK_FAQ Follow this
> link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/insight-developers
>
> _______________________________________________ Powered by www.kitware.com
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html Kitware offers ITK
> Training Courses, for more information visit:
> http://kitware.com/products/protraining.php Please keep messages on-topic
> and check the ITK FAQ at: http://www.itk.org/Wiki/ITK_FAQ Follow this
> link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/insight-developers
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.php
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/insight-developers
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/community/attachments/20150625/c8c0c37f/attachment-0001.html>
-------------- next part --------------
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-developers

More information about the Community mailing list