[ITK-dev] Refresh cmake ITKData

Fri Jun 26 00:22:06 EDT 2015

It's not working...
I don't know where the ${algo} hash 663d525d9d94726bcea707a0a0b2cd73 is coming from.
I
 have also noticed that there is no file generated in the 
ExternalData/Object/MD5 folder, not a .tmp${random}, nothing. Even 
though it says it has downloaded fine.
I have tried a completely new build with the same source, I should try with a completely fresh source, just in case.
The proxy issues are solved.
I have also tried with -DExternalData_NO_SYMLINKS=ON, with the same result.

Date: Thu, 25 Jun 2015 12:41:38 -0400
Subject: Re: [ITK-dev] Refresh cmake ITKData
From: matt.mccormick at kitware.com
To: pablo.hernandez.cerdan at outlook.com
CC: brad at lowekamp.net; insight-developers at itk.org

Hi Pablo,
CMake will look in the directories listed in the "ExternalData_OBJECT_STORES" CMake cache variable (note the required underscore). It also looks in the ".ExternalData" directory of the source tree [1].  
To clear all the cache data, "make clean" will not do it -- instead:
1) delete the "MD5" directory in these locations2) then delete the "ExternalData" directory in the build tree3) "make ITKData" to regenerate the data
Hope this helps,Matt

[1] http://itk.org/gitweb?p=ITK.git;a=blob;f=CMake/ITKExternalData.cmake;h=0a55a5bf1274be63cffdfddbf6acf036f94fdc46;hb=HEAD#l21
On Thu, Jun 25, 2015 at 2:52 AM, Pablo Hernández <pablo.hernandez.cerdan at outlook.com> wrote:
I set the ExternalData OBJECT_STORES to a local folder outside the build. But nothing there, nothing in the build ExternalData folder...

On 25/06/2015, at 17:02, Pablo Hernández <pablo.hernandez.cerdan at outlook.com> wrote:

The fetching/download part seems all right though, where is cmake storing those files? Maybe I should clean that temporary folder, to get rid off the wrong hash...
 I have no much idea about the ExternalData process

From: pablo.hernandez.cerdan at outlook.com
To: brad at lowekamp.net; matt.mccormick at kitware.com
Date: Thu, 25 Jun 2015 06:59:19 +0200
CC: insight-developers at itk.org
Subject: Re: [ITK-dev] Refresh cmake ITKData

Wget is now downloading properly, but the wrong hash is still there...

From: pablo.hernandez.cerdan at outlook.com
To: brad at lowekamp.net; matt.mccormick at kitware.com
Date: Thu, 25 Jun 2015 05:08:02 +0200
CC: insight-developers at itk.org
Subject: Re: [ITK-dev] Refresh cmake ITKData

Ups...
Wget is successful, but of course the downloaded page says:

Your access has been blocked by firewall policy 100967.

...
I will ask the proxy guys...
Sorry for your extra time

From: pablo.hernandez.cerdan at outlook.com
To: brad at lowekamp.net; matt.mccormick at kitware.com
Date: Thu, 25 Jun 2015 05:03:28 +0200
CC: insight-developers at itk.org
Subject: Re: [ITK-dev] Refresh cmake ITKData

I have upgraded to cmake 3.3. I hadn't have the option of OPENSSL, so I turned it on.
But still having the same hash problem with existing build.
I am behind a university proxy.

From: brad at lowekamp.net
Subject: Re: [ITK-dev] Refresh cmake ITKData
Date: Wed, 24 Jun 2015 22:34:46 -0400
CC: pablo.hernandez.cerdan at outlook.com; insight-developers at itk.org
To: matt.mccormick at kitware.com

Maybe it's a firewall or proxy issue. What is the size of the files? Are the contents of gave file an html error code?
On Jun 24, 2015, at 10:18 PM, Matt McCormick <matt.mccormick at kitware.com> wrote:

Was CMake built with OpenSSL support? Check for
  CMAKE_USE_OPENSSL
It is required to make https connections.
On Wed, Jun 24, 2015 at 9:39 PM, Pablo Hernández <pablo.hernandez.cerdan at outlook.com> wrote:

It's pretty new...

[phc at phc build]$ cmake --version

cmake version 3.2.20150216-gf724a

Date: Wed, 24 Jun 2015 21:33:13 -0400
Subject: Re: [ITK-dev] Refresh cmake ITKData
From: matt.mccormick at kitware.com
To: pablo.hernandez.cerdan at outlook.com
CC: insight-developers at itk.org

What version of CMake? A newer version of CMake may at least be more informative on the source of the error.
HTH,Matt
On Wed, Jun 24, 2015 at 9:20 PM, Pablo Hernández <pablo.hernandez.cerdan at outlook.com> wrote:

If I do make -j4, I receive the same errors but with different objects, and the  (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73) is always the same...

Where is this hash coming from?

-- Fetching "http://itkexternaldata.blob.core.windows.net/md5/8aa41348437f595b21e08eb46726002a"
[  6%] Built target gdcmjpeg12
-- [download 100% complete]
-- Fetching "http://itkexternaldata.blob.core.windows.net/md5/5970c467ac9b44e55829ccf0a1296fd1"
[  6%] Generating /home/phc/Software/ITK-dev/build/ExternalData/Modules/Filtering/AntiAlias/test/Baseline/itkAntiAliasBinaryImageFilterTestBaseline.mha
-- [download 100% complete]
-- Fetching "http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5"
-- [download 100% complete]
-- Fetching "http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=5970c467ac9b44e55829ccf0a1296fd1&algorithm=MD5"
-- [download 100% complete]
-- Fetching "http://www.itk.org/files/ExternalData/MD5/8aa41348437f595b21e08eb46726002a"
-- [download 100% complete]
-- Fetching "http://www.itk.org/files/ExternalData/MD5/5970c467ac9b44e55829ccf0a1296fd1"
-- Fetching "http://itkexternaldata.blob.core.windows.net/md5/8063914920e15a3958953fc8e4e4f6ff"
-- [download 100% complete]
-- [download 100% complete]
-- Fetching "http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5"
-- [download 100% complete]
-- Fetching "http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=5970c467ac9b44e55829ccf0a1296fd1&algorithm=MD5"
-- Fetching "http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8063914920e15a3958953fc8e4e4f6ff&algorithm=MD5"
-- [download 100% complete]
-- [download 100% complete]
CMake Error at /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1012 (message):
    Object MD5=8aa41348437f595b21e08eb46726002a not found at:

    http://itkexternaldata.blob.core.windows.net/md5/8aa41348437f595b21e08eb46726002a (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5 (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
http://www.itk.org/files/ExternalData/MD5/8aa41348437f595b21e08eb46726002a (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5 (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
Call Stack (most recent call first):
    /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1034 (_ExternalData_download_object)

     make[2]: *** [ExternalData/Testing/Data/Baseline/BasicFilters/GradientAnisotropicDiffusionImageFilterTest2.png.md5-stamp] Error 1
     make[2]: *** Waiting for unfinished jobs....
     CMake Error at /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1012 (message):
         Object MD5=5970c467ac9b44e55829ccf0a1296fd1 not found at:

         http://itkexternaldata.blob.core.windows.net/md5/5970c467ac9b44e55829ccf0a1296fd1 (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=5970c467ac9b44e55829ccf0a1296fd1&algorithm=MD5 (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
http://www.itk.org/files/ExternalData/MD5/5970c467ac9b44e55829ccf0a1296fd1 (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=5970c467ac9b44e55829ccf0a1296fd1&algorithm=MD5 (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
Call Stack (most recent call first):
    /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1034 (_ExternalData_download_object)

     -- Fetching "http://www.itk.org/files/ExternalData/MD5/8063914920e15a3958953fc8e4e4f6ff"

     make[2]: *** [ExternalData/Testing/Data/Input/cake_easy.png.md5-stamp] Error 1
     -- [download 100% complete]
     -- Fetching "http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8063914920e15a3958953fc8e4e4f6ff&algorithm=MD5"
     -- [download 100% complete]
     CMake Error at /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1012 (message):
         Object MD5=8063914920e15a3958953fc8e4e4f6ff not found at:

         http://itkexternaldata.blob.core.windows.net/md5/8063914920e15a3958953fc8e4e4f6ff (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8063914920e15a3958953fc8e4e4f6ff&algorithm=MD5 (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
http://www.itk.org/files/ExternalData/MD5/8063914920e15a3958953fc8e4e4f6ff (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8063914920e15a3958953fc8e4e4f6ff&algorithm=MD5 (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
Call Stack (most recent call first):
    /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1034 (_ExternalData_download_object)

     [  6%] Built target ITKznz
     make[2]: *** [ExternalData/Modules/Filtering/AntiAlias/test/Baseline/itkAntiAliasBinaryImageFilterTestBaseline.mha.md5-stamp] Error 1
     make[1]: *** [CMakeFiles/ITKData.dir/all] Error 2
     make[1]: *** Waiting for unfinished jobs....

From: pablo.hernandez.cerdan at outlook.com
To: matt.mccormick at kitware.com
Date: Thu, 25 Jun 2015 03:12:26 +0200
CC: insight-developers at itk.org
Subject: Re: [ITK-dev] Refresh cmake ITKData

Yes, it works.

Date: Wed, 24 Jun 2015 21:08:57 -0400
Subject: Re: [ITK-dev] Refresh cmake ITKData
From: matt.mccormick at kitware.com
To: pablo.hernandez.cerdan at outlook.com
CC: insight-developers at itk.org

Does the following work?

  wget http://itkexternaldata.blob.core.windows.net/md5/8aa41348437f595b21e08eb46726002a

It is working from my location.
On Wed, Jun 24, 2015 at 8:59 PM, Pablo Hernández <pablo.hernandez.cerdan at outlook.com> wrote:

After "make" in build directory: There are quite a lot of similar errors.

[ 
 0%] Generating 
/home/phc/Software/ITK-dev/build/ExternalData/Testing/Data/Baseline/BasicFilters/GradientAnisotropicDiffusionImageFilterTest2.png
-- Fetching "http://itkexternaldata.blob.core.windows.net/md5/8aa41348437f595b21e08eb46726002a"
-- [download 100% complete]
-- Fetching "http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5"
-- [download 100% complete]
-- Fetching "http://www.itk.org/files/ExternalData/MD5/8aa41348437f595b21e08eb46726002a"
-- [download 100% complete]
-- Fetching "http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5"
-- [download 100% complete]
CMake Error at /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1012 (message):
  Object MD5=8aa41348437f595b21e08eb46726002a not found at:

http://itkexternaldata.blob.core.windows.net/md5/8aa41348437f595b21e08eb46726002a (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5 (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
http://www.itk.org/files/ExternalData/MD5/8aa41348437f595b21e08eb46726002a (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5 (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
Call Stack (most recent call first):
    /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1034 (_ExternalData_download_object)

 make[2]: *** 
[ExternalData/Testing/Data/Baseline/BasicFilters/GradientAnisotropicDiffusionImageFilterTest2.png.md5-stamp]
 Error 1
     make[1]: *** [CMakeFiles/ITKData.dir/all] Error 2
     make: *** [all] Error 2

From: pablo.hernandez.cerdan at outlook.com
To: matt.mccormick at kitware.com
Subject: RE: [ITK-dev] Refresh cmake ITKData
Date: Thu, 25 Jun 2015 02:58:57 +0200

After "make" in build directory: There are quite a lot of similar errors.

[  0%] Generating /home/phc/Software/ITK-dev/build/ExternalData/Testing/Data/Baseline/BasicFilters/GradientAnisotropicDiffusionImageFilterTest2.png
-- Fetching "http://itkexternaldata.blob.core.windows.net/md5/8aa41348437f595b21e08eb46726002a"
-- [download 100% complete]
-- Fetching "http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5"
-- [download 100% complete]
-- Fetching "http://www.itk.org/files/ExternalData/MD5/8aa41348437f595b21e08eb46726002a"
-- [download 100% complete]
-- Fetching "http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5"
-- [download 100% complete]
CMake Error at /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1012 (message):
  Object MD5=8aa41348437f595b21e08eb46726002a not found at:

http://itkexternaldata.blob.core.windows.net/md5/8aa41348437f595b21e08eb46726002a (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
http://midas3.kitware.com/midas/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5 (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
http://www.itk.org/files/ExternalData/MD5/8aa41348437f595b21e08eb46726002a (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
http://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=8aa41348437f595b21e08eb46726002a&algorithm=MD5 (wrong hash MD5=663d525d9d94726bcea707a0a0b2cd73)
Call Stack (most recent call first):
    /home/phc/Software/ITK-dev/CMake/ExternalData.cmake:1034 (_ExternalData_download_object)

     make[2]: *** [ExternalData/Testing/Data/Baseline/BasicFilters/GradientAnisotropicDiffusionImageFilterTest2.png.md5-stamp] Error 1
     make[1]: *** [CMakeFiles/ITKData.dir/all] Error 2
     make: *** [all] Error 2

Date: Wed, 24 Jun 2015 20:43:50 -0400
Subject: Re: [ITK-dev] Refresh cmake ITKData
From: matt.mccormick at kitware.com
To: pablo.hernandez.cerdan at outlook.com
CC: insight-developers at itk.org

Hi Pablo,

Could you please share the full text of an error?

Thanks,
Matt

On Wed, Jun 24, 2015 at 8:20 PM, Pablo Hernández <pablo.hernandez.cerdan at outlook.com> wrote:

Hi, I am receiving a lot of wrong hash errors when building ITKData. I am guessing that is because I am updating an old build, but not sure.
Is there a way to force cmake to rerun the scripts related with ITKData to refresh all thoses hashes? I mean, the CMakesLists.txt from all the tests for example.

make clean ITKData, rebuilds everything but the hash stuff is handled by cmake, I am keeping seeing errors downloading ExternalData.

Thanks in advance.
Pablo

_______________________________________________

Powered by www.kitware.com

Visit other Kitware open-source projects at

http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:

http://kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:

http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:

http://public.kitware.com/mailman/listinfo/insight-developers

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-developers 		 	   		  

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-developers

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-developers 		 	   		  

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-developers 		 	   		  

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-developers 		 	   		  
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-developers

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/insight-developers/attachments/20150626/3802f96d/attachment-0001.html>