[CMake] How to avoid continuous download while using ExternalProjects

[Luigi Calori]

David Cole wrote:
> If you want to trigger a "rebuild" of any given step (and later steps) 
> for an ExternalProject, simply open up the project folder in Visual 
> Studio, so you can see the list of steps. Right click on the step you 
> want to re-do and tell Visual Studio to "compile" the step.
>
> With custom build steps, "compiling" is executing the custom command 
> for the step. If you re-execute an earlier step in the chain, all the 
> subsequent steps should also rebuild because of the dependencies among 
> the stamp files.
I had already found that.. It is indeed working, the problem is that the
stamp files for config,build and install are set up even if there are
errors in the step, so rebuilding as a whole does not work and I need to
"compile" the individual steps
>
> This is a finer-grained way to get the same effect you got from the 
> "clean" without re-executing the download step.
I would really like to just have it working as other targets
>
> If you want to avoid re-executing the download step, then you have to 
> avoid using "clean" or you have to move the download step to something 
> truly external.
Have you any example of that? I was also trying to separate the two
project in download+patch and configure+build+install .... but did not
had uccess in building a good dependency chain (it has to be file-level)
>
> One idea might be to add a chain of external projects that *only* do 
> download steps, and make each of your configure/build/install external 
> projects depend on *that* download-only external project.
Yes.. but how do I chain on FILES ... not on targets, I need a step that
"produce" something so when i clean from the main project the already
downloaded+patched sources stay there, and require explicit (manual) delete
>
>
> If anybody has other ideas to share downloads between "Release" and 
> "Debug" configurations, or other ways to avoid "unnecessary" downloads 
> better, or other ideas for any ExternalProject improvement, I'm all 
> ears. :-)
could the stamp files for the "source" (download+patch) be put in a
folder not dependent on Release and Debug ?

Anyway, tahnks a lot for providing the feature, any example available to
learn best practice usage really welcomed...

I had also to add a small patch to avoid bad downloads cases:  there
were some url that were downloadable with wget but not with cmake file
download..
that was apparently successful but the downloaded file was zero length...
I added a workaround: check the file legth and try with wget if too
short....
Probably better to allow for different url and wget download option...
if multiple url are specified, repeat download step on failure?

>
>
> HTH,
> David
>
>
> On Wed, Feb 10, 2010 at 10:40 AM, Michael Wild <themiwi at gmail.com 
> <mailto:themiwi at gmail.com>> wrote:
>
>
>     On 10. Feb, 2010, at 15:37 , Luigi Calori wrote:
>
>     >
>     > Hi Michael:
>     >
>     > They are more or less something like:
>     >
>     >
>     > ExternalProject_Add(
>     >   ${PACKAGE}
>     >   SOURCE_DIR ${BASE_SOURCE}/${PACKAGE}/src
>     >   BINARY_DIR ${BASE_BUILD}/${PACKAGE}/build
>     >   INSTALL_DIR ${CMAKE_INSTALL_PREFIX}
>
>     I don't think this is correct! The INSTALL_DIR should be IMHO in
>     the build-tree. Otherwise the files will be installed into the
>     system while building (which, apart from being very bad practice,
>     might fail if the user doesn't have the rights to write in this
>     location).
>
>     >   URL http://www.zlib.net/zlib-1.2.3.tar.gz
>     >   PATCH_COMMAND ${CMAKE_COMMAND} -E copy
>     "${Package_Dir}/Patch/CMakeLists.txt" <SOURCE_DIR>/CMakeLists.txt
>     >   CMAKE_COMMAND ${CMAKE_COMMAND}
>     >   CMAKE_ARGS
>     >       ${Package_std_cmake_args}
>     > )
>     >
>     >
>     > The problem is that I do not know how to easily issue a re-build
>     without a re-download:
>     > The steps are always done, even if there is something broken in
>     the configure or build step, so if I just clean the whole project,
>     the download happen again, even if the (patched)source dir is
>     already there.
>
>     Does it also happen if you just re-build without the clean step?
>     Probably the stamp files get removed by the clean... Usually it
>     should not be necessary to do a "clean" when developing.
>
>     >
>     > I am working in MSVC9, from the master project, a download is
>     re-issued even whn I switch from debug to release (i think because
>     stamp file are config dependent)
>
>     Hmm. Haven't tried using a multi-config IDE so far. Are the
>     stamp-files in a configuration-specific subdirectory? What's the
>     value of BASE_SOURCE?
>
>     >
>     > Thanks in advance.
>     >
>     > Luigi
>     >
>     >
>     > Michael Wild wrote:
>     >> On 10. Feb, 2010, at 14:57 , Luigi Calori wrote:
>     >>
>     >>
>     >>> I' m trying to develop a project for building several lib
>     dependencies using ExternalProjectAdd
>     >>>
>     >>> Is there a way to avoid re-download of upstream tar.gz while
>     developing config flags?
>     >>>
>     >>> ANY examples of ExternalProjectAdd would REALLY welcome
>     >>>
>     >>> Thanks in advance.
>     >>>
>     >>> Luigi
>     >>>
>     >>
>     >> That's strange, this doesn't happen in my projects...
>     >>
>     >> Can you show an example of your ExternalProject_Add calls?
>     >>
>     >> Michael
>     >>
>     >>
>     >
>
>     _______________________________________________
>     Powered by www.kitware.com <http://www.kitware.com>
>
>     Visit other Kitware open-source projects at
>     http://www.kitware.com/opensource/opensource.html
>
>     Please keep messages on-topic and check the CMake FAQ at:
>     http://www.cmake.org/Wiki/CMake_FAQ
>
>     Follow this link to subscribe/unsubscribe:
>     http://www.cmake.org/mailman/listinfo/cmake
>
>