[vtk-developers] VTK ExternalData default object store (was: vtk_common dashboard script)

Tue May 21 13:32:39 EDT 2013

-----Original Message-----
From: Sebastien Jourdain <sebastien.jourdain at kitware.com>
To: Robert Maynard <robert.maynard at kitware.com>
Cc: Bill Lorensen <bill.lorensen at gmail.com>; VTK Developers 
<vtk-developers at vtk.org>; Brad King <brad.king at kitware.com>; David Cole 
<dlrdave at aol.com>
Sent: Tue, May 21, 2013 1:22 pm
Subject: Re: [vtk-developers] VTK ExternalData default object store 
(was: vtk_common dashboard script)

Sorry to get into that already long thread, but I see a greater 
potential to external data which go beyond TESTING.
In fact, I was planning to use it to provide external documentation 
component at install/build time.
Otherwise, I like VTK_DATA_STORAGE with a default to sibling if found 
or simply $HOME/.externalData/VTK/.

Where I partially care, is why we don't have a "CMake" external data 
repository like Maven do ($HOME/.m2/repository*).
But I guess, it is just to show my support to Dave Cole point of view.

Anyway, don't waist time to try to convince me, I will go with what 
ever gets decided after all...
I just wanted to give you my point of view.

Seb

On Tue, May 21, 2013 at 11:25 AM, Robert Maynard 
<robert.maynard at kitware.com> wrote:

If we do move to some CMake variable, I believe that the name
ExternalData_OBJECT_STORES is a bad choice as it doesn't express what
project or purpose it has unless you are already understand CMake
packages and how VTK stores testing data.

I purpose a name similar to: VTK_TESTING_DATA_STORAGE which is clear
to all people on what the intended purpose is.

On Tue, May 21, 2013 at 11:18 AM, Bill Lorensen 
<bill.lorensen at gmail.com> wrote:
> Since I use both ITK and VTK, I use the .bashrc approach that keeps 
all of
> my external data in one place.
>
>
>
>
> On Tue, May 21, 2013 at 10:59 AM, David Cole <dlrdave at aol.com> wrote:
>>
>> -----Original Message-----
>> From: Brad King <brad.king at kitware.com>
>> To: David Cole <dlrdave at aol.com>
>> Cc: vtk-developers <vtk-developers at vtk.org>
>> Sent: Tue, May 21, 2013 10:48 am
>> Subject: Re: [vtk-developers] VTK ExternalData default object store 
(was:
>> vtk_common dashboard script)
>>
>>
>> On 05/21/2013 09:35 AM, Marcus D. Hanwell wrote:
>>>
>>> The expectation is also that data is persistent if we don't change
>>> the default perhaps we can at least add a warning at configure time 
to
>>> advise that data will be downloaded multiple times (the default is
>>> each build tree?) I would love to see a default parallel to the 
source
>>> tree, like VTKExternalData or something, but it seems like you are
>>> strongly opposed to that.
>>
>>
>> My original statement was about a general expectation of projects
>> staying isolated in their source/build tree until "make install" 
time.
>> CMake used to have tests that touched the $HOME directory and we got
>> complaints so we changed the tests to run with a custom HOME env var
>> pointing at the build tree.
>>
>> On 05/21/2013 10:34 AM, David Cole wrote:
>>>
>>> Additionally, in favor of a parallel sibling directory to the source
>>> tree... I think of this as a replacement for a VTKData checkout,
>>
>> which
>>>
>>> has traditionally always been recommended to be a parallel sibling
>>> directory.
>>>
>>> Wasn't VTK and VTKData in the same parent directory the recommended
>>> structure? And didn't the common script do a checkout of VTKData 
such
>>> that they ended up that way?
>>
>>
>> Yes, but both of those required the user to do something local 
besides
>> checkout and build VTK: either checkout VTKData manually or write a
>> dashboard script referencing vtk_common.cmake.
>>
>>> Seems completely reasonable to me to say: the default is *here*, and
>>> it's a cache setting... if you want it somewhere else, change the
>>
>> cache
>>>
>>> setting.
>>
>>
>> That requires reading the documentation.  A user or automatic script
>> that is not familiar with VTK in particular may not do that.
>>
>>> What I don't want is *this*:
>>> to have multiple build trees of VTK going on my laptop, and to
>>> *accidentally* get multiple copies of VTKData and VTKLargeData just
>>> because I left something at its default value.
>>
>>
>> Even with the sibling directory you could still end up with multiple
>> copies if you have VTK sources in separate directories.
>>
>> A similar discussion a few years ago in ITK led to this feature:
>>
>> http://itk.org/gitweb?p=ITK.git;a=commitdiff;h=d80e341a
>>
>> Just add
>>
>> export ExternalData_OBJECT_STORES=$HOME/.ExternalData
>>
>> to your .bashrc and all your new ITK and VTK build trees will use
>> that for persistent data.
>>
>> -Brad
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.vtk.org/mailman/listinfo/vtk-developers
>>
>>
>>
>>
>> OK.
>>
>> But with your solution, *everybody* has to do work to achieve
>> non-duplication of very large data files when they do multiple 
builds of
>> VTK. *Everybody*.
>>
>> If you would just use a default value in $HOME or %USERPROFILE% 
instead,
>> then *nobody* would have to to work to achieve non-duplication.
>>
>> And I thought the whole main point of ExternalData was to download 
each
>> item needed *exactly* *once* and to enable non-duplication of each 
item.
>>
>> So I guess I'm puzzled by the default value.
>>
>> But if that's the way it is, I'll set up my machines so that at 
least I
>> don't have duplicated data files.
>>
>>
>> Thanks,
>>
>> D
>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.vtk.org/mailman/listinfo/vtk-developers
>>
>
>
>
> --
> Unpaid intern in BillsBasement at noware dot com
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://www.vtk.org/mailman/listinfo/vtk-developers
>
>
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://www.vtk.org/mailman/listinfo/vtk-developers

Thanks, Seb, for the support.

Just to be clear: I advocate for a default setting which avoids 
multiple copies of the data directory regardless of the number of 
source and build directories involved.

Also just to be clear: I've said my piece, and now I'll be quiet. I, 
too, will live with whatever the final decision is, and will adapt my 
own environments to avoid duplication, even if it does not happen 
automatically by default.

Thanks,
David