[Insight-developers] Testing/Data submodule

[Brad King]

Hi Folks,

A sequence of changes recently took place involving the Testing/Data
submodule.  The situation requires more education and documentation
to address.  The short answer is that everyone should run

 $ git submodule update

after "git pull" and before "git add" unless you know what you
are doing and really want to commit a change to the submodule.
I added this command to the Wiki:

  http://www.itk.org/Wiki/ITK/Git#Updating

Now on to the full story.

In the ITK.git repository, Testing/Data is not really a directory.
It is a *submodule*, meaning that its content does not actually
appear in ITK.git, but in the ITKData.git repository.  In ITK.git
Git stores in the Testing directory an entry called "Data" that
refers to a *commit* from the ITKData repository.  Indeed, one
can see this using a low-level "git ls-tree" command:

 $ git ls-tree -r 9ec7c03d -- Testing/Data
 160000 commit bc9550f3215104818f0464fd6ede7c8ea3462aeb  Testing/Data
               ^^^^^^^^
We can view this commit from ITKData.git:

  http://itk.org/gitweb?p=ITKData.git;a=commitdiff;h=bc9550f3

This approach allows us to keep the bulky data out of the main
repository and version it separately.  Every version of ITK
throughout history refers to the exact version of ITKData that
it needs using this submodule link.

Here is what happened recently.  One can see all the changes
using the command

 $ git log -p eb1ee9cd..9ec7c03d -- Testing/Data

(1) Brad L. correctly updated the Testing/Data submodule reference
in commit e529adfe and merged it to master using the topic stage:

  http://itk.org/gitweb?p=ITK.git;a=commitdiff;h=e529adfe#patch2
  http://itk.org/gitweb?p=ITK.git;a=commitdiff;h=73f66bdb

(2) Hans accidentally reverted the submodule reference back to
the previous value in commit 6abbc796 directly on master:

  http://itk.org/gitweb?p=ITK.git;a=commitdiff;h=6abbc796#patch42

This was not his fault; it was due to an incomplete part of Git's
submodule interface, and the fact I haven't mentioned this before.

(3) I restored Brad L's reference and merged it:

  http://itk.org/gitweb?p=ITK.git;a=commitdiff;h=cb31b152
  http://itk.org/gitweb?p=ITK.git;a=commitdiff;h=9ec7c03d

Git shows patches to Testing/Data in this form:

  diff --git a/Testing/Data b/Testing/Data
  index 60c53c9..bc9550f 160000
  --- a/Testing/Data
  +++ b/Testing/Data
  @@ -1 +1 @@
  -Subproject commit 60c53c94d4fb7ac1ca164939b0378b981ec98de2
  +Subproject commit bc9550f3215104818f0464fd6ede7c8ea3462aeb

One can think of the Testing/Data entry in ITK's tree kind of
like a *file* that tells Git which version of ITKData.git we want.
If we want to change the version of ITKData.git to which we refer
from ITK (perhaps because we've created a new commit in ITKData
with some updated images), we commit a change to ITK that updates
the Testing/Data "file" to refer to the new version.

Git's interface for updating the submodule reference does not
involve actually editing a file.  The Testing/Data "file" on
disk is actually a clone of the whole ITKData.git repository.