MantisBT - CMake
View Issue Details
0012461CMakeCMakepublic2011-09-17 06:152012-04-02 10:10
Valentin Rusu 
David Cole 
normalminoralways
closedno change required 
IntelLinuxopenSuSE 11.4
CMake 2.8.5 
CMake 2.8.7CMake 2.8.7 
0012461: externalproject_add with GIT : fetch without merge
Using externalproject_add via KDE superbuild project.

First time it correctly clones GIT repositories.

Subsequent builds doing "git fetch". This updates GIT internal structures in the clone, but does not affect local sources, as the "git merge" step is not done.

Possible solutions:
- add "git merge" step,
- replace "git fetch" with "git pull".

No tags attached.
related to 0012445closed David Cole Do not use git.cmd in FindGit module 
Issue History
2011-09-17 06:15Valentin RusuNew Issue
2011-09-19 10:00Brad KingNote Added: 0027447
2011-09-19 10:01Brad KingRelationship addedrelated to 0012445
2011-09-19 10:05Brad KingNote Added: 0027448
2011-09-19 10:05Brad KingAssigned To => David Cole
2011-09-19 10:05Brad KingStatusnew => assigned
2011-10-25 21:51David ColeTarget Version => CMake 2.8.7
2011-11-16 15:12Alex NeundorfNote Added: 0027813
2011-11-17 13:09Brad KingNote Added: 0027814
2011-11-18 10:38David ColeNote Added: 0027819
2011-11-18 13:55Alex NeundorfNote Added: 0027821
2011-11-18 13:55Alex NeundorfNote Edited: 0027821bug_revision_view_page.php?bugnote_id=27821#r455
2011-11-18 14:08Brad KingNote Added: 0027822
2011-11-18 14:20Alex NeundorfNote Added: 0027823
2011-11-18 14:26Brad KingNote Added: 0027824
2011-11-18 14:39David ColeNote Added: 0027825
2011-11-18 14:40David ColeNote Added: 0027826
2011-11-18 15:39Alex NeundorfNote Added: 0027827
2011-11-18 15:39Alex NeundorfStatusassigned => closed
2011-11-18 15:39Alex NeundorfResolutionopen => no change required
2011-11-18 15:41David ColeNote Added: 0027828
2011-11-18 15:41David ColeStatusclosed => resolved
2011-11-18 15:41David ColeFixed in Version => CMake 2.8.7
2012-04-02 10:10David ColeNote Added: 0029046
2012-04-02 10:10David ColeStatusresolved => closed

Notes
(0027447)
Brad King   
2011-09-19 10:00   
Try using git.exe instead of git.cmd.
(0027448)
Brad King   
2011-09-19 10:05   
The update step uses multiple commands:

  git fetch ...
  git checkout ...
  ...

so that the update is not dependent on local tracking branch configuration.

However multiple commands does not work in VS generators because git.cmd the whole command sequence after the first command runs. See 0012445:0027375 for the proposed fix.

This is really happening to you in Linux too? What generator are you using?
(0027813)
Alex Neundorf   
2011-11-16 15:12   
Yes, I still have to manually do "git pull" or git merge afterwards.
git is 1.7.4.4.

It looks like this:

[ 95%] Performing update step (git fetch) for 'kdeexamples'
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 4 (delta 3), reused 0 (delta 0)
Unpacking objects: 100% (4/4), done.
From git://anongit.kde.org/kdeexamples [^]
   bef0736..c210877 master -> origin/master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.

The git checkout afterwards doesn't seem to do a lot, it simply stays at the branch it already is. git pull or merge updates the working tree:

kde_kdeexamples$ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
kde_kdeexamples$ git pull
Updating bef0736..c210877
Fast-forward
 kdedemo/examples.xml | 1 +
 1 files changed, 1 insertions(+), 0 deletions(-)


Is this the intended behaviour ?
I'd like to be able to just get the sources updated to current git master.

Alex
(0027814)
Brad King   
2011-11-17 13:09   
Re 0012461:0027813: I'm not sure whether the current behavior is intended or desirable but you can follow the upstream master by using

  GIT_TAG origin/master

in the ExternalProject_Add call.
(0027819)
David Cole   
2011-11-18 10:38   
Just a note here, before we decide what the best fix is for this issue. I do not regard this as a high priority issue at all, for the following reasons:

The "vast majority use case" of connecting to a git repository with ExternalProject_Add is as follows (and all of these work right now):

Track an upstream branch, use "origin/" explicitly in GIT_TAG:

  GIT_TAG origin/master

Snap to a known tag or commit, never update, leave it static unless GIT_TAG changes:

  GIT_TAG v2.8.6
  UPDATE_COMMAND ""

or

  GIT_TAG 46c70540ede37c5f2ebeeeba4178340abf92fadb
  UPDATE_COMMAND ""
(0027821)
Alex Neundorf   
2011-11-18 13:55   
If I use origin/master, I get:

[ 0%] Performing download step (git clone) for 'extra-cmake-modules'
Cloning into kdesupport_extra-cmake-modules...
remote: Counting objects: 9541, done.
...
Resolving deltas: 100% (6307/6307), done.
Note: checking out 'origin/master'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
...


This doesn't seem to be what we need.

The use case is the following:
with the transition from svn to git the situation for KDE has changed a lot.
Before, I could do svn co trunk/KDE/, and got all KDE modules (15 or so).

Now, with git, it's not only that every KDE developer who wants to access all (or multiple) "modules" would have to git clone those 15 git repositories, but the repositories are also being split into more modular repositories.
E.g. the kdegraphics module has been split into 14 git repositories holding one application each, and 4 git repositories holding one library each.
If I counted correctly, we have 192 git repositories currently.

Add to that that there are dependencies between some of them, this is unmanagable manually.

We have a python script kdesrc-build, which automates this.

I'm trying to write a SuperBuild CMakeLists.txt for KDE, adding one ExternalProject for each git repository.
So I can then select which subprojects I want, enable only those in this superbuild, and update them all at once:

https://projects.kde.org/projects/kde/superbuild [^]

This is basically working, only that I cannot really update them because of the git fetch instead git pull (or the missing git merge).

So, I'd really like to get that working, and I can also work on this, if we agree how it should be done.

(0027822)
Brad King   
2011-11-18 14:08   
Ignore the "detached head" output. That is Git's explanation for newcomers that enter a command unintentionally entering that state. Using "origin/master" will do what you need. The resulting update sequence is

 git fetch # updates refs/remotes/origin/master
 git checkout origin/master # checks out updated version
 git submodule ... # brings contained submodules up to date

The reason we don't use "git pull" is because that cannot cleanly handle upstream branch rewrites.
(0027823)
Alex Neundorf   
2011-11-18 14:20   
Not completely, at least not directly.

When I try to push in this state, I get:

~/src/kde-git/KDESB2/src/kdesupport_extra-cmake-modules$ git push
fatal: You are not currently on a branch.
~/src/kde-git/KDESB2/src/kdesupport_extra-cmake-modules$


When I do then git checkout master, I can push again.
But doing the update via ExternalProject afterwards, takes me back to the detached HEAD.

Can I do something in my git setup to make this behave different ?

Or how about adding some switch or option to ExternalProject so that it does pull or merge or something ?
(0027824)
Brad King   
2011-11-18 14:26   
You can always use a custom UPDATE_COMMAND to work around this and make it "git pull".
(0027825)
David Cole   
2011-11-18 14:39   
ExternalProject was not intended for use with several checked out repositories where pushing back to origin is expected... If that's what you're intending, there may be other issues that crop up because we haven't thought that scenario would be a common one.

It's intended for convenience of building things you depend on, not as a git repository management system.
(0027826)
David Cole   
2011-11-18 14:40   
Sounds like what you want is simply:

  GIT_TAG master
  UPDATE_COMMAND ${GIT_EXECUTABLE} pull
(0027827)
Alex Neundorf   
2011-11-18 15:39   
Indeed, works.
Now that was simple :-)

I committed a patch to KDE superbuild, it should work now as expected.

Valentin, feel free to reopen this issue if it still doesn't work as expected.

Alex
(0027828)
David Cole   
2011-11-18 15:41   
Changing status to "resolved" so that Valentin can re-open if necessary...
(0029046)
David Cole   
2012-04-02 10:10   
Closing resolved issues that have not been updated in more than 4 months.