Yeah Brad knows about it and helped create this temporary solution. We should still make a formal bug report though so it isn't forgotten.<br><br><div class="gmail_quote">On Mon, May 3, 2010 at 1:20 PM, Moreland, Kenneth <span dir="ltr"><<a href="mailto:kmorel@sandia.gov">kmorel@sandia.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div>
<font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt">OK, I finally got a chance to implement this, and it looks alright so far.<br>
<br>
This seems like something CTest should simply handle as part of updating a project. Has anyone submitted a bug (or feature request depending on your POV) to the CMake project?<br>
<br>
-Ken<div class="im"><br>
<br>
<br>
On 4/28/10 8:56 AM, "Dave Partyka" <<a href="http://dave.partyka@kitware.com" target="_blank">dave.partyka@kitware.com</a>> wrote:<br>
<br>
</div></span></font><blockquote><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt"><div class="im">Hey Ken and Kevin, Here is how to fix ctest_update so that the update step doesn't submit erroneous modified files due to submodule update.<br>
<br>
ctest_update only does a pull and then other things to see if there are locally modified files. Because of this, it will always fail in the presence of submodules because after the pull they will report they have been modified until you submodule update. To fix this we have made tiny wrapper scripts that will pull and submodule update in a single step. What you have to do in your dashboard script is configure the appropriate git script (sets the path to git). And then override CTEST_GIT_COMMAND to call that script rather than git directly. Here is what I am doing in paraview_common.<br>
<br>
# look for the git mod scripts in the same dir as the script.<br>
get_filename_component(dashboard_self_dir ${CMAKE_CURRENT_LIST_FILE} PATH)<br>
<br>
if(UNIX)<br></div>
configure_file(${dashboard_self_dir}/<a href="http://gitmod.sh.in" target="_blank">gitmod.sh.in</a> <<a href="http://gitmod.sh.in" target="_blank">http://gitmod.sh.in</a>> <br><div class="im">
${CTEST_DASHBOARD_ROOT}/gitmod.sh<br>
@ONLY)<br>
set(CTEST_GIT_COMMAND ${CTEST_DASHBOARD_ROOT}/gitmod.sh)<br>
else()<br></div>
configure_file(${dashboard_self_dir}/<a href="http://gitmod.bat.in" target="_blank">gitmod.bat.in</a> <<a href="http://gitmod.bat.in" target="_blank">http://gitmod.bat.in</a>> <br><div class="im">
${CTEST_DASHBOARD_ROOT}/gitmod.bat<br>
@ONLY)<br>
set(CTEST_GIT_COMMAND ${CTEST_DASHBOARD_ROOT}/gitmod.bat)<br>
endif()<br>
<br>
-------------------------------------------------------------------------------------------------------------------<br></div>
contents of <a href="http://gitmod.bat.in" target="_blank">gitmod.bat.in</a> <<a href="http://gitmod.bat.in" target="_blank">http://gitmod.bat.in</a>> <br><div class="im">
<br>
@echo off<br>
set GIT=@CTEST_GIT_COMMAND@<br>
call "%GIT%" %*<br>
if "%1" equ "pull" (<br>
"%GIT%" submodule update<br>
)<br>
<br>
-------------------------------------------------------------------------------------------------------------------<br></div>
contents of <a href="http://gitmod.sh.in" target="_blank">gitmod.sh.in</a> <<a href="http://gitmod.sh.in" target="_blank">http://gitmod.sh.in</a>> <br><div class="im">
<br>
#!/bin/sh<br>
GIT="@CTEST_GIT_COMMAND@"<br>
"$GIT" "$@" &&<br>
if test "$1" == "pull"; then<br>
"$GIT" submodule update<br>
fi<br>
<br>
<br>
<br>
On Thu, Apr 22, 2010 at 5:49 PM, Moreland, Kenneth <<a href="http://kmorel@sandia.gov" target="_blank">kmorel@sandia.gov</a>> wrote:<br>
</div></span></font><blockquote><div class="im"><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt">My script does <i>not</i> do the initial clone. It updates an existing repository.<br>
<br>
The problem is that ctest_update is performing two basic operations: pull from the origin repository (calls git pull) and check that the local repository is consistent (I don’t know how, but let’s say by calling git status). The problem is that the local repository won’t actually be consistent until the submodules are updated.<br>
<br>
Thus, if I call submodule update before ctest_update, I get the following sequence of events.<br>
<br>
</span></font><blockquote><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt">git submodule update<br>
git pull<br>
<i>check </i>git status<br>
</span></font></blockquote><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt"><br>
If I call submodule update after calling ctest_update, I get this sequence of events.<br>
<br>
</span></font><blockquote><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt">git pull<br>
<i>check </i>git status<br>
git submodule update<br>
</span></font></blockquote></div><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt"><div class="im"><br>
This is marginally better because now at least the end result is a fully updated repository. But the problem is that by the time the submodules are updated, the consistency check has already occurred and the erroneous error has already been recorded.<br>
<br>
-Ken<br>
<br>
<br></div><div class="im">
On 4/22/10 3:38 PM, "Dave Partyka" <<a href="http://dave.partyka@kitware.com" target="_blank">dave.partyka@kitware.com</a> <<a href="http://dave.partyka@kitware.com" target="_blank">http://dave.partyka@kitware.com</a>> > wrote:<br>
<br>
</div></span></font><blockquote><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt"><div class="im">Sorry you are right, it should be after ctest_update(). What is the reason that ctest_update() is failing for you? Because the submodules don't exist yet? <br>
<br>
Does your script do the initial clone?<br>
<br>
If so, you're going to want to do something like:<br>
<br>
if(NOT EXISTS src_dir)<br>
clone<br>
<optionally checkout a branch><br>
submodule init<br>
submodule update<br>
endif()<br>
<br>
ctest_update()<br>
<br>
submodule update<br>
<br>
<br></div><div class="im">
On Thu, Apr 22, 2010 at 5:18 PM, Moreland, Kenneth <<a href="http://kmorel@sandia.gov" target="_blank">kmorel@sandia.gov</a> <<a href="http://kmorel@sandia.gov" target="_blank">http://kmorel@sandia.gov</a>> > wrote:<br>
</div></span></font><blockquote><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt"><div class="im">How will that work? If I put it before ctest_update, then it will run before the VTK submodule SHA gets updated. Thus, it my not grab the commit that you actually need. Or does the submodule update command implicitly fetch everything from the remote regardless of whether it is needed?<br>
<br>
-Ken<br>
<br>
<br>
<br></div><div class="im">
On 4/22/10 3:02 PM, "Dave Partyka" <<a href="http://dave.partyka@kitware.com" target="_blank">dave.partyka@kitware.com</a> <<a href="http://dave.partyka@kitware.com" target="_blank">http://dave.partyka@kitware.com</a>> <<a href="http://dave.partyka@kitware.com" target="_blank">http://dave.partyka@kitware.com</a>> > wrote:<br>
<br>
</div></span></font><blockquote><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt"><div class="im">You want to put it before ctest_update is called.<br>
<br></div><div class="im">
On Thu, Apr 22, 2010 at 3:40 PM, Moreland, Kenneth <<a href="http://kmorel@sandia.gov" target="_blank">kmorel@sandia.gov</a> <<a href="http://kmorel@sandia.gov" target="_blank">http://kmorel@sandia.gov</a>> <<a href="http://kmorel@sandia.gov" target="_blank">http://kmorel@sandia.gov</a>> > wrote:<br>
</div></span></font><blockquote><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt"><div class="im">I tried adding your execute_process after my call to ctest_update, but I don’t think it’s quite working. The problem is that the submodule update command does not get run until after the ctest_update call returns, and by that time CTest has already determined that the update has failed.<br>
<br>
Where exactly should I be putting the execute_process?<br>
<br>
-Ken<br>
<br>
<br>
<br></div><div class="im">
On 4/20/10 7:07 AM, "Dave Partyka" <<a href="http://dave.partyka@kitware.com" target="_blank">dave.partyka@kitware.com</a> <<a href="http://dave.partyka@kitware.com" target="_blank">http://dave.partyka@kitware.com</a>> <<a href="http://dave.partyka@kitware.com" target="_blank">http://dave.partyka@kitware.com</a>> <<a href="http://dave.partyka@kitware.com" target="_blank">http://dave.partyka@kitware.com</a>> > wrote:<br>
<br>
</div></span></font><blockquote><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt"><div class="im">You're going to want to add something like this in your script for initial checkout:<br>
<br>
execute_process(<br>
COMMAND \"${CTEST_GIT_COMMAND}\" submodule init<br>
WORKING_DIRECTORY \"${CTEST_SOURCE_DIRECTORY}\"<br>
)<br>
<br>
And for each update you'll want:<br>
<br>
execute_process(<br>
COMMAND \"${CTEST_GIT_COMMAND}\" submodule update --recursive --<br>
WORKING_DIRECTORY \"${CTEST_SOURCE_DIRECTORY}\"<br>
)<br>
<br>
Also this will be handy:<br>
<br>
# Look for a GIT command-line client.<br>
if(NOT DEFINED CTEST_GIT_COMMAND)<br>
find_program(CTEST_GIT_COMMAND NAMES git git.cmd)<br>
endif()<br>
<br></div><div class="im">
On Mon, Apr 19, 2010 at 7:16 PM, Dave Partyka <<a href="http://dave.partyka@kitware.com" target="_blank">dave.partyka@kitware.com</a> <<a href="http://dave.partyka@kitware.com" target="_blank">http://dave.partyka@kitware.com</a>> <<a href="http://dave.partyka@kitware.com" target="_blank">http://dave.partyka@kitware.com</a>> <<a href="http://dave.partyka@kitware.com" target="_blank">http://dave.partyka@kitware.com</a>> > wrote:<br>
</div></span></font></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt"><div class="im">I do not know if ctest knows how to update submodules automatically. I have it on my todo list to talk to Brad King about this for the same reason for updating the ParaView dashboards here at Kitware. I'll share what I find out.<br>
<br>
<br></div><div class="im">
On Mon, Apr 19, 2010 at 7:13 PM, Moreland, Kenneth <<a href="http://kmorel@sandia.gov" target="_blank">kmorel@sandia.gov</a> <<a href="http://kmorel@sandia.gov" target="_blank">http://kmorel@sandia.gov</a>> <<a href="http://kmorel@sandia.gov" target="_blank">http://kmorel@sandia.gov</a>> <<a href="http://kmorel@sandia.gov" target="_blank">http://kmorel@sandia.gov</a>> > wrote:<br>
Yes. That seems to fix the problem. However, is there a way to get ctest to run that after pulling? Otherwise we’ll still get a bunch of errors in the dashboard.<br>
<br>
-Ken<br>
<br>
<br>
<br></div><div class="im">
On 4/19/10 5:09 PM, "Dave Partyka" <<a href="http://dave.partyka@kitware.com" target="_blank">dave.partyka@kitware.com</a> <<a href="http://dave.partyka@kitware.com" target="_blank">http://dave.partyka@kitware.com</a>> <<a href="http://dave.partyka@kitware.com" target="_blank">http://dave.partyka@kitware.com</a>> <<a href="http://dave.partyka@kitware.com" target="_blank">http://dave.partyka@kitware.com</a>> <<a href="http://dave.partyka@kitware.com" target="_blank">http://dave.partyka@kitware.com</a>> > wrote:<br>
<br>
</div></span></font><blockquote><blockquote><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt"><div class="im">does 'git submodule update' help?<br>
<br></div><div class="im">
On Mon, Apr 19, 2010 at 7:02 PM, Moreland, Kenneth <<a href="http://kmorel@sandia.gov" target="_blank">kmorel@sandia.gov</a> <<a href="http://kmorel@sandia.gov" target="_blank">http://kmorel@sandia.gov</a>> <<a href="http://kmorel@sandia.gov" target="_blank">http://kmorel@sandia.gov</a>> <<a href="http://kmorel@sandia.gov" target="_blank">http://kmorel@sandia.gov</a>> <<a href="http://kmorel@sandia.gov" target="_blank">http://kmorel@sandia.gov</a>> > wrote:<br>
I noticed something odd when converting to the new git repositories today. After only cloning the repository, setting up the modules, and then pulling (using the --rebase flag as described on the Wiki), status reports that I have made a change in my repository even though I have not. Here is a terminal capture of a newly cloned ParaView repository (after modules and hooks are set up).<br>
<br>
</div></span></font><blockquote><blockquote><font size="2"><font face="Consolas, Courier New, Courier"><span style="font-size:10pt"><div class="im">kmorel2 0> git status /Users/kmorel/src/ParaView<br>
# On branch master<br>
nothing to commit (working directory clean)<br>
kmorel2 1> git pull --rebase /Users/kmorel/src/ParaView<br>
remote: Counting objects: 44, done.<br>
remote: Compressing objects: 100% (26/26), done.<br>
remote: Total 26 (delta 20), reused 0 (delta 0)<br>
Unpacking objects: 100% (26/26), done.<br></div>