[Insight-users] Dashboard Submit Hangs
Kevin H. Hobbs
hobbsk at ohiou.edu
Wed Aug 27 10:46:53 EDT 2008
On Wednesday 27 August 2008 10:05:44 am David Cole wrote:
> Can you get a call stack for this hang?
There are two ctest processes:
$ ps -A | grep ctest
1156 ? 00:00:08 ctest
28150 ? 00:00:00 ctest
The back trace from gdb attached to 1156 is:
#0 0x0000003ae7ccbd2f in __poll (fds=0x7fff604117b0, nfds=1, timeout=1000) at ../sysdeps/unix/sysv/linux/poll.c:83
#1 0x0000003144a30556 in Curl_socket_ready () from /usr/lib64/libcurl.so.4
#2 0x0000003144a268f1 in Curl_perform () from /usr/lib64/libcurl.so.4
#3 0x00000000004cb1d3 in cmCTestSubmitHandler::SubmitUsingHTTP (this=0xc69780, localprefix=@0x7fff604129d0,
files=<value optimized out>, remoteprefix=@0x7fff604129c0, url=@0x7fff604129b0)
at /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestSubmitHandler.cxx:366
#4 0x00000000004cfc99 in cmCTestSubmitHandler::ProcessHandler (this=0xc69780)
at /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestSubmitHandler.cxx:1006
#5 0x000000000048514e in cmCTest::ProcessTests (this=0x7fff60413450)
at /home/kevin/kitware/cmake-2.6.1/Source/cmCTest.cxx:911
#6 0x0000000000488735 in cmCTest::Run (this=0x7fff60413450, args=@0x7fff604138f0, output=0x7fff60413910)
at /home/kevin/kitware/cmake-2.6.1/Source/cmCTest.cxx:1988
#7 0x0000000000474458 in main (argc=5, argv=0x7fff60413950) at /home/kevin/kitware/cmake-2.6.1/Source/ctest.cxx:274
The back trace from gdb attached to 28150 is:
#0 0x0000003ae7ccdda3 in __select_nocancel () from /lib64/libc.so.6
#1 0x00000000006a3c83 in cmsysProcess_WaitForData (cp=0x239ebe0, data=0x7fff72817ab0, length=0x7fff72817acc,
userTimeout=0x0) at /home/kevin/kitware/cmake-2.6.1/Source/kwsys/ProcessUNIX.c:1120
#2 0x0000000000567d68 in cmSystemTools::RunSingleCommand (command=<value optimized out>, output=0x7fff72818190,
retVal=0x7fff72818278, dir=0x238d488 "/home/kevin/kitware/Insight_build", verbose=false, timeout=0)
at /home/kevin/kitware/cmake-2.6.1/Source/cmSystemTools.cxx:627
#3 0x000000000056a1ee in cmSystemTools::RunSingleCommand (command=<value optimized out>, output=0x7fff72818190,
retVal=0x7fff72818278, dir=0x238d488 "/home/kevin/kitware/Insight_build", verbose=false, timeout=0)
at /home/kevin/kitware/cmake-2.6.1/Source/cmSystemTools.cxx:738
#4 0x00000000004c0655 in cmCTestScriptHandler::RunConfigurationDashboard (this=0x23568d0)
at /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestScriptHandler.cxx:898
#5 0x00000000004c166b in cmCTestScriptHandler::RunCurrentScript (this=0x23568d0)
at /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestScriptHandler.cxx:618
#6 0x00000000004c2938 in cmCTestScriptHandler::RunConfigurationScript (this=0x23568d0, total_script_arg=@0x7fff72818540,
pscope=<value optimized out>) at /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestScriptHandler.cxx:552
#7 0x00000000004c2b8e in cmCTestScriptHandler::ProcessHandler (this=0x23568d0)
at /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestScriptHandler.cxx:191
#8 0x000000000048840a in cmCTest::Run (this=0x7fff72818880, args=@0x7fff72818d20, output=0x7fff72818d40)
at /home/kevin/kitware/cmake-2.6.1/Source/cmCTest.cxx:1965
#9 0x0000000000474458 in main (argc=3, argv=0x7fff72818d80) at /home/kevin/kitware/cmake-2.6.1/Source/ctest.cxx:274
Is that what you want?
> Or tell us what CMake is trying to
> do when it happens?
lsof says they have the following open files:
$ /usr/sbin/lsof | grep ctest
ctest 1156 kevin cwd DIR 9,3 4096 41844737 /home/kevin/kitware/Insight_build
ctest 1156 kevin rtd DIR 9,2 4096 2 /
ctest 1156 kevin txt REG 9,3 35211646 51807747 /home/kevin/kitware/cmake-2.6.1_Build/bin/ctest
ctest 1156 kevin mem REG 9,2 53816 6730460 /lib64/libnss_files-2.7.so
ctest 1156 kevin mem REG 9,2 23496 6730458 /lib64/libnss_dns-2.7.so
ctest 1156 kevin mem REG 9,2 175280 1894831 /usr/lib64/libsmime3.so
ctest 1156 kevin mem REG 9,2 118440 1880925 /usr/lib64/libnssutil3.so
ctest 1156 kevin mem REG 9,2 1317984 1892726 /usr/lib64/libnss3.so
ctest 1156 kevin mem REG 9,2 196840 1894620 /usr/lib64/libssl3.so
ctest 1156 kevin mem REG 9,2 267464 8462089 /usr/lib64/libcurl.so.4.1.0
ctest 1156 kevin mem REG 9,2 11824 6730608 /lib64/libcom_err.so.2.1
ctest 1156 kevin mem REG 9,2 36216 1869427 /usr/lib64/libkrb5support.so.0.1
ctest 1156 kevin mem REG 9,2 191400 1871936 /usr/lib64/libgssapi_krb5.so.2.2
ctest 1156 kevin mem REG 9,2 153664 1871652 /usr/lib64/libk5crypto.so.3.1
ctest 1156 kevin mem REG 9,2 602376 1871808 /usr/lib64/libkrb5.so.3.3
ctest 1156 kevin mem REG 9,2 133616 6730743 /lib64/ld-2.7.so
ctest 1156 kevin mem REG 9,2 53464 6730753 /lib64/librt-2.7.so
ctest 1156 kevin mem REG 9,2 1717544 6730744 /lib64/libc-2.7.so
ctest 1156 kevin mem REG 9,2 611096 6730745 /lib64/libm-2.7.so
ctest 1156 kevin mem REG 9,2 23288 6730746 /lib64/libdl-2.7.so
ctest 1156 kevin mem REG 9,2 143336 6730752 /lib64/libpthread-2.7.so
ctest 1156 kevin mem REG 9,2 85904 6730750 /lib64/libz.so.1.2.3
ctest 1156 kevin mem REG 9,2 92816 6730759 /lib64/libresolv-2.7.so
ctest 1156 kevin mem REG 9,2 58600 6730767 /lib64/libgcc_s-4.1.2-20070925.so.1
ctest 1156 kevin mem REG 9,2 9728 6730758 /lib64/libkeyutils-1.2.so
ctest 1156 kevin mem REG 9,2 976512 1892647 /usr/lib64/libstdc++.so.6.0.8
ctest 1156 kevin mem REG 9,2 108064 6730452 /lib64/libselinux.so.1
ctest 1156 kevin mem REG 9,2 208848 1876559 /usr/lib64/libidn.so.11.5.28
ctest 1156 kevin mem REG 9,2 14064 1892811 /usr/lib64/libplds4.so
ctest 1156 kevin mem REG 9,2 18064 1892815 /usr/lib64/libplc4.so
ctest 1156 kevin mem REG 9,2 229280 1892809 /usr/lib64/libnspr4.so
ctest 1156 kevin 0r FIFO 0,6 2136348 pipe
ctest 1156 kevin 1w FIFO 0,6 2282007 pipe
ctest 1156 kevin 2w FIFO 0,6 2282006 pipe
ctest 1156 kevin 3w REG 9,3 776 59179715 /home/kevin/kitware/Insight_build/Testing/Temporary/LastSubmit_20080827-0100.log.tmp
ctest 1156 kevin 4r REG 9,3 7425867 59637761 /home/kevin/kitware/Insight_build/Testing/20080827-0100/Test.xml
ctest 1156 kevin 5u IPv4 2619763 TCP gargon.hooperlab:40824->66-194-253-26.static.twtelecom.net:http (ESTABLISHED)
ctest 28150 kevin cwd DIR 9,3 4096 41058388 /home/kevin/kitware
ctest 28150 kevin rtd DIR 9,2 4096 2 /
ctest 28150 kevin txt REG 9,3 35211646 51807747 /home/kevin/kitware/cmake-2.6.1_Build/bin/ctest
ctest 28150 kevin mem REG 9,2 175280 1894831 /usr/lib64/libsmime3.so
ctest 28150 kevin mem REG 9,2 118440 1880925 /usr/lib64/libnssutil3.so
ctest 28150 kevin mem REG 9,2 1317984 1892726 /usr/lib64/libnss3.so
ctest 28150 kevin mem REG 9,2 196840 1894620 /usr/lib64/libssl3.so
ctest 28150 kevin mem REG 9,2 267464 8462089 /usr/lib64/libcurl.so.4.1.0
ctest 28150 kevin mem REG 9,2 11824 6730608 /lib64/libcom_err.so.2.1
ctest 28150 kevin mem REG 9,2 36216 1869427 /usr/lib64/libkrb5support.so.0.1
ctest 28150 kevin mem REG 9,2 191400 1871936 /usr/lib64/libgssapi_krb5.so.2.2
ctest 28150 kevin mem REG 9,2 153664 1871652 /usr/lib64/libk5crypto.so.3.1
ctest 28150 kevin mem REG 9,2 602376 1871808 /usr/lib64/libkrb5.so.3.3
ctest 28150 kevin mem REG 9,2 133616 6730743 /lib64/ld-2.7.so
ctest 28150 kevin mem REG 9,2 53464 6730753 /lib64/librt-2.7.so
ctest 28150 kevin mem REG 9,2 1717544 6730744 /lib64/libc-2.7.so
ctest 28150 kevin mem REG 9,2 611096 6730745 /lib64/libm-2.7.so
ctest 28150 kevin mem REG 9,2 23288 6730746 /lib64/libdl-2.7.so
ctest 28150 kevin mem REG 9,2 143336 6730752 /lib64/libpthread-2.7.so
ctest 28150 kevin mem REG 9,2 85904 6730750 /lib64/libz.so.1.2.3
ctest 28150 kevin mem REG 9,2 92816 6730759 /lib64/libresolv-2.7.so
ctest 28150 kevin mem REG 9,2 58600 6730767 /lib64/libgcc_s-4.1.2-20070925.so.1
ctest 28150 kevin mem REG 9,2 9728 6730758 /lib64/libkeyutils-1.2.so
ctest 28150 kevin mem REG 9,2 976512 1892647 /usr/lib64/libstdc++.so.6.0.8
ctest 28150 kevin mem REG 9,2 108064 6730452 /lib64/libselinux.so.1
ctest 28150 kevin mem REG 9,2 208848 1876559 /usr/lib64/libidn.so.11.5.28
ctest 28150 kevin mem REG 9,2 14064 1892811 /usr/lib64/libplds4.so
ctest 28150 kevin mem REG 9,2 18064 1892815 /usr/lib64/libplc4.so
ctest 28150 kevin mem REG 9,2 229280 1892809 /usr/lib64/libnspr4.so
ctest 28150 kevin 0r FIFO 0,6 2136348 pipe
ctest 28150 kevin 1w FIFO 0,6 2136349 pipe
ctest 28150 kevin 2w FIFO 0,6 2136349 pipe
ctest 28150 kevin 3r FIFO 0,6 2282005 pipe
ctest 28150 kevin 4w FIFO 0,6 2282005 pipe
ctest 28150 kevin 5r FIFO 0,6 2282006 pipe
ctest 28150 kevin 7r FIFO 0,6 2282007 pipe
> Or the last thing it did before the hang...?
The last thing it wrote to Testing/Temporary/LastSubmit_20080827-0100.log.tmp was:
Upload files:
0 Build.xml
1 Configure.xml
2 Notes.xml
3 Test.xml
4 Update.xml
Using drop method: HTTP
Upload file: /home/kevin/kitware/Insight_build/Testing/20080827-0100/Build.xml to
gargon.hooperlab___Linux-gcc-4.1.2-x86_64___20080827-0100-Nightly___XML___Build.xml
Upload file: /home/kevin/kitware/Insight_build/Testing/20080827-0100/Configure.xml to
gargon.hooperlab___Linux-gcc-4.1.2-x86_64___20080827-0100-Nightly___XML___Configure.xml
Upload file: /home/kevin/kitware/Insight_build/Testing/20080827-0100/Notes.xml to
gargon.hooperlab___Linux-gcc-4.1.2-x86_64___20080827-0100-Nightly___XML___Notes.xml
Upload file: /home/kevin/kitware/Insight_build/Testing/20080827-0100/Test.xml to
gargon.hooperlab___Linux-gcc-4.1.2-x86_64___20080827-0100-Nightly___XML___Test.xml
The last thing it wrote to Testing/20080827-0100/Test.xml was:
<Measurement>
<Value>MaximumRatioDecisionRule: The index of the chosen = 1
</Value>
</Measurement>
</Results>
</Test>
<EndDateTime>Aug 27 02:14 EDT</EndDateTime>
<EndTestTime>1219817640</EndTestTime>
<ElapsedMinutes>26.4</ElapsedMinutes></Testing>
</Site>
Is there anything else I can do with the running processes?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
URL: <http://www.itk.org/pipermail/insight-users/attachments/20080827/d5439ae1/attachment.pgp>
More information about the Insight-users
mailing list