[Insight-users] Dashboard Submit Hangs
David Cole
david.cole at kitware.com
Wed Aug 27 10:57:02 EDT 2008
Wow! Really nice data... thanks.
Maybe you could send me Test.xml off the list to see if there's anything
funky about it. Or try uploading it to a random location (where you have
write access) using curl directly to see if the same problem exists through
raw curl...?
Thanks,
David
On Wed, Aug 27, 2008 at 10:46 AM, Kevin H. Hobbs <hobbsk at ohiou.edu> wrote:
> On Wednesday 27 August 2008 10:05:44 am David Cole wrote:
> > Can you get a call stack for this hang?
>
> There are two ctest processes:
> $ ps -A | grep ctest
> 1156 ? 00:00:08 ctest
> 28150 ? 00:00:00 ctest
>
> The back trace from gdb attached to 1156 is:
>
> #0 0x0000003ae7ccbd2f in __poll (fds=0x7fff604117b0, nfds=1, timeout=1000)
> at ../sysdeps/unix/sysv/linux/poll.c:83
> #1 0x0000003144a30556 in Curl_socket_ready () from /usr/lib64/libcurl.so.4
> #2 0x0000003144a268f1 in Curl_perform () from /usr/lib64/libcurl.so.4
> #3 0x00000000004cb1d3 in cmCTestSubmitHandler::SubmitUsingHTTP
> (this=0xc69780, localprefix=@0x7fff604129d0,
> files=<value optimized out>, remoteprefix=@0x7fff604129c0,
> url=@0x7fff604129b0)
> at
> /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestSubmitHandler.cxx:366
> #4 0x00000000004cfc99 in cmCTestSubmitHandler::ProcessHandler
> (this=0xc69780)
> at
> /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestSubmitHandler.cxx:1006
> #5 0x000000000048514e in cmCTest::ProcessTests (this=0x7fff60413450)
> at /home/kevin/kitware/cmake-2.6.1/Source/cmCTest.cxx:911
> #6 0x0000000000488735 in cmCTest::Run (this=0x7fff60413450,
> args=@0x7fff604138f0, output=0x7fff60413910)
> at /home/kevin/kitware/cmake-2.6.1/Source/cmCTest.cxx:1988
> #7 0x0000000000474458 in main (argc=5, argv=0x7fff60413950) at
> /home/kevin/kitware/cmake-2.6.1/Source/ctest.cxx:274
>
> The back trace from gdb attached to 28150 is:
>
> #0 0x0000003ae7ccdda3 in __select_nocancel () from /lib64/libc.so.6
> #1 0x00000000006a3c83 in cmsysProcess_WaitForData (cp=0x239ebe0,
> data=0x7fff72817ab0, length=0x7fff72817acc,
> userTimeout=0x0) at
> /home/kevin/kitware/cmake-2.6.1/Source/kwsys/ProcessUNIX.c:1120
> #2 0x0000000000567d68 in cmSystemTools::RunSingleCommand (command=<value
> optimized out>, output=0x7fff72818190,
> retVal=0x7fff72818278, dir=0x238d488
> "/home/kevin/kitware/Insight_build", verbose=false, timeout=0)
> at /home/kevin/kitware/cmake-2.6.1/Source/cmSystemTools.cxx:627
> #3 0x000000000056a1ee in cmSystemTools::RunSingleCommand (command=<value
> optimized out>, output=0x7fff72818190,
> retVal=0x7fff72818278, dir=0x238d488
> "/home/kevin/kitware/Insight_build", verbose=false, timeout=0)
> at /home/kevin/kitware/cmake-2.6.1/Source/cmSystemTools.cxx:738
> #4 0x00000000004c0655 in cmCTestScriptHandler::RunConfigurationDashboard
> (this=0x23568d0)
> at
> /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestScriptHandler.cxx:898
> #5 0x00000000004c166b in cmCTestScriptHandler::RunCurrentScript
> (this=0x23568d0)
> at
> /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestScriptHandler.cxx:618
> #6 0x00000000004c2938 in cmCTestScriptHandler::RunConfigurationScript
> (this=0x23568d0, total_script_arg=@0x7fff72818540,
> pscope=<value optimized out>) at
> /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestScriptHandler.cxx:552
> #7 0x00000000004c2b8e in cmCTestScriptHandler::ProcessHandler
> (this=0x23568d0)
> at
> /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestScriptHandler.cxx:191
> #8 0x000000000048840a in cmCTest::Run (this=0x7fff72818880,
> args=@0x7fff72818d20, output=0x7fff72818d40)
> at /home/kevin/kitware/cmake-2.6.1/Source/cmCTest.cxx:1965
> #9 0x0000000000474458 in main (argc=3, argv=0x7fff72818d80) at
> /home/kevin/kitware/cmake-2.6.1/Source/ctest.cxx:274
>
> Is that what you want?
>
> > Or tell us what CMake is trying to
> > do when it happens?
>
> lsof says they have the following open files:
>
> $ /usr/sbin/lsof | grep ctest
> ctest 1156 kevin cwd DIR 9,3 4096
> 41844737 /home/kevin/kitware/Insight_build
> ctest 1156 kevin rtd DIR 9,2 4096
> 2 /
> ctest 1156 kevin txt REG 9,3 35211646
> 51807747 /home/kevin/kitware/cmake-2.6.1_Build/bin/ctest
> ctest 1156 kevin mem REG 9,2 53816
> 6730460 /lib64/libnss_files-2.7.so
> ctest 1156 kevin mem REG 9,2 23496
> 6730458 /lib64/libnss_dns-2.7.so
> ctest 1156 kevin mem REG 9,2 175280
> 1894831 /usr/lib64/libsmime3.so
> ctest 1156 kevin mem REG 9,2 118440
> 1880925 /usr/lib64/libnssutil3.so
> ctest 1156 kevin mem REG 9,2 1317984
> 1892726 /usr/lib64/libnss3.so
> ctest 1156 kevin mem REG 9,2 196840
> 1894620 /usr/lib64/libssl3.so
> ctest 1156 kevin mem REG 9,2 267464
> 8462089 /usr/lib64/libcurl.so.4.1.0
> ctest 1156 kevin mem REG 9,2 11824
> 6730608 /lib64/libcom_err.so.2.1
> ctest 1156 kevin mem REG 9,2 36216
> 1869427 /usr/lib64/libkrb5support.so.0.1
> ctest 1156 kevin mem REG 9,2 191400
> 1871936 /usr/lib64/libgssapi_krb5.so.2.2
> ctest 1156 kevin mem REG 9,2 153664
> 1871652 /usr/lib64/libk5crypto.so.3.1
> ctest 1156 kevin mem REG 9,2 602376
> 1871808 /usr/lib64/libkrb5.so.3.3
> ctest 1156 kevin mem REG 9,2 133616
> 6730743 /lib64/ld-2.7.so
> ctest 1156 kevin mem REG 9,2 53464
> 6730753 /lib64/librt-2.7.so
> ctest 1156 kevin mem REG 9,2 1717544
> 6730744 /lib64/libc-2.7.so
> ctest 1156 kevin mem REG 9,2 611096
> 6730745 /lib64/libm-2.7.so
> ctest 1156 kevin mem REG 9,2 23288
> 6730746 /lib64/libdl-2.7.so
> ctest 1156 kevin mem REG 9,2 143336
> 6730752 /lib64/libpthread-2.7.so
> ctest 1156 kevin mem REG 9,2 85904
> 6730750 /lib64/libz.so.1.2.3
> ctest 1156 kevin mem REG 9,2 92816
> 6730759 /lib64/libresolv-2.7.so
> ctest 1156 kevin mem REG 9,2 58600
> 6730767 /lib64/libgcc_s-4.1.2-20070925.so.1
> ctest 1156 kevin mem REG 9,2 9728
> 6730758 /lib64/libkeyutils-1.2.so
> ctest 1156 kevin mem REG 9,2 976512
> 1892647 /usr/lib64/libstdc++.so.6.0.8
> ctest 1156 kevin mem REG 9,2 108064
> 6730452 /lib64/libselinux.so.1
> ctest 1156 kevin mem REG 9,2 208848
> 1876559 /usr/lib64/libidn.so.11.5.28
> ctest 1156 kevin mem REG 9,2 14064
> 1892811 /usr/lib64/libplds4.so
> ctest 1156 kevin mem REG 9,2 18064
> 1892815 /usr/lib64/libplc4.so
> ctest 1156 kevin mem REG 9,2 229280
> 1892809 /usr/lib64/libnspr4.so
> ctest 1156 kevin 0r FIFO 0,6
> 2136348 pipe
> ctest 1156 kevin 1w FIFO 0,6
> 2282007 pipe
> ctest 1156 kevin 2w FIFO 0,6
> 2282006 pipe
> ctest 1156 kevin 3w REG 9,3 776
> 59179715
> /home/kevin/kitware/Insight_build/Testing/Temporary/LastSubmit_20080827-0100.log.tmp
> ctest 1156 kevin 4r REG 9,3 7425867
> 59637761 /home/kevin/kitware/Insight_build/Testing/20080827-0100/Test.xml
> ctest 1156 kevin 5u IPv4 2619763
> TCP gargon.hooperlab:40824->66-194-253-26.static.twtelecom.net:http
> (ESTABLISHED)
> ctest 28150 kevin cwd DIR 9,3 4096
> 41058388 /home/kevin/kitware
> ctest 28150 kevin rtd DIR 9,2 4096
> 2 /
> ctest 28150 kevin txt REG 9,3 35211646
> 51807747 /home/kevin/kitware/cmake-2.6.1_Build/bin/ctest
> ctest 28150 kevin mem REG 9,2 175280
> 1894831 /usr/lib64/libsmime3.so
> ctest 28150 kevin mem REG 9,2 118440
> 1880925 /usr/lib64/libnssutil3.so
> ctest 28150 kevin mem REG 9,2 1317984
> 1892726 /usr/lib64/libnss3.so
> ctest 28150 kevin mem REG 9,2 196840
> 1894620 /usr/lib64/libssl3.so
> ctest 28150 kevin mem REG 9,2 267464
> 8462089 /usr/lib64/libcurl.so.4.1.0
> ctest 28150 kevin mem REG 9,2 11824
> 6730608 /lib64/libcom_err.so.2.1
> ctest 28150 kevin mem REG 9,2 36216
> 1869427 /usr/lib64/libkrb5support.so.0.1
> ctest 28150 kevin mem REG 9,2 191400
> 1871936 /usr/lib64/libgssapi_krb5.so.2.2
> ctest 28150 kevin mem REG 9,2 153664
> 1871652 /usr/lib64/libk5crypto.so.3.1
> ctest 28150 kevin mem REG 9,2 602376
> 1871808 /usr/lib64/libkrb5.so.3.3
> ctest 28150 kevin mem REG 9,2 133616
> 6730743 /lib64/ld-2.7.so
> ctest 28150 kevin mem REG 9,2 53464
> 6730753 /lib64/librt-2.7.so
> ctest 28150 kevin mem REG 9,2 1717544
> 6730744 /lib64/libc-2.7.so
> ctest 28150 kevin mem REG 9,2 611096
> 6730745 /lib64/libm-2.7.so
> ctest 28150 kevin mem REG 9,2 23288
> 6730746 /lib64/libdl-2.7.so
> ctest 28150 kevin mem REG 9,2 143336
> 6730752 /lib64/libpthread-2.7.so
> ctest 28150 kevin mem REG 9,2 85904
> 6730750 /lib64/libz.so.1.2.3
> ctest 28150 kevin mem REG 9,2 92816
> 6730759 /lib64/libresolv-2.7.so
> ctest 28150 kevin mem REG 9,2 58600
> 6730767 /lib64/libgcc_s-4.1.2-20070925.so.1
> ctest 28150 kevin mem REG 9,2 9728
> 6730758 /lib64/libkeyutils-1.2.so
> ctest 28150 kevin mem REG 9,2 976512
> 1892647 /usr/lib64/libstdc++.so.6.0.8
> ctest 28150 kevin mem REG 9,2 108064
> 6730452 /lib64/libselinux.so.1
> ctest 28150 kevin mem REG 9,2 208848
> 1876559 /usr/lib64/libidn.so.11.5.28
> ctest 28150 kevin mem REG 9,2 14064
> 1892811 /usr/lib64/libplds4.so
> ctest 28150 kevin mem REG 9,2 18064
> 1892815 /usr/lib64/libplc4.so
> ctest 28150 kevin mem REG 9,2 229280
> 1892809 /usr/lib64/libnspr4.so
> ctest 28150 kevin 0r FIFO 0,6
> 2136348 pipe
> ctest 28150 kevin 1w FIFO 0,6
> 2136349 pipe
> ctest 28150 kevin 2w FIFO 0,6
> 2136349 pipe
> ctest 28150 kevin 3r FIFO 0,6
> 2282005 pipe
> ctest 28150 kevin 4w FIFO 0,6
> 2282005 pipe
> ctest 28150 kevin 5r FIFO 0,6
> 2282006 pipe
> ctest 28150 kevin 7r FIFO 0,6
> 2282007 pipe
>
> > Or the last thing it did before the hang...?
>
> The last thing it wrote to
> Testing/Temporary/LastSubmit_20080827-0100.log.tmp was:
>
> Upload files:
> 0 Build.xml
> 1 Configure.xml
> 2 Notes.xml
> 3 Test.xml
> 4 Update.xml
> Using drop method: HTTP
> Upload file:
> /home/kevin/kitware/Insight_build/Testing/20080827-0100/Build.xml to
>
> gargon.hooperlab___Linux-gcc-4.1.2-x86_64___20080827-0100-Nightly___XML___Build.xml
> Upload file:
> /home/kevin/kitware/Insight_build/Testing/20080827-0100/Configure.xml to
>
> gargon.hooperlab___Linux-gcc-4.1.2-x86_64___20080827-0100-Nightly___XML___Configure.xml
> Upload file:
> /home/kevin/kitware/Insight_build/Testing/20080827-0100/Notes.xml to
>
> gargon.hooperlab___Linux-gcc-4.1.2-x86_64___20080827-0100-Nightly___XML___Notes.xml
> Upload file:
> /home/kevin/kitware/Insight_build/Testing/20080827-0100/Test.xml to
>
> gargon.hooperlab___Linux-gcc-4.1.2-x86_64___20080827-0100-Nightly___XML___Test.xml
>
>
> The last thing it wrote to Testing/20080827-0100/Test.xml was:
>
> <Measurement>
> <Value>MaximumRatioDecisionRule: The index
> of the chosen = 1
> </Value>
> </Measurement>
> </Results>
> </Test>
> <EndDateTime>Aug 27 02:14 EDT</EndDateTime>
> <EndTestTime>1219817640</EndTestTime>
> <ElapsedMinutes>26.4</ElapsedMinutes></Testing>
> </Site>
>
> Is there anything else I can do with the running processes?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/pipermail/insight-users/attachments/20080827/63ee6035/attachment-0001.htm>
More information about the Insight-users
mailing list