[Cdash] Upload to CDash fails for huge tests amount

[Matějů Miroslav, Ing.]

Hi Julien,

Thanks for your advice. I have set CDASH_ASYNCHRONOUS_SUBMISSION to true and my submissions are processed correctly now – but only after running cdash/processsubmissions.php manually (following http://public.kitware.com/pipermail/cdash/2014-January/001405.html). However, the processing does not start automatically. I tried to find some documentation about CDash Asynchronous submissions with no success. The only useful page is http://www.kitware.com/source/home/post/59 but it does not mention what (apart from the config file) should I do to process the asynchronous submissions. I’m supposing it should work automatically (without setting cron, for example) after reading the mail archive mentioned above.

The server configuration does not allow to access localhost (or 127.0.0.1) using HTTP, I have to use the local IP address to access webpages. So I’ve set the following values in cdash/config.local.php:

$CDASH_USE_HTTPS ='0';
// Name of the server running CDash.
// Leave empty to use current name and default port.
$CDASH_SERVER_NAME = '192.168.2.105';
$CDASH_SERVER_PORT = '';
// If the remote request should use localhost or the full name
// This variable should be set to 1 in most of the server configurations
$CDASH_CURL_REQUEST_LOCALHOST='0';
$CDASH_CURL_LOCALHOST_PREFIX='cdash';
$CDASH_BASE_URL='';
// Define the location of the local directory
$CDASH_USE_LOCAL_DIRECTORY = '0';

Is that correct? I miss any documentation of these variables except the comments (which do not help when I’m not the “most”). I am not sure with CDASH_CURL_LOCALHOST_PREFIX, the full path to process submissions manually is http://192.168.2.105/cdash/cdash/processsubmissions.php.

Best regards,

Miroslav Matějů

-----Original Message-----
From: Julien Jomier [mailto:julien.jomier at kitware.com] 
Sent: Thursday, May 15, 2014 9:29 AM

Hi Miroslav,

Are you running CDash with Asynchronous submissions? 
($CDASH_ASYNCHRONOUS_SUBMISSION=true in the config.local.php)

If not that might be the issue,
Julien

On 15/05/2014 09:16, Matějů Miroslav, Ing. wrote:
> Hello there,
>
> I’ve got a problem with the upload from CTest to CDash. I have many 
> test suites (considered as named builds for CDash), each of them with 
> several tests. The update info is not transferred to CDash for some 
> test suites with a huge amount of tests. To give you an idea what “huge amount”
> means: There is currently being a transfer to new hardware for both 
> the CTest and CDash machines. For the older HW, it means more than 
> 1000, and about 10000 for the newer one.
>
> Some useful info can be found in LastSubmit_*.log on the CTest machine 
> (I shortened it where I put “< >” marks):
>
> Upload files:
>
> 0       Build.xml
>
> 1       Configure.xml
>
> 2       Coverage.xml
>
> 3       CoverageLog-0.xml
>
> 4       DynamicAnalysis.xml
>
> 5       Notes.xml
>
> 6       Test.xml
>
> 7       Update.xml
>
> Using drop method: http
>
>          Upload file: <0 to 5>
>
>          Upload file: /<mypath>/Testing/20140514-0000/Test.xml to 
> <machine>___<build>___20140514-0000-Nightly___XML___Test.xml
>
>     Error when uploading file: 
> /<mypath>/Testing/20140514-0000/Test.xml
>
>     Error message was: Operation too slow. Less than 1 bytes/sec 
> transferred the last 120 seconds
>
>     Problems when submitting via HTTP
>
> <END OF FILE>
>
> Although CTest complains about sending Test.xml, it seems to be 
> transferred completely. The tests (including the last ones) present in 
> this file are shown correctly in CDash as well as configure, build, 
> coverage etc. However, it does not seem to even try to upload 
> Update.xml and there is no update info shown in CDash accordingly.
>
> I tried to enable repeated sending to CDash using
>
> CTEST_SUBMIT(RETRY_COUNT 10)
>
> in the CTest script. The only result was an overloaded CDash server 
> and even more upload errors.
>
> Could you give me any hints what can I do to correct this issue? Or is 
> it a bug which has to be corrected in the CDash itself? I’m running 
> CTest 2.8.7 and sending results to CDash 2.0.2.
>
> Thanks in advance!
>
> Miroslav Matějů
>
>
>
> _______________________________________________
> Cdash mailing list
> Cdash at public.kitware.com
> http://public.kitware.com/cgi-bin/mailman/listinfo/cdash
>