[Cdash] CDash submission problem

Tue Dec 2 19:55:30 UTC 2008

Julien,

thanks for the compliment but as I already said in my last email: the  
actual work has been done by William Candillon. ;-)
We would really appreciate if the changes make it into a future  
version of CDash. We think that it will solve this issue for a lot of  
other people also.

Best regards,

Matthias

On 01.12.2008, at 15:13, Julien Jomier wrote:

> Mathias,
>
> I have reviewed your modifications over the weekend. Very nice work.
> I had to change a couple of things to make it work with our current  
> client (CTest), but these were minor changes. I have also added the  
> support for backup of file when they arrived to the system.
>
> The plan is to switch to using the new model classes. I hope to have  
> time to work on this, this week.
>
> Thanks again,
> Julien
>
> Matthias Brantner wrote:
>> Hello Julien, hello Emmanuel,
>> some time has passed but we finally managed to come up with a patch  
>> that changes the CDash submission mechanism. William Candillon (in  
>> CC) is a colleague of mine at 28msec, Inc. He made the following  
>> two changes to CDash:
>> (1) remove the current test submission parse mechanism that stores  
>> all tests in an array
>> (2) instead there is one sax handler for each test submission (e.g.  
>> Build, Testing, Coverage). As a result, the incoming data is fully  
>> streamed and there is no need to load the complete input.
>> We have carefully tested the changes in our environment (approx.  
>> 16000 tests per submission) and with the test scenarios that you  
>> provide (see http://public.kitware.com/Wiki/CDash:XML). Everything  
>> seems to work fine and we were able to make a submission with ca.  
>> 16000 tests that required less then 8MB of memory. However, we had  
>> to pay the price that a submission might take slightly longer (but  
>> we think it's worth it because we are now able to reliably process  
>> our huge submissions).
>> We created a diff against the current svn head (Rev. 1267). You can  
>> find the diff at http://www.28msec.com/CDashSAX_rev1267.diff.
>> Could you please review this patch (if you find the time) and let  
>> us know what you think. As we have seen in your FAQ, there seem to  
>> be a lot of people that have the same problem as we had. Hence, it  
>> would be great if you could consider to integrate our changes in a  
>> next CDash release.
>> Best regards,
>> Matthias
>> On 07.10.2008, at 17:39, Julien Jomier wrote:
>>> Matthias,
>>>
>>> We have seen this behavior for very large submission and since  
>>> tests are not split by CTest the submission can be very big.
>>>
>>> I think we can improve this by customizing the XML parser in PHP  
>>> and instead of storing in an array, do the processing at parsing  
>>> time (as you are suggesting).
>>>
>>> If you have any experience with this, feel free to share :)
>>>
>>> I'll look at this when I get a chance. Thanks for the input,
>>> Julien
>>>
>>> Matthias Brantner wrote:
>>>> Julien,
>>>> unfortunately, I also encountered this problem for our project.
>>>> We have several nightly submissions each having approx. 15000  
>>>> tests.
>>>> I already changed the PHP variables to
>>>> max_execution_time = 1500;
>>>> max_input_time = 1500;
>>>> memory_limit = 1000M;
>>>> With these settings the submission is sometimes parsed  
>>>> successfully. However,
>>>> it does not always work.
>>>> I looked at the code and I've seen that the full submission is  
>>>> stored in an array
>>>> which is afterwards written into the database. Would it be an  
>>>> option to use a SAX parser and
>>>> immediately write each submission into the database, i.e. without  
>>>> creating the big array but do
>>>> it in a streaming fashion? I think this could dramatically reduce  
>>>> the submission time and the
>>>> main memory usage.
>>>> Maybe our test submissions are a to big for the scenarios CDash  
>>>> was designed for.
>>>> What do you think?
>>>> Thanks in advance.
>>>> Matthias
>>>> On 07.10.2008, at 13:46, Julien Jomier wrote:
>>>>> Emmanuel,
>>>>>
>>>>> You should increase the memory requirement for PHP otherwise it  
>>>>> runs out of memory when trying to parse large submission:
>>>>>
>>>>> Try changing these variables in your php.ini file (and restart  
>>>>> the web server)
>>>>>
>>>>> max_execution_time = 30;
>>>>> max_input_time = 60;
>>>>> memory_limit = 200M;
>>>>>
>>>>> http://public.kitware.com/Wiki/CDash:FAQ#PHP_cannot_parse_my_submission
>>>>>
>>>>> Let us know if that was the problem,
>>>>> Julien
>>>>>
>>>>> Emmanuel Christophe wrote:
>>>>>> Hi,
>>>>>> I'm trying to set up a cdash dashboard. It seems to be working  
>>>>>> pretty
>>>>>> well for small submissions (*-Experimental_Test.xml about 1.5  
>>>>>> Mo of
>>>>>> size) as I can see
>>>>>> http://www.orfeo-toolbox.org/Dashboard/index.php?project=OTB&date=20081006
>>>>>> for the test done at 14:12:57 CEST. This test is done without the
>>>>>> DATA_ROOT set for the test data, resulting in many fails but  
>>>>>> shorter
>>>>>> xml file.
>>>>>> However, when I set properly the DATA_ROOT, the xml file is  
>>>>>> about 6 Mo
>>>>>> and something is wrong during the submission: as you can see on  
>>>>>> the
>>>>>> 14:33:20 CEST test, no test result appear.
>>>>>> I add a similar problem at the beginning, with some error in  
>>>>>> apache
>>>>>> log showing a lack of memory for php to parse the xml file and  
>>>>>> the
>>>>>> *-Experimental_Test.xml not appearing in the backup directory.  
>>>>>> After
>>>>>> increasing the memory allowed to php, the *- 
>>>>>> Experimental_Test.xml file
>>>>>> now does appear in the backup directory, there is no more error  
>>>>>> in the
>>>>>> apache log, but the test result do not display (even after  
>>>>>> rerunning
>>>>>> the tests).
>>>>>> My CTestConfig.cmake is :
>>>>>> SET(CTEST_PROJECT_NAME "OTB")
>>>>>> SET(CTEST_NIGHTLY_START_TIME "00:00:00 EST")
>>>>>> IF(NOT DEFINED CTEST_DROP_METHOD)
>>>>>> SET(CTEST_DROP_METHOD "http")
>>>>>> ENDIF(NOT DEFINED CTEST_DROP_METHOD)
>>>>>> IF(CTEST_DROP_METHOD STREQUAL "http")
>>>>>> SET(CTEST_DROP_SITE "www.orfeo-toolbox.org")
>>>>>> SET(CTEST_DROP_LOCATION "/Dashboard/submit.php?project=OTB")
>>>>>> SET(CTEST_TRIGGER_SITE "")
>>>>>> ENDIF(CTEST_DROP_METHOD STREQUAL "http")
>>>>>> I'm using CDash-1.0.2.zip, php5, apache2.
>>>>>> Any suggestions? Where should I look to find out the source of  
>>>>>> the problem?
>>>>>> Thanks,
>>>>>> Emmanuel
>>>>>> _______________________________________________
>>>>>> Cdash mailing list
>>>>>> Cdash at public.kitware.com
>>>>>> http://public.kitware.com/cgi-bin/mailman/listinfo/cdash
>>>>> _______________________________________________
>>>>> Cdash mailing list
>>>>> Cdash at public.kitware.com
>>>>> http://public.kitware.com/cgi-bin/mailman/listinfo/cdash