[Cdash] CDash submission problem

[Matthias Brantner]

Julien,

unfortunately, I also encountered this problem for our project.
We have several nightly submissions each having approx. 15000 tests.
I already changed the PHP variables to

max_execution_time = 1500;
max_input_time = 1500;
memory_limit = 1000M;

With these settings the submission is sometimes parsed successfully.  
However,
it does not always work.

I looked at the code and I've seen that the full submission is stored  
in an array
which is afterwards written into the database. Would it be an option  
to use a SAX parser and
immediately write each submission into the database, i.e. without  
creating the big array but do
it in a streaming fashion? I think this could dramatically reduce the  
submission time and the
main memory usage.

Maybe our test submissions are a to big for the scenarios CDash was  
designed for.
What do you think?

Thanks in advance.

Matthias


On 07.10.2008, at 13:46, Julien Jomier wrote:

> Emmanuel,
>
> You should increase the memory requirement for PHP otherwise it runs  
> out of memory when trying to parse large submission:
>
> Try changing these variables in your php.ini file (and restart the  
> web server)
>
> max_execution_time = 30;
> max_input_time = 60;
> memory_limit = 200M;
>
> http://public.kitware.com/Wiki/ 
> CDash:FAQ#PHP_cannot_parse_my_submission
>
> Let us know if that was the problem,
> Julien
>
> Emmanuel Christophe wrote:
>> Hi,
>> I'm trying to set up a cdash dashboard. It seems to be working pretty
>> well for small submissions (*-Experimental_Test.xml about 1.5 Mo of
>> size) as I can see
>> http://www.orfeo-toolbox.org/Dashboard/index.php?project=OTB&date=20081006
>> for the test done at 14:12:57 CEST. This test is done without the
>> DATA_ROOT set for the test data, resulting in many fails but shorter
>> xml file.
>> However, when I set properly the DATA_ROOT, the xml file is about 6  
>> Mo
>> and something is wrong during the submission: as you can see on the
>> 14:33:20 CEST test, no test result appear.
>> I add a similar problem at the beginning, with some error in apache
>> log showing a lack of memory for php to parse the xml file and the
>> *-Experimental_Test.xml not appearing in the backup directory. After
>> increasing the memory allowed to php, the *-Experimental_Test.xml  
>> file
>> now does appear in the backup directory, there is no more error in  
>> the
>> apache log, but the test result do not display (even after rerunning
>> the tests).
>> My CTestConfig.cmake is :
>> SET(CTEST_PROJECT_NAME "OTB")
>> SET(CTEST_NIGHTLY_START_TIME "00:00:00 EST")
>> IF(NOT DEFINED CTEST_DROP_METHOD)
>>  SET(CTEST_DROP_METHOD "http")
>> ENDIF(NOT DEFINED CTEST_DROP_METHOD)
>> IF(CTEST_DROP_METHOD STREQUAL "http")
>>  SET(CTEST_DROP_SITE "www.orfeo-toolbox.org")
>>  SET(CTEST_DROP_LOCATION "/Dashboard/submit.php?project=OTB")
>>  SET(CTEST_TRIGGER_SITE "")
>> ENDIF(CTEST_DROP_METHOD STREQUAL "http")
>> I'm using CDash-1.0.2.zip, php5, apache2.
>> Any suggestions? Where should I look to find out the source of the  
>> problem?
>> Thanks,
>> Emmanuel
>> _______________________________________________
>> Cdash mailing list
>> Cdash at public.kitware.com
>> http://public.kitware.com/cgi-bin/mailman/listinfo/cdash
> _______________________________________________
> Cdash mailing list
> Cdash at public.kitware.com
> http://public.kitware.com/cgi-bin/mailman/listinfo/cdash