[CDash] testing_junit XML handler questions

[Zack Galbreath]

Thanks for reaching out. You can submit JUnit XML files like so:

ctest_submit(FILES "/path/to/junit.xml")

That being said, the JUnit XML format does not offer any additional
information that CTest does not provide in its own Test.xml format. In
other words, both formats should be useful for tracking the behavior of
tests over time.




On Mon, Apr 22, 2024 at 8:16 AM Bukuli Norbert <bukuli.norbert at gmail.com>
wrote:

> Hi All,
>
> TL;DR:
> I would like to monitor the passing, failing and skipped test cases of our
> project in CDash, not just the overall result of each test executable. Is
> `testing_junit` XML handler the right tool for that, if my test executables
> are able to emit junit XML files?
>
> A bit more detailed explanation:
> Firstly, the following explanations are targeting cdash-3.2.3. I am stuck
> with the installation process of cdash-3.3.0. If there are developments in
> this area, please let me know.
>
> The `testing_junit` XML handler (
> https://github.com/Kitware/CDash/blob/master/app/cdash/xml_handlers/testing_junit_handler.php)
> seems to address my needs, but it is not clearly documented, whether it is
> the right tool or not. I have managed to figure out from the CDash and
> ctest source code, that `ctest_submit(CDASH_UPLOAD "test_output_junit.xml"
>   CDASH_UPLOAD_TYPE "testing_junit")` might do the right thing, but the
> CDash side unfortunately throws some exceptions during the process of the
> submission.
>
> After tweaking slightly the code of CDash, the processing of XML files ran
> into this area:
> https://github.com/Kitware/CDash/blob/master/app/cdash/xml_handlers/testing_junit_handler.php#L73-L97
> what made me wonder:
> * Is that the right tool to monitor test cases through the junit XML files
> emitted by gtest, or...
> * Is that to process junit output generated by ctest
> * If the first one, what is the XML schema that could be processed
> * Can I (or ctest) easily inject site related information into the junit
> XMLs generated by gtest, because it is required during the process. Or...
> * Shall I encapsulate the junit XML-s generated by gtest into another XML
> similarly to Update.xml, Build.xml, Test.xml (generated by ctest), and so on
> * Is the testing_junit_handler.php maintained?
> * Can I do the per test case monitoring with CDash? (gtest_discover_tests
> is not an option here, because it slows down the testing
> process significantly (around 5 times as slow)
>
> Any guidance is highly appreciated. Thank you in advance!
>
> Best regards
> Norbert
> _______________________________________________
> CDash mailing list
> CDash at public.kitware.com
> https://public.kitware.com/cgi-bin/mailman/listinfo/cdash
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cdash/attachments/20240425/33484929/attachment.htm>