From bukuli.norbert at gmail.com Mon Apr 22 12:16:13 2024 From: bukuli.norbert at gmail.com (Bukuli Norbert) Date: Mon, 22 Apr 2024 14:16:13 +0200 Subject: [CDash] testing_junit XML handler questions Message-ID: Hi All, TL;DR: I would like to monitor the passing, failing and skipped test cases of our project in CDash, not just the overall result of each test executable. Is `testing_junit` XML handler the right tool for that, if my test executables are able to emit junit XML files? A bit more detailed explanation: Firstly, the following explanations are targeting cdash-3.2.3. I am stuck with the installation process of cdash-3.3.0. If there are developments in this area, please let me know. The `testing_junit` XML handler ( https://github.com/Kitware/CDash/blob/master/app/cdash/xml_handlers/testing_junit_handler.php) seems to address my needs, but it is not clearly documented, whether it is the right tool or not. I have managed to figure out from the CDash and ctest source code, that `ctest_submit(CDASH_UPLOAD "test_output_junit.xml" CDASH_UPLOAD_TYPE "testing_junit")` might do the right thing, but the CDash side unfortunately throws some exceptions during the process of the submission. After tweaking slightly the code of CDash, the processing of XML files ran into this area: https://github.com/Kitware/CDash/blob/master/app/cdash/xml_handlers/testing_junit_handler.php#L73-L97 what made me wonder: * Is that the right tool to monitor test cases through the junit XML files emitted by gtest, or... * Is that to process junit output generated by ctest * If the first one, what is the XML schema that could be processed * Can I (or ctest) easily inject site related information into the junit XMLs generated by gtest, because it is required during the process. Or... * Shall I encapsulate the junit XML-s generated by gtest into another XML similarly to Update.xml, Build.xml, Test.xml (generated by ctest), and so on * Is the testing_junit_handler.php maintained? * Can I do the per test case monitoring with CDash? (gtest_discover_tests is not an option here, because it slows down the testing process significantly (around 5 times as slow) Any guidance is highly appreciated. Thank you in advance! Best regards Norbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From zack.galbreath at kitware.com Thu Apr 25 15:50:02 2024 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Thu, 25 Apr 2024 11:50:02 -0400 Subject: [CDash] testing_junit XML handler questions In-Reply-To: References: Message-ID: Thanks for reaching out. You can submit JUnit XML files like so: ctest_submit(FILES "/path/to/junit.xml") That being said, the JUnit XML format does not offer any additional information that CTest does not provide in its own Test.xml format. In other words, both formats should be useful for tracking the behavior of tests over time. On Mon, Apr 22, 2024 at 8:16?AM Bukuli Norbert wrote: > Hi All, > > TL;DR: > I would like to monitor the passing, failing and skipped test cases of our > project in CDash, not just the overall result of each test executable. Is > `testing_junit` XML handler the right tool for that, if my test executables > are able to emit junit XML files? > > A bit more detailed explanation: > Firstly, the following explanations are targeting cdash-3.2.3. I am stuck > with the installation process of cdash-3.3.0. If there are developments in > this area, please let me know. > > The `testing_junit` XML handler ( > https://github.com/Kitware/CDash/blob/master/app/cdash/xml_handlers/testing_junit_handler.php) > seems to address my needs, but it is not clearly documented, whether it is > the right tool or not. I have managed to figure out from the CDash and > ctest source code, that `ctest_submit(CDASH_UPLOAD "test_output_junit.xml" > CDASH_UPLOAD_TYPE "testing_junit")` might do the right thing, but the > CDash side unfortunately throws some exceptions during the process of the > submission. > > After tweaking slightly the code of CDash, the processing of XML files ran > into this area: > https://github.com/Kitware/CDash/blob/master/app/cdash/xml_handlers/testing_junit_handler.php#L73-L97 > what made me wonder: > * Is that the right tool to monitor test cases through the junit XML files > emitted by gtest, or... > * Is that to process junit output generated by ctest > * If the first one, what is the XML schema that could be processed > * Can I (or ctest) easily inject site related information into the junit > XMLs generated by gtest, because it is required during the process. Or... > * Shall I encapsulate the junit XML-s generated by gtest into another XML > similarly to Update.xml, Build.xml, Test.xml (generated by ctest), and so on > * Is the testing_junit_handler.php maintained? > * Can I do the per test case monitoring with CDash? (gtest_discover_tests > is not an option here, because it slows down the testing > process significantly (around 5 times as slow) > > Any guidance is highly appreciated. Thank you in advance! > > Best regards > Norbert > _______________________________________________ > CDash mailing list > CDash at public.kitware.com > https://public.kitware.com/cgi-bin/mailman/listinfo/cdash > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Chad.Schrock at ll.mit.edu Fri Apr 26 12:22:56 2024 From: Chad.Schrock at ll.mit.edu (Schrock, Chad - 0336 - MITLL) Date: Fri, 26 Apr 2024 12:22:56 +0000 Subject: [CDash] CDash 3.3.0 GitLab OAuth Message-ID: Hi - I've just upgraded my CDash 3.1.0 instance to 3.3.0 and everything appears to be in order, except we can no longer use GitLab as an authentication source. GitLab comes back with an error: "The redirect URI included is not valid" We have been using https:// /oauth/callback/gitlab from our earlier installs. I saw in the release notes that the GitLab login has been converted to Socialite; did that change the Callback URL? (A few greps through the source didn't turn up anything useful, but my coffee hasn't totally kicked in yet.) Thanks, Chad -- Chad Schrock, he/him Supporting MIT Lincoln Laboratory, Lexington, MA chad.schrock at ll.mit.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6961 bytes Desc: not available URL: From joe.snyder at kitware.com Fri Apr 26 18:48:25 2024 From: joe.snyder at kitware.com (Joe Snyder) Date: Fri, 26 Apr 2024 14:48:25 -0400 Subject: [CDash] CDash 3.3.0 GitLab OAuth In-Reply-To: References: Message-ID: Hi Chad, Thanks for reaching out. We should have set up the callback URL so that both the "old" style ("/oauth/callback/gitlab" ) and the new one ("/auth/gitlab/callback") would work. Did you set a callback URL in the settings for the GitLab authentication? We have a discussion about this here: https://github.com/Kitware/CDash/discussions/2032. I've also generated an issue so that I don't continue to forget to update the documentation for setting up these OAuth providers: https://github.com/Kitware/CDash/issues/2150 Thanks again for the report. - Joe Joseph Snyder Senior R&D Engineer Kitware, Inc. 1712 Route 9, Suite 300 Clifton Park, NY 12065 (518) 881-4409 On Fri, Apr 26, 2024 at 2:36?PM Schrock, Chad - 0336 - MITLL < Chad.Schrock at ll.mit.edu> wrote: > > > Hi ? > > > > I?ve just upgraded my CDash 3.1.0 instance to 3.3.0 and everything appears > to be in order, except we can no longer use GitLab as an authentication > source. > > > > GitLab comes back with an error: ?The redirect URI included is not valid? > > > > We have been using https:///oauth/callback/gitlab from our > earlier installs. > > > > I saw in the release notes that the GitLab login has been converted to > Socialite; did that change the Callback URL? (A few greps through the > source didn?t turn up anything useful, but my coffee hasn?t totally kicked > in yet.) > > > > > > Thanks, > > Chad > > > > -- > > Chad Schrock, he/him > > Supporting MIT Lincoln Laboratory, Lexington, MA > > chad.schrock at ll.mit.edu > > > _______________________________________________ > CDash mailing list > CDash at public.kitware.com > https://public.kitware.com/cgi-bin/mailman/listinfo/cdash > -------------- next part -------------- An HTML attachment was scrubbed... URL: