From zack.galbreath at kitware.com Fri Feb 1 16:32:25 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Fri, 1 Feb 2019 16:32:25 -0500 Subject: [CDash] [cmake-developers] [vtk-developers] CDash frontend refresh In-Reply-To: References: <8702064F-D659-48BF-9150-6A432DD911CC@kitware.com> Message-ID: Thanks for the feedback, everyone. I pushed a couple tweaks to testing.cdash.org. * The white numbers on index.php should be more legible now * reduced the padding of these tables slightly so more builds can fit on the screen at once * fixed the positioning of the calendar On Fri, Feb 1, 2019 at 8:40 AM Robert Maynard via cmake-developers < cmake-developers at cmake.org> wrote: > My feeling on this is that the green should be stronger color. A fully > green dashboard is a happy thing and it should make me the viewer > happy :) > This is a tough issue. I definitely want CDash to promote feelings of happiness and joy. But at the same time, CDash's main value is helping you figure out what's going wrong and fixing it. This is the motivation behind making warnings and errors more prominent in a sea of clean builds. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zack.galbreath at kitware.com Fri Feb 1 17:14:10 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Fri, 1 Feb 2019 17:14:10 -0500 Subject: [CDash] [cmake-developers] [vtk-developers] CDash frontend refresh In-Reply-To: References: <8702064F-D659-48BF-9150-6A432DD911CC@kitware.com> Message-ID: On Fri, Feb 1, 2019 at 5:00 PM Craig Scott wrote: > FWIW, my reaction was pretty similar to Robert's. A more greenish hue for > the passing fields would make them more subconsciously "these are okay" in > the reader's mind. The yellowish hue at the moment doesn't seem to evoke > that same subconscious response (to me at least), so I'm actually more > distracted by the new color than it helping me subconsciously filter out > those fields. > > The icon for the view settings (see attachment) also seems to have > changed. Took me a while to work out what it was and with no tooltip, I had > to click it to find out. Had I not used the earlier version, I may have > just assumed it was a broken image link and not realised it was > clickable/useful. > > [image: CDashRefresh_viewSettings.png] > Thanks for the useful feedback, Craig. I'll experiment with darker (yet still subdued) shades of green for passing builds. I'll also add a tooltip for the view settings icon. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: CDashRefresh_viewSettings.png Type: image/png Size: 19445 bytes Desc: not available URL: From Pavlech.Juraj at azd.cz Tue Feb 19 03:57:01 2019 From: Pavlech.Juraj at azd.cz (Pavlech Juraj) Date: Tue, 19 Feb 2019 08:57:01 +0000 Subject: [CDash] GetFeed Message-ID: <1550566621030.77202@azd.cz> Hello, After creating a new CDash project on localhost, I can not open this project and the CDash returns an error: Feed::GetFeed(): 0 not set Warning: Invalid argument supplied for foreach() in CDash/public/ajax/getfeed.php on line 99 Warning: count(): Parameter must be an array or an object that implements Countable in /CDash/public/ajax/getfeed.php on line 116 Thanks in advance for your help. -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.morales.vega at gmail.com Wed Feb 20 12:27:32 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Wed, 20 Feb 2019 17:27:32 +0000 Subject: [CDash] GitHub Status API support Message-ID: Hi, Currently when I push to GitHub it notifies my Jenkins instance. I want Jenkins to run a ctest script, which will include set(CTEST_UPDATE_VERSION_ONLY ON) ctest_update() And submit the results to CDash. ..and I would like CDash to tell GitHub about what it finds using its Status API (https://developer.github.com/v3/repos/statuses/). There is a simple way to do this? I know about https://cmake.org/cmake/help/latest/variable/CTEST_CHANGE_ID.html, but I have been unable to find too much documentation about it. It let's me "pass arbitrary information" which could for example be used "to post comments on your pull request". How "arbitrary" becomes "pull request ID", and whether I can make it mean "git commit" is unclear. Regards. From jcfr at kitware.com Wed Feb 20 12:54:16 2019 From: jcfr at kitware.com (Jean-Christophe Fillion-Robin) Date: Wed, 20 Feb 2019 12:54:16 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: Hi Cristian, You could great an app using ProBot (see https://github.com/probot/probot) Here is such an app created by Matt M for ITK: https://github.com/InsightSoftwareConsortium/ITKGitHubCDashStatus Hth Jc On Wed, Feb 20, 2019 at 12:27 PM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > Hi, > > Currently when I push to GitHub it notifies my Jenkins instance. I > want Jenkins to run a ctest script, which will include > > set(CTEST_UPDATE_VERSION_ONLY ON) > ctest_update() > > And submit the results to CDash. > ..and I would like CDash to tell GitHub about what it finds using its > Status API (https://developer.github.com/v3/repos/statuses/). > > There is a simple way to do this? > > I know about > https://cmake.org/cmake/help/latest/variable/CTEST_CHANGE_ID.html, > but I have been unable to find too much documentation about it. It > let's me "pass arbitrary information" which could for example be used > "to post comments on your pull request". How "arbitrary" becomes "pull > request ID", and whether I can make it mean "git commit" is unclear. > > Regards. > _______________________________________________ > CDash mailing list > CDash at public.kitware.com > https://public.kitware.com/mailman/listinfo/cdash > -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.morales.vega at gmail.com Wed Feb 20 13:59:51 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Wed, 20 Feb 2019 18:59:51 +0000 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Wed, 20 Feb 2019 at 17:27, Cristian Morales Vega wrote: > I know about https://cmake.org/cmake/help/latest/variable/CTEST_CHANGE_ID.html, > but I have been unable to find too much documentation about it. It > let's me "pass arbitrary information" which could for example be used > "to post comments on your pull request". How "arbitrary" becomes "pull > request ID", and whether I can make it mean "git commit" is unclear. I have just found this: https://public.kitware.com/pipermail/cdash/2018-January/001832.html Zack, what are your plans on this? Maybe it's already implemented but not documented? From zack.galbreath at kitware.com Wed Feb 20 14:42:17 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Wed, 20 Feb 2019 14:42:17 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Wed, Feb 20, 2019 at 2:00 PM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > Zack, what are your plans on this? Maybe it's already implemented but > not documented? > We are really, really close :) First up, here's a blog post describing CDash's current integration with GitHub: https://blog.kitware.com/cdash-integration-with-github/ This works by commenting on your pull request whenever something goes wrong (build error, test failure, etc). But as you indicated, rather than commenting on a PR, it would be better to have CDash post a status to the commit on GitHub with a link back to itself. The biggest barrier to this goal was the fact that CDash didn't actually ever know when a build was done. So we could only ever post bad news (this is what went wrong), not good news (everything passed, hurray). There was no mechanism in place for CTest to say "Okay CDash, that's the last file I'm going to send you for this build". CTest and CDash finally gained this capability as of CMake v3.14 (currently in the release candidate cycle). So now that CDash can actually know when a build is done, we need to teach CDash to use GitHub's status API. This is implemented in CDash PR #748 . I've already reviewed these changes and they look good to me. I still need to find the time to test it out & make sure the GitHub status gets updated as expected. Once I do that I'll merge this change into CDash, and our goal of linking back to CDash from the commit status on GitHub will finally be achieved. -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.morales.vega at gmail.com Wed Feb 20 14:54:40 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Wed, 20 Feb 2019 19:54:40 +0000 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Wed, 20 Feb 2019 at 19:42, Zack Galbreath wrote: > This works by commenting on your pull request whenever something goes wrong (build error, test failure, etc). But as you indicated, rather than commenting on a PR, it would be better to have CDash post a status to the commit on GitHub with a link back to itself. The biggest barrier to this goal was the fact that CDash didn't actually ever know when a build was done. So we could only ever post bad news (this is what went wrong), not good news (everything passed, hurray). There was no mechanism in place for CTest to say "Okay CDash, that's the last file I'm going to send you for this build". CTest and CDash finally gained this capability as of CMake v3.14 (currently in the release candidate cycle). > > So now that CDash can actually know when a build is done, we need to teach CDash to use GitHub's status API. This is implemented in CDash PR #748. I've already reviewed these changes and they look good to me. I still need to find the time to test it out & make sure the GitHub status gets updated as expected. Once I do that I'll merge this change into CDash, and our goal of linking back to CDash from the commit status on GitHub will finally be achieved. That's great! I had not though about the lack of "done". For the time being... my Jenkins actually does know when it has finished. If I could retrieve the CDash buildid when doing ctest_submit() I could store/show it somewhere. There is any way to do this? From zack.galbreath at kitware.com Wed Feb 20 16:19:58 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Wed, 20 Feb 2019 16:19:58 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Wed, Feb 20, 2019 at 2:54 PM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > For the time being... my Jenkins actually does know when it has > finished. If I could retrieve the CDash buildid when doing > ctest_submit() I could store/show it somewhere. There is any way to do > this? > As of CMake 3.13, yes. CTest now provides enough information for CDash to initialize a buildid at submit time (rather than parse time). CDash includes this buildid in its XML response. It looks like the only way to see this from the command-line currently is to call ctest with the --debug argument. Here's an example: OK 56 663d1dbb340ef6166058d94b990107e2 There's a potential complication here if the CDash instance you're submitting to is configured for asynchronous submissions (as most of our public CDash instances are). The concept here is that CDash can get busy at certain times of day. So instead of asking the testing clients to wait while CDash parses their XML files, the submissions get added to a queue that CDash works on offline. So just because your build has completed in Jenkins doesn't mean that CDash has parsed its results yet. -------------- next part -------------- An HTML attachment was scrubbed... URL: From DLRdave at aol.com Wed Feb 20 19:30:20 2019 From: DLRdave at aol.com (David Cole) Date: Wed, 20 Feb 2019 19:30:20 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: Surely there's a way to query for the build id given the components that go into making up the unique key (build stamp, build type, site, .... I can't remember all the components required to get the unique key....?) Is there a way to send that up to CDash and get a build id response back? (Because those things should be well-known by the dashboard submitter.) On Wed, Feb 20, 2019 at 4:19 PM Zack Galbreath via CDash wrote: > > On Wed, Feb 20, 2019 at 2:54 PM Cristian Morales Vega wrote: >> >> For the time being... my Jenkins actually does know when it has >> finished. If I could retrieve the CDash buildid when doing >> ctest_submit() I could store/show it somewhere. There is any way to do >> this? > > > As of CMake 3.13, yes. CTest now provides enough information for CDash to initialize a buildid at submit time (rather than parse time). CDash includes this buildid in its XML response It looks like the only way to see this from the command-line currently is to call ctest with the --debug argument. Here's an example: > > > OK > > 56 > 663d1dbb340ef6166058d94b990107e2 > > > There's a potential complication here if the CDash instance you're submitting to is configured for asynchronous submissions (as most of our public CDash instances are). The concept here is that CDash can get busy at certain times of day. So instead of asking the testing clients to wait while CDash parses their XML files, the submissions get added to a queue that CDash works on offline. So just because your build has completed in Jenkins doesn't mean that CDash has parsed its results yet. > _______________________________________________ > CDash mailing list > CDash at public.kitware.com > https://public.kitware.com/mailman/listinfo/cdash From christian.morales.vega at gmail.com Thu Feb 21 06:00:28 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Thu, 21 Feb 2019 11:00:28 +0000 Subject: [CDash] Random questions Message-ID: Hi, * What's a "track"? https://cmake.org/cmake/help/latest/command/ctest_start.html#command:ctest_start "If TRACK is used, the submissions will go to the specified track on the CDash server." Does it refer to "Build Group"? The only thing linking both I have found is this small note in the Administrator's guide (https://public.kitware.com/Wiki/CDash:Administration#Build_Groups): "(also called tracks in Dart2)". * What are the CDash Append semantics? Is it configurable? https://cmake.org/cmake/help/latest/command/ctest_configure.html#command:ctest_configure and others says: "Append semantics are defined by the dashboard server in use". Fair enough, but I have not found any mention to append semantics in the dashboard server documentation. When the "append" feature was created, what was its intended use? * What are "Dynamic BuildGroups" I have only seen them referenced in http://mycdash.com/manageBuildGroup.php?projectid=1#!#dynamic From christian.morales.vega at gmail.com Thu Feb 21 07:14:43 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Thu, 21 Feb 2019 12:14:43 +0000 Subject: [CDash] Scripting project creation Message-ID: We have lots of small independent projects instead of a few big ones (in general the small ones are not really subprojects of others). So I would like to script away the creation of projects in CDash. Can this be done? In https://public.kitware.com/Wiki/CDash:API#List I see how to list projects, but no how to create them. From zack.galbreath at kitware.com Thu Feb 21 08:03:30 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Thu, 21 Feb 2019 08:03:30 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Wed, Feb 20, 2019 at 7:30 PM David Cole wrote: > Surely there's a way to query for the build id given the components > that go into making up the unique key (build stamp, build type, site, > .... I can't remember all the components required to get the unique > key....?) > > Is there a way to send that up to CDash and get a build id response > back? (Because those things should be well-known by the dashboard > submitter.) > Yes there is. https://github.com/Kitware/CDash/blob/master/public/api/v1/addBuild.php Provided the minimum details to define a build (site, name, stamp, project) this endpoint either returns an existing buildid (200) or creates a new one for you (201). -------------- next part -------------- An HTML attachment was scrubbed... URL: From zack.galbreath at kitware.com Thu Feb 21 15:26:47 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Thu, 21 Feb 2019 15:26:47 -0500 Subject: [CDash] Random questions In-Reply-To: References: Message-ID: On Thu, Feb 21, 2019 at 6:00 AM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > Hi, > > * What's a "track"? Does it refer to "Build Group"? > Yes, you're right. What CTest refers to as a "track" is a Build Group in CDash. * What are the CDash Append semantics? Is it configurable? > I'm not sure. I would have to do some manual experiments to see what effect (if any) the Append flag has on results submitted to CDash. * What are "Dynamic BuildGroups" > I have only seen them referenced in > http://mycdash.com/manageBuildGroup.php?projectid=1#!#dynamic You can use dynamic build groups to show the most recent results regardless of when they occurred. https://blog.kitware.com/dynamic-buildgroups-in-cdash/ An example of this is the "latest master" group on VTK's dashboard: https://open.cdash.org/index.php?project=VTK It shows the most recent results of building master across a variety of configurations. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zack.galbreath at kitware.com Thu Feb 21 15:39:30 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Thu, 21 Feb 2019 15:39:30 -0500 Subject: [CDash] Scripting project creation In-Reply-To: References: Message-ID: On Thu, Feb 21, 2019 at 7:15 AM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > We have lots of small independent projects instead of a few big ones > (in general the small ones are not really subprojects of others). So I > would like to script away the creation of projects in CDash. Can this > be done? > > In https://public.kitware.com/Wiki/CDash:API#List I see how to list > projects, but no how to create them. > Yes, this should be doable. When you click the "Create Project" button, CDash sends a POST request to /api/v1/project.php. An example of the JSON payload for this request is attached. In order to use this API endpoint you'll need to provide a bearer API token from a CDash user that has permission to create projects. You can create such a token on user.php after logging in. Include it in the header of your request like so: "Authorization: Bearer " -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: example.json Type: application/json Size: 700 bytes Desc: not available URL: From christian.morales.vega at gmail.com Fri Feb 22 04:54:54 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Fri, 22 Feb 2019 09:54:54 +0000 Subject: [CDash] Random questions In-Reply-To: References: Message-ID: On Thu, 21 Feb 2019 at 20:26, Zack Galbreath wrote: >> * What are the CDash Append semantics? Is it configurable? > > > I'm not sure. I would have to do some manual experiments to see what effect (if any) the Append flag has on results submitted to CDash. I'm guessing that puts it in the "nobody uses it -> is not really too tested -> avoid it" category? >> * What are "Dynamic BuildGroups" >> I have only seen them referenced in >> http://mycdash.com/manageBuildGroup.php?projectid=1#!#dynamic > > > You can use dynamic build groups to show the most recent results regardless of when they occurred. > > https://blog.kitware.com/dynamic-buildgroups-in-cdash/ > > An example of this is the "latest master" group on VTK's dashboard: > https://open.cdash.org/index.php?project=VTK > It shows the most recent results of building master across a variety of configurations. Thanks. FWIW I have already understood that half of the best documentation of CDash is in blog posts. And have read everything in https://blog.kitware.com/tag/cdash/. That specific post lacks the CDash tag! Probably I should have looked at https://blog.kitware.com/author/zack-galbreath/, though :-p I would look at https://blog.kitware.com/tag/software-process, but it's polluted with too many cmake X.Y.Z released/ready for testing posts. From christian.morales.vega at gmail.com Fri Feb 22 08:35:32 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Fri, 22 Feb 2019 13:35:32 +0000 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Thu, 21 Feb 2019 at 13:03, Zack Galbreath wrote: > Yes there is. > > https://github.com/Kitware/CDash/blob/master/public/api/v1/addBuild.php > > Provided the minimum details to define a build (site, name, stamp, project) this endpoint either returns an existing buildid (200) or creates a new one for you (201). Probably it would be useful to make ctest_submit() able to return this buildid? I can imagine this being call from Jenkins and wanting to store a reference to this buildid in the Jenkins logs. From zack.galbreath at kitware.com Fri Feb 22 09:15:28 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Fri, 22 Feb 2019 09:15:28 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Fri, Feb 22, 2019 at 8:35 AM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > Probably it would be useful to make ctest_submit() able to return this > buildid? > I can imagine this being call from Jenkins and wanting to store a > reference to this buildid in the Jenkins logs. > Good idea, I can definitely see that being a useful improvement to ctest_submit() -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.morales.vega at gmail.com Fri Feb 22 09:29:40 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Fri, 22 Feb 2019 14:29:40 +0000 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Fri, 22 Feb 2019 at 14:15, Zack Galbreath wrote: > > On Fri, Feb 22, 2019 at 8:35 AM Cristian Morales Vega wrote: >> >> Probably it would be useful to make ctest_submit() able to return this buildid? >> I can imagine this being call from Jenkins and wanting to store a >> reference to this buildid in the Jenkins logs. > > > Good idea, I can definitely see that being a useful improvement to ctest_submit() So here it is -> https://gitlab.kitware.com/cmake/cmake/issues/18968 From christian.morales.vega at gmail.com Fri Feb 22 12:02:49 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Fri, 22 Feb 2019 17:02:49 +0000 Subject: [CDash] Coverage reports Message-ID: Hi, I'm finding the coverage report support confusing, mainly the default "Directories" view. Not sure if I'm finding bugs or if I just don't get the view intention. So in my http://cdash.samknows.com/viewCoverage.php?buildid= view I can see the coverage of my "include//" directory is 0%. 2 out of 2 lines inside are not covered. The "status" is "Zero" and "2/2" under "Lines not covered" is in green. Inside that directory there are actually two files, but since the other one has 100% line coverage it seems to get filtered out by the "Directories" view. This is specially confusing when, from the "Directories" view, you click on the directory and you start seeing both files. I'm missing something? Regards. From christian.morales.vega at gmail.com Mon Feb 25 12:53:47 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Mon, 25 Feb 2019 17:53:47 +0000 Subject: [CDash] Branch Coverage with GCov Message-ID: Hi, I have tried to reproduce https://blog.kitware.com/additional-coverage-features-in-cdash/ but I have had no luck. The gcov.tar file is being created. I guess uploaded, but it's not 100% clear from the ctest output. In CDash I can't see coverage information at all. And I'm wondering why the script uses ctest_coverage_collect_gcov() instead of ctest_coverage(). Before trying to reproduce with branch_coverage_example.tar.gz I was trying to do it in my project by using ctest_coverage and basically just trying to add -b or -c to CTEST_COVERAGE_EXTRA_FLAGS. There I got the coverage info, as always... but with no extra branch information I could see. Regression? There is something obvious I can't see? From christian.morales.vega at gmail.com Tue Feb 26 11:07:28 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Tue, 26 Feb 2019 16:07:28 +0000 Subject: [CDash] Submitted "files" presentation Message-ID: ctest_submit() let's you submit PARTS, but also FILES. I have submitted a file with the output of "llvm-cov" with Source-based Code Coverage. I know CDash only supports gcov coverage right now, I don't expect to get full support for it. But I was hoping to be able to access the submitted file somehow from the CDash web view, linked somehow to the build. Maybe some "Extra files" section or something. I can't see the file at all. I know it was "submitted" to CDash... and that's it. No idea if it was actually stored somewhere or anything. What's the purpose of ctest_submit(FILES) and, does CDash has any support for it? From christian.morales.vega at gmail.com Tue Feb 26 11:57:44 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Tue, 26 Feb 2019 16:57:44 +0000 Subject: [CDash] Submitted "files" presentation In-Reply-To: References: Message-ID: On Tue, 26 Feb 2019 at 16:07, Cristian Morales Vega wrote: > ctest_submit() let's you submit PARTS, but also FILES. I have > submitted a file with the output of "llvm-cov" with Source-based Code > Coverage. I know CDash only supports gcov coverage right now, I don't > expect to get full support for it. But I was hoping to be able to > access the submitted file somehow from the CDash web view, linked > somehow to the build. Maybe some "Extra files" section or something. I have actually managed to get what I wanted with ctest_upload(). Not really sure what's supposed to be the difference, though. From zack.galbreath at kitware.com Fri Feb 1 21:32:25 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Fri, 1 Feb 2019 16:32:25 -0500 Subject: [CDash] [cmake-developers] [vtk-developers] CDash frontend refresh In-Reply-To: References: <8702064F-D659-48BF-9150-6A432DD911CC@kitware.com> Message-ID: Thanks for the feedback, everyone. I pushed a couple tweaks to testing.cdash.org. * The white numbers on index.php should be more legible now * reduced the padding of these tables slightly so more builds can fit on the screen at once * fixed the positioning of the calendar On Fri, Feb 1, 2019 at 8:40 AM Robert Maynard via cmake-developers < cmake-developers at cmake.org> wrote: > My feeling on this is that the green should be stronger color. A fully > green dashboard is a happy thing and it should make me the viewer > happy :) > This is a tough issue. I definitely want CDash to promote feelings of happiness and joy. But at the same time, CDash's main value is helping you figure out what's going wrong and fixing it. This is the motivation behind making warnings and errors more prominent in a sea of clean builds. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zack.galbreath at kitware.com Fri Feb 1 22:14:10 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Fri, 1 Feb 2019 17:14:10 -0500 Subject: [CDash] [cmake-developers] [vtk-developers] CDash frontend refresh In-Reply-To: References: <8702064F-D659-48BF-9150-6A432DD911CC@kitware.com> Message-ID: On Fri, Feb 1, 2019 at 5:00 PM Craig Scott wrote: > FWIW, my reaction was pretty similar to Robert's. A more greenish hue for > the passing fields would make them more subconsciously "these are okay" in > the reader's mind. The yellowish hue at the moment doesn't seem to evoke > that same subconscious response (to me at least), so I'm actually more > distracted by the new color than it helping me subconsciously filter out > those fields. > > The icon for the view settings (see attachment) also seems to have > changed. Took me a while to work out what it was and with no tooltip, I had > to click it to find out. Had I not used the earlier version, I may have > just assumed it was a broken image link and not realised it was > clickable/useful. > > [image: CDashRefresh_viewSettings.png] > Thanks for the useful feedback, Craig. I'll experiment with darker (yet still subdued) shades of green for passing builds. I'll also add a tooltip for the view settings icon. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: CDashRefresh_viewSettings.png Type: image/png Size: 19445 bytes Desc: not available URL: From Pavlech.Juraj at azd.cz Tue Feb 19 08:57:01 2019 From: Pavlech.Juraj at azd.cz (Pavlech Juraj) Date: Tue, 19 Feb 2019 08:57:01 +0000 Subject: [CDash] GetFeed Message-ID: <1550566621030.77202@azd.cz> Hello, After creating a new CDash project on localhost, I can not open this project and the CDash returns an error: Feed::GetFeed(): 0 not set Warning: Invalid argument supplied for foreach() in CDash/public/ajax/getfeed.php on line 99 Warning: count(): Parameter must be an array or an object that implements Countable in /CDash/public/ajax/getfeed.php on line 116 Thanks in advance for your help. -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.morales.vega at gmail.com Wed Feb 20 17:27:32 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Wed, 20 Feb 2019 17:27:32 +0000 Subject: [CDash] GitHub Status API support Message-ID: Hi, Currently when I push to GitHub it notifies my Jenkins instance. I want Jenkins to run a ctest script, which will include set(CTEST_UPDATE_VERSION_ONLY ON) ctest_update() And submit the results to CDash. ..and I would like CDash to tell GitHub about what it finds using its Status API (https://developer.github.com/v3/repos/statuses/). There is a simple way to do this? I know about https://cmake.org/cmake/help/latest/variable/CTEST_CHANGE_ID.html, but I have been unable to find too much documentation about it. It let's me "pass arbitrary information" which could for example be used "to post comments on your pull request". How "arbitrary" becomes "pull request ID", and whether I can make it mean "git commit" is unclear. Regards. From jcfr at kitware.com Wed Feb 20 17:54:16 2019 From: jcfr at kitware.com (Jean-Christophe Fillion-Robin) Date: Wed, 20 Feb 2019 12:54:16 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: Hi Cristian, You could great an app using ProBot (see https://github.com/probot/probot) Here is such an app created by Matt M for ITK: https://github.com/InsightSoftwareConsortium/ITKGitHubCDashStatus Hth Jc On Wed, Feb 20, 2019 at 12:27 PM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > Hi, > > Currently when I push to GitHub it notifies my Jenkins instance. I > want Jenkins to run a ctest script, which will include > > set(CTEST_UPDATE_VERSION_ONLY ON) > ctest_update() > > And submit the results to CDash. > ..and I would like CDash to tell GitHub about what it finds using its > Status API (https://developer.github.com/v3/repos/statuses/). > > There is a simple way to do this? > > I know about > https://cmake.org/cmake/help/latest/variable/CTEST_CHANGE_ID.html, > but I have been unable to find too much documentation about it. It > let's me "pass arbitrary information" which could for example be used > "to post comments on your pull request". How "arbitrary" becomes "pull > request ID", and whether I can make it mean "git commit" is unclear. > > Regards. > _______________________________________________ > CDash mailing list > CDash at public.kitware.com > https://public.kitware.com/mailman/listinfo/cdash > -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.morales.vega at gmail.com Wed Feb 20 18:59:51 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Wed, 20 Feb 2019 18:59:51 +0000 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Wed, 20 Feb 2019 at 17:27, Cristian Morales Vega wrote: > I know about https://cmake.org/cmake/help/latest/variable/CTEST_CHANGE_ID.html, > but I have been unable to find too much documentation about it. It > let's me "pass arbitrary information" which could for example be used > "to post comments on your pull request". How "arbitrary" becomes "pull > request ID", and whether I can make it mean "git commit" is unclear. I have just found this: https://public.kitware.com/pipermail/cdash/2018-January/001832.html Zack, what are your plans on this? Maybe it's already implemented but not documented? From zack.galbreath at kitware.com Wed Feb 20 19:42:17 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Wed, 20 Feb 2019 14:42:17 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Wed, Feb 20, 2019 at 2:00 PM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > Zack, what are your plans on this? Maybe it's already implemented but > not documented? > We are really, really close :) First up, here's a blog post describing CDash's current integration with GitHub: https://blog.kitware.com/cdash-integration-with-github/ This works by commenting on your pull request whenever something goes wrong (build error, test failure, etc). But as you indicated, rather than commenting on a PR, it would be better to have CDash post a status to the commit on GitHub with a link back to itself. The biggest barrier to this goal was the fact that CDash didn't actually ever know when a build was done. So we could only ever post bad news (this is what went wrong), not good news (everything passed, hurray). There was no mechanism in place for CTest to say "Okay CDash, that's the last file I'm going to send you for this build". CTest and CDash finally gained this capability as of CMake v3.14 (currently in the release candidate cycle). So now that CDash can actually know when a build is done, we need to teach CDash to use GitHub's status API. This is implemented in CDash PR #748 . I've already reviewed these changes and they look good to me. I still need to find the time to test it out & make sure the GitHub status gets updated as expected. Once I do that I'll merge this change into CDash, and our goal of linking back to CDash from the commit status on GitHub will finally be achieved. -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.morales.vega at gmail.com Wed Feb 20 19:54:40 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Wed, 20 Feb 2019 19:54:40 +0000 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Wed, 20 Feb 2019 at 19:42, Zack Galbreath wrote: > This works by commenting on your pull request whenever something goes wrong (build error, test failure, etc). But as you indicated, rather than commenting on a PR, it would be better to have CDash post a status to the commit on GitHub with a link back to itself. The biggest barrier to this goal was the fact that CDash didn't actually ever know when a build was done. So we could only ever post bad news (this is what went wrong), not good news (everything passed, hurray). There was no mechanism in place for CTest to say "Okay CDash, that's the last file I'm going to send you for this build". CTest and CDash finally gained this capability as of CMake v3.14 (currently in the release candidate cycle). > > So now that CDash can actually know when a build is done, we need to teach CDash to use GitHub's status API. This is implemented in CDash PR #748. I've already reviewed these changes and they look good to me. I still need to find the time to test it out & make sure the GitHub status gets updated as expected. Once I do that I'll merge this change into CDash, and our goal of linking back to CDash from the commit status on GitHub will finally be achieved. That's great! I had not though about the lack of "done". For the time being... my Jenkins actually does know when it has finished. If I could retrieve the CDash buildid when doing ctest_submit() I could store/show it somewhere. There is any way to do this? From zack.galbreath at kitware.com Wed Feb 20 21:19:58 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Wed, 20 Feb 2019 16:19:58 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Wed, Feb 20, 2019 at 2:54 PM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > For the time being... my Jenkins actually does know when it has > finished. If I could retrieve the CDash buildid when doing > ctest_submit() I could store/show it somewhere. There is any way to do > this? > As of CMake 3.13, yes. CTest now provides enough information for CDash to initialize a buildid at submit time (rather than parse time). CDash includes this buildid in its XML response. It looks like the only way to see this from the command-line currently is to call ctest with the --debug argument. Here's an example: OK 56 663d1dbb340ef6166058d94b990107e2 There's a potential complication here if the CDash instance you're submitting to is configured for asynchronous submissions (as most of our public CDash instances are). The concept here is that CDash can get busy at certain times of day. So instead of asking the testing clients to wait while CDash parses their XML files, the submissions get added to a queue that CDash works on offline. So just because your build has completed in Jenkins doesn't mean that CDash has parsed its results yet. -------------- next part -------------- An HTML attachment was scrubbed... URL: From DLRdave at aol.com Thu Feb 21 00:30:20 2019 From: DLRdave at aol.com (David Cole) Date: Wed, 20 Feb 2019 19:30:20 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: Surely there's a way to query for the build id given the components that go into making up the unique key (build stamp, build type, site, .... I can't remember all the components required to get the unique key....?) Is there a way to send that up to CDash and get a build id response back? (Because those things should be well-known by the dashboard submitter.) On Wed, Feb 20, 2019 at 4:19 PM Zack Galbreath via CDash wrote: > > On Wed, Feb 20, 2019 at 2:54 PM Cristian Morales Vega wrote: >> >> For the time being... my Jenkins actually does know when it has >> finished. If I could retrieve the CDash buildid when doing >> ctest_submit() I could store/show it somewhere. There is any way to do >> this? > > > As of CMake 3.13, yes. CTest now provides enough information for CDash to initialize a buildid at submit time (rather than parse time). CDash includes this buildid in its XML response It looks like the only way to see this from the command-line currently is to call ctest with the --debug argument. Here's an example: > > > OK > > 56 > 663d1dbb340ef6166058d94b990107e2 > > > There's a potential complication here if the CDash instance you're submitting to is configured for asynchronous submissions (as most of our public CDash instances are). The concept here is that CDash can get busy at certain times of day. So instead of asking the testing clients to wait while CDash parses their XML files, the submissions get added to a queue that CDash works on offline. So just because your build has completed in Jenkins doesn't mean that CDash has parsed its results yet. > _______________________________________________ > CDash mailing list > CDash at public.kitware.com > https://public.kitware.com/mailman/listinfo/cdash From christian.morales.vega at gmail.com Thu Feb 21 11:00:28 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Thu, 21 Feb 2019 11:00:28 +0000 Subject: [CDash] Random questions Message-ID: Hi, * What's a "track"? https://cmake.org/cmake/help/latest/command/ctest_start.html#command:ctest_start "If TRACK is used, the submissions will go to the specified track on the CDash server." Does it refer to "Build Group"? The only thing linking both I have found is this small note in the Administrator's guide (https://public.kitware.com/Wiki/CDash:Administration#Build_Groups): "(also called tracks in Dart2)". * What are the CDash Append semantics? Is it configurable? https://cmake.org/cmake/help/latest/command/ctest_configure.html#command:ctest_configure and others says: "Append semantics are defined by the dashboard server in use". Fair enough, but I have not found any mention to append semantics in the dashboard server documentation. When the "append" feature was created, what was its intended use? * What are "Dynamic BuildGroups" I have only seen them referenced in http://mycdash.com/manageBuildGroup.php?projectid=1#!#dynamic From christian.morales.vega at gmail.com Thu Feb 21 12:14:43 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Thu, 21 Feb 2019 12:14:43 +0000 Subject: [CDash] Scripting project creation Message-ID: We have lots of small independent projects instead of a few big ones (in general the small ones are not really subprojects of others). So I would like to script away the creation of projects in CDash. Can this be done? In https://public.kitware.com/Wiki/CDash:API#List I see how to list projects, but no how to create them. From zack.galbreath at kitware.com Thu Feb 21 13:03:30 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Thu, 21 Feb 2019 08:03:30 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Wed, Feb 20, 2019 at 7:30 PM David Cole wrote: > Surely there's a way to query for the build id given the components > that go into making up the unique key (build stamp, build type, site, > .... I can't remember all the components required to get the unique > key....?) > > Is there a way to send that up to CDash and get a build id response > back? (Because those things should be well-known by the dashboard > submitter.) > Yes there is. https://github.com/Kitware/CDash/blob/master/public/api/v1/addBuild.php Provided the minimum details to define a build (site, name, stamp, project) this endpoint either returns an existing buildid (200) or creates a new one for you (201). -------------- next part -------------- An HTML attachment was scrubbed... URL: From zack.galbreath at kitware.com Thu Feb 21 20:26:47 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Thu, 21 Feb 2019 15:26:47 -0500 Subject: [CDash] Random questions In-Reply-To: References: Message-ID: On Thu, Feb 21, 2019 at 6:00 AM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > Hi, > > * What's a "track"? Does it refer to "Build Group"? > Yes, you're right. What CTest refers to as a "track" is a Build Group in CDash. * What are the CDash Append semantics? Is it configurable? > I'm not sure. I would have to do some manual experiments to see what effect (if any) the Append flag has on results submitted to CDash. * What are "Dynamic BuildGroups" > I have only seen them referenced in > http://mycdash.com/manageBuildGroup.php?projectid=1#!#dynamic You can use dynamic build groups to show the most recent results regardless of when they occurred. https://blog.kitware.com/dynamic-buildgroups-in-cdash/ An example of this is the "latest master" group on VTK's dashboard: https://open.cdash.org/index.php?project=VTK It shows the most recent results of building master across a variety of configurations. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zack.galbreath at kitware.com Thu Feb 21 20:39:30 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Thu, 21 Feb 2019 15:39:30 -0500 Subject: [CDash] Scripting project creation In-Reply-To: References: Message-ID: On Thu, Feb 21, 2019 at 7:15 AM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > We have lots of small independent projects instead of a few big ones > (in general the small ones are not really subprojects of others). So I > would like to script away the creation of projects in CDash. Can this > be done? > > In https://public.kitware.com/Wiki/CDash:API#List I see how to list > projects, but no how to create them. > Yes, this should be doable. When you click the "Create Project" button, CDash sends a POST request to /api/v1/project.php. An example of the JSON payload for this request is attached. In order to use this API endpoint you'll need to provide a bearer API token from a CDash user that has permission to create projects. You can create such a token on user.php after logging in. Include it in the header of your request like so: "Authorization: Bearer " -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: example.json Type: application/json Size: 700 bytes Desc: not available URL: From christian.morales.vega at gmail.com Fri Feb 22 09:54:54 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Fri, 22 Feb 2019 09:54:54 +0000 Subject: [CDash] Random questions In-Reply-To: References: Message-ID: On Thu, 21 Feb 2019 at 20:26, Zack Galbreath wrote: >> * What are the CDash Append semantics? Is it configurable? > > > I'm not sure. I would have to do some manual experiments to see what effect (if any) the Append flag has on results submitted to CDash. I'm guessing that puts it in the "nobody uses it -> is not really too tested -> avoid it" category? >> * What are "Dynamic BuildGroups" >> I have only seen them referenced in >> http://mycdash.com/manageBuildGroup.php?projectid=1#!#dynamic > > > You can use dynamic build groups to show the most recent results regardless of when they occurred. > > https://blog.kitware.com/dynamic-buildgroups-in-cdash/ > > An example of this is the "latest master" group on VTK's dashboard: > https://open.cdash.org/index.php?project=VTK > It shows the most recent results of building master across a variety of configurations. Thanks. FWIW I have already understood that half of the best documentation of CDash is in blog posts. And have read everything in https://blog.kitware.com/tag/cdash/. That specific post lacks the CDash tag! Probably I should have looked at https://blog.kitware.com/author/zack-galbreath/, though :-p I would look at https://blog.kitware.com/tag/software-process, but it's polluted with too many cmake X.Y.Z released/ready for testing posts. From christian.morales.vega at gmail.com Fri Feb 22 13:35:32 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Fri, 22 Feb 2019 13:35:32 +0000 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Thu, 21 Feb 2019 at 13:03, Zack Galbreath wrote: > Yes there is. > > https://github.com/Kitware/CDash/blob/master/public/api/v1/addBuild.php > > Provided the minimum details to define a build (site, name, stamp, project) this endpoint either returns an existing buildid (200) or creates a new one for you (201). Probably it would be useful to make ctest_submit() able to return this buildid? I can imagine this being call from Jenkins and wanting to store a reference to this buildid in the Jenkins logs. From zack.galbreath at kitware.com Fri Feb 22 14:15:28 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Fri, 22 Feb 2019 09:15:28 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Fri, Feb 22, 2019 at 8:35 AM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > Probably it would be useful to make ctest_submit() able to return this > buildid? > I can imagine this being call from Jenkins and wanting to store a > reference to this buildid in the Jenkins logs. > Good idea, I can definitely see that being a useful improvement to ctest_submit() -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.morales.vega at gmail.com Fri Feb 22 14:29:40 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Fri, 22 Feb 2019 14:29:40 +0000 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Fri, 22 Feb 2019 at 14:15, Zack Galbreath wrote: > > On Fri, Feb 22, 2019 at 8:35 AM Cristian Morales Vega wrote: >> >> Probably it would be useful to make ctest_submit() able to return this buildid? >> I can imagine this being call from Jenkins and wanting to store a >> reference to this buildid in the Jenkins logs. > > > Good idea, I can definitely see that being a useful improvement to ctest_submit() So here it is -> https://gitlab.kitware.com/cmake/cmake/issues/18968 From christian.morales.vega at gmail.com Fri Feb 22 17:02:49 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Fri, 22 Feb 2019 17:02:49 +0000 Subject: [CDash] Coverage reports Message-ID: Hi, I'm finding the coverage report support confusing, mainly the default "Directories" view. Not sure if I'm finding bugs or if I just don't get the view intention. So in my http://cdash.samknows.com/viewCoverage.php?buildid= view I can see the coverage of my "include//" directory is 0%. 2 out of 2 lines inside are not covered. The "status" is "Zero" and "2/2" under "Lines not covered" is in green. Inside that directory there are actually two files, but since the other one has 100% line coverage it seems to get filtered out by the "Directories" view. This is specially confusing when, from the "Directories" view, you click on the directory and you start seeing both files. I'm missing something? Regards. From christian.morales.vega at gmail.com Mon Feb 25 17:53:47 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Mon, 25 Feb 2019 17:53:47 +0000 Subject: [CDash] Branch Coverage with GCov Message-ID: Hi, I have tried to reproduce https://blog.kitware.com/additional-coverage-features-in-cdash/ but I have had no luck. The gcov.tar file is being created. I guess uploaded, but it's not 100% clear from the ctest output. In CDash I can't see coverage information at all. And I'm wondering why the script uses ctest_coverage_collect_gcov() instead of ctest_coverage(). Before trying to reproduce with branch_coverage_example.tar.gz I was trying to do it in my project by using ctest_coverage and basically just trying to add -b or -c to CTEST_COVERAGE_EXTRA_FLAGS. There I got the coverage info, as always... but with no extra branch information I could see. Regression? There is something obvious I can't see? From christian.morales.vega at gmail.com Tue Feb 26 16:07:28 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Tue, 26 Feb 2019 16:07:28 +0000 Subject: [CDash] Submitted "files" presentation Message-ID: ctest_submit() let's you submit PARTS, but also FILES. I have submitted a file with the output of "llvm-cov" with Source-based Code Coverage. I know CDash only supports gcov coverage right now, I don't expect to get full support for it. But I was hoping to be able to access the submitted file somehow from the CDash web view, linked somehow to the build. Maybe some "Extra files" section or something. I can't see the file at all. I know it was "submitted" to CDash... and that's it. No idea if it was actually stored somewhere or anything. What's the purpose of ctest_submit(FILES) and, does CDash has any support for it? From christian.morales.vega at gmail.com Tue Feb 26 16:57:44 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Tue, 26 Feb 2019 16:57:44 +0000 Subject: [CDash] Submitted "files" presentation In-Reply-To: References: Message-ID: On Tue, 26 Feb 2019 at 16:07, Cristian Morales Vega wrote: > ctest_submit() let's you submit PARTS, but also FILES. I have > submitted a file with the output of "llvm-cov" with Source-based Code > Coverage. I know CDash only supports gcov coverage right now, I don't > expect to get full support for it. But I was hoping to be able to > access the submitted file somehow from the CDash web view, linked > somehow to the build. Maybe some "Extra files" section or something. I have actually managed to get what I wanted with ctest_upload(). Not really sure what's supposed to be the difference, though. From zack.galbreath at kitware.com Fri Feb 1 21:32:25 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Fri, 1 Feb 2019 16:32:25 -0500 Subject: [CDash] [cmake-developers] [vtk-developers] CDash frontend refresh In-Reply-To: References: <8702064F-D659-48BF-9150-6A432DD911CC@kitware.com> Message-ID: Thanks for the feedback, everyone. I pushed a couple tweaks to testing.cdash.org. * The white numbers on index.php should be more legible now * reduced the padding of these tables slightly so more builds can fit on the screen at once * fixed the positioning of the calendar On Fri, Feb 1, 2019 at 8:40 AM Robert Maynard via cmake-developers < cmake-developers at cmake.org> wrote: > My feeling on this is that the green should be stronger color. A fully > green dashboard is a happy thing and it should make me the viewer > happy :) > This is a tough issue. I definitely want CDash to promote feelings of happiness and joy. But at the same time, CDash's main value is helping you figure out what's going wrong and fixing it. This is the motivation behind making warnings and errors more prominent in a sea of clean builds. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zack.galbreath at kitware.com Fri Feb 1 22:14:10 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Fri, 1 Feb 2019 17:14:10 -0500 Subject: [CDash] [cmake-developers] [vtk-developers] CDash frontend refresh In-Reply-To: References: <8702064F-D659-48BF-9150-6A432DD911CC@kitware.com> Message-ID: On Fri, Feb 1, 2019 at 5:00 PM Craig Scott wrote: > FWIW, my reaction was pretty similar to Robert's. A more greenish hue for > the passing fields would make them more subconsciously "these are okay" in > the reader's mind. The yellowish hue at the moment doesn't seem to evoke > that same subconscious response (to me at least), so I'm actually more > distracted by the new color than it helping me subconsciously filter out > those fields. > > The icon for the view settings (see attachment) also seems to have > changed. Took me a while to work out what it was and with no tooltip, I had > to click it to find out. Had I not used the earlier version, I may have > just assumed it was a broken image link and not realised it was > clickable/useful. > > [image: CDashRefresh_viewSettings.png] > Thanks for the useful feedback, Craig. I'll experiment with darker (yet still subdued) shades of green for passing builds. I'll also add a tooltip for the view settings icon. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: CDashRefresh_viewSettings.png Type: image/png Size: 19445 bytes Desc: not available URL: From Pavlech.Juraj at azd.cz Tue Feb 19 08:57:01 2019 From: Pavlech.Juraj at azd.cz (Pavlech Juraj) Date: Tue, 19 Feb 2019 08:57:01 +0000 Subject: [CDash] GetFeed Message-ID: <1550566621030.77202@azd.cz> Hello, After creating a new CDash project on localhost, I can not open this project and the CDash returns an error: Feed::GetFeed(): 0 not set Warning: Invalid argument supplied for foreach() in CDash/public/ajax/getfeed.php on line 99 Warning: count(): Parameter must be an array or an object that implements Countable in /CDash/public/ajax/getfeed.php on line 116 Thanks in advance for your help. -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.morales.vega at gmail.com Wed Feb 20 17:27:32 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Wed, 20 Feb 2019 17:27:32 +0000 Subject: [CDash] GitHub Status API support Message-ID: Hi, Currently when I push to GitHub it notifies my Jenkins instance. I want Jenkins to run a ctest script, which will include set(CTEST_UPDATE_VERSION_ONLY ON) ctest_update() And submit the results to CDash. ..and I would like CDash to tell GitHub about what it finds using its Status API (https://developer.github.com/v3/repos/statuses/). There is a simple way to do this? I know about https://cmake.org/cmake/help/latest/variable/CTEST_CHANGE_ID.html, but I have been unable to find too much documentation about it. It let's me "pass arbitrary information" which could for example be used "to post comments on your pull request". How "arbitrary" becomes "pull request ID", and whether I can make it mean "git commit" is unclear. Regards. From jcfr at kitware.com Wed Feb 20 17:54:16 2019 From: jcfr at kitware.com (Jean-Christophe Fillion-Robin) Date: Wed, 20 Feb 2019 12:54:16 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: Hi Cristian, You could great an app using ProBot (see https://github.com/probot/probot) Here is such an app created by Matt M for ITK: https://github.com/InsightSoftwareConsortium/ITKGitHubCDashStatus Hth Jc On Wed, Feb 20, 2019 at 12:27 PM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > Hi, > > Currently when I push to GitHub it notifies my Jenkins instance. I > want Jenkins to run a ctest script, which will include > > set(CTEST_UPDATE_VERSION_ONLY ON) > ctest_update() > > And submit the results to CDash. > ..and I would like CDash to tell GitHub about what it finds using its > Status API (https://developer.github.com/v3/repos/statuses/). > > There is a simple way to do this? > > I know about > https://cmake.org/cmake/help/latest/variable/CTEST_CHANGE_ID.html, > but I have been unable to find too much documentation about it. It > let's me "pass arbitrary information" which could for example be used > "to post comments on your pull request". How "arbitrary" becomes "pull > request ID", and whether I can make it mean "git commit" is unclear. > > Regards. > _______________________________________________ > CDash mailing list > CDash at public.kitware.com > https://public.kitware.com/mailman/listinfo/cdash > -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.morales.vega at gmail.com Wed Feb 20 18:59:51 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Wed, 20 Feb 2019 18:59:51 +0000 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Wed, 20 Feb 2019 at 17:27, Cristian Morales Vega wrote: > I know about https://cmake.org/cmake/help/latest/variable/CTEST_CHANGE_ID.html, > but I have been unable to find too much documentation about it. It > let's me "pass arbitrary information" which could for example be used > "to post comments on your pull request". How "arbitrary" becomes "pull > request ID", and whether I can make it mean "git commit" is unclear. I have just found this: https://public.kitware.com/pipermail/cdash/2018-January/001832.html Zack, what are your plans on this? Maybe it's already implemented but not documented? From zack.galbreath at kitware.com Wed Feb 20 19:42:17 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Wed, 20 Feb 2019 14:42:17 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Wed, Feb 20, 2019 at 2:00 PM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > Zack, what are your plans on this? Maybe it's already implemented but > not documented? > We are really, really close :) First up, here's a blog post describing CDash's current integration with GitHub: https://blog.kitware.com/cdash-integration-with-github/ This works by commenting on your pull request whenever something goes wrong (build error, test failure, etc). But as you indicated, rather than commenting on a PR, it would be better to have CDash post a status to the commit on GitHub with a link back to itself. The biggest barrier to this goal was the fact that CDash didn't actually ever know when a build was done. So we could only ever post bad news (this is what went wrong), not good news (everything passed, hurray). There was no mechanism in place for CTest to say "Okay CDash, that's the last file I'm going to send you for this build". CTest and CDash finally gained this capability as of CMake v3.14 (currently in the release candidate cycle). So now that CDash can actually know when a build is done, we need to teach CDash to use GitHub's status API. This is implemented in CDash PR #748 . I've already reviewed these changes and they look good to me. I still need to find the time to test it out & make sure the GitHub status gets updated as expected. Once I do that I'll merge this change into CDash, and our goal of linking back to CDash from the commit status on GitHub will finally be achieved. -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.morales.vega at gmail.com Wed Feb 20 19:54:40 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Wed, 20 Feb 2019 19:54:40 +0000 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Wed, 20 Feb 2019 at 19:42, Zack Galbreath wrote: > This works by commenting on your pull request whenever something goes wrong (build error, test failure, etc). But as you indicated, rather than commenting on a PR, it would be better to have CDash post a status to the commit on GitHub with a link back to itself. The biggest barrier to this goal was the fact that CDash didn't actually ever know when a build was done. So we could only ever post bad news (this is what went wrong), not good news (everything passed, hurray). There was no mechanism in place for CTest to say "Okay CDash, that's the last file I'm going to send you for this build". CTest and CDash finally gained this capability as of CMake v3.14 (currently in the release candidate cycle). > > So now that CDash can actually know when a build is done, we need to teach CDash to use GitHub's status API. This is implemented in CDash PR #748. I've already reviewed these changes and they look good to me. I still need to find the time to test it out & make sure the GitHub status gets updated as expected. Once I do that I'll merge this change into CDash, and our goal of linking back to CDash from the commit status on GitHub will finally be achieved. That's great! I had not though about the lack of "done". For the time being... my Jenkins actually does know when it has finished. If I could retrieve the CDash buildid when doing ctest_submit() I could store/show it somewhere. There is any way to do this? From zack.galbreath at kitware.com Wed Feb 20 21:19:58 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Wed, 20 Feb 2019 16:19:58 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Wed, Feb 20, 2019 at 2:54 PM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > For the time being... my Jenkins actually does know when it has > finished. If I could retrieve the CDash buildid when doing > ctest_submit() I could store/show it somewhere. There is any way to do > this? > As of CMake 3.13, yes. CTest now provides enough information for CDash to initialize a buildid at submit time (rather than parse time). CDash includes this buildid in its XML response. It looks like the only way to see this from the command-line currently is to call ctest with the --debug argument. Here's an example: OK 56 663d1dbb340ef6166058d94b990107e2 There's a potential complication here if the CDash instance you're submitting to is configured for asynchronous submissions (as most of our public CDash instances are). The concept here is that CDash can get busy at certain times of day. So instead of asking the testing clients to wait while CDash parses their XML files, the submissions get added to a queue that CDash works on offline. So just because your build has completed in Jenkins doesn't mean that CDash has parsed its results yet. -------------- next part -------------- An HTML attachment was scrubbed... URL: From DLRdave at aol.com Thu Feb 21 00:30:20 2019 From: DLRdave at aol.com (David Cole) Date: Wed, 20 Feb 2019 19:30:20 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: Surely there's a way to query for the build id given the components that go into making up the unique key (build stamp, build type, site, .... I can't remember all the components required to get the unique key....?) Is there a way to send that up to CDash and get a build id response back? (Because those things should be well-known by the dashboard submitter.) On Wed, Feb 20, 2019 at 4:19 PM Zack Galbreath via CDash wrote: > > On Wed, Feb 20, 2019 at 2:54 PM Cristian Morales Vega wrote: >> >> For the time being... my Jenkins actually does know when it has >> finished. If I could retrieve the CDash buildid when doing >> ctest_submit() I could store/show it somewhere. There is any way to do >> this? > > > As of CMake 3.13, yes. CTest now provides enough information for CDash to initialize a buildid at submit time (rather than parse time). CDash includes this buildid in its XML response It looks like the only way to see this from the command-line currently is to call ctest with the --debug argument. Here's an example: > > > OK > > 56 > 663d1dbb340ef6166058d94b990107e2 > > > There's a potential complication here if the CDash instance you're submitting to is configured for asynchronous submissions (as most of our public CDash instances are). The concept here is that CDash can get busy at certain times of day. So instead of asking the testing clients to wait while CDash parses their XML files, the submissions get added to a queue that CDash works on offline. So just because your build has completed in Jenkins doesn't mean that CDash has parsed its results yet. > _______________________________________________ > CDash mailing list > CDash at public.kitware.com > https://public.kitware.com/mailman/listinfo/cdash From christian.morales.vega at gmail.com Thu Feb 21 11:00:28 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Thu, 21 Feb 2019 11:00:28 +0000 Subject: [CDash] Random questions Message-ID: Hi, * What's a "track"? https://cmake.org/cmake/help/latest/command/ctest_start.html#command:ctest_start "If TRACK is used, the submissions will go to the specified track on the CDash server." Does it refer to "Build Group"? The only thing linking both I have found is this small note in the Administrator's guide (https://public.kitware.com/Wiki/CDash:Administration#Build_Groups): "(also called tracks in Dart2)". * What are the CDash Append semantics? Is it configurable? https://cmake.org/cmake/help/latest/command/ctest_configure.html#command:ctest_configure and others says: "Append semantics are defined by the dashboard server in use". Fair enough, but I have not found any mention to append semantics in the dashboard server documentation. When the "append" feature was created, what was its intended use? * What are "Dynamic BuildGroups" I have only seen them referenced in http://mycdash.com/manageBuildGroup.php?projectid=1#!#dynamic From christian.morales.vega at gmail.com Thu Feb 21 12:14:43 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Thu, 21 Feb 2019 12:14:43 +0000 Subject: [CDash] Scripting project creation Message-ID: We have lots of small independent projects instead of a few big ones (in general the small ones are not really subprojects of others). So I would like to script away the creation of projects in CDash. Can this be done? In https://public.kitware.com/Wiki/CDash:API#List I see how to list projects, but no how to create them. From zack.galbreath at kitware.com Thu Feb 21 13:03:30 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Thu, 21 Feb 2019 08:03:30 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Wed, Feb 20, 2019 at 7:30 PM David Cole wrote: > Surely there's a way to query for the build id given the components > that go into making up the unique key (build stamp, build type, site, > .... I can't remember all the components required to get the unique > key....?) > > Is there a way to send that up to CDash and get a build id response > back? (Because those things should be well-known by the dashboard > submitter.) > Yes there is. https://github.com/Kitware/CDash/blob/master/public/api/v1/addBuild.php Provided the minimum details to define a build (site, name, stamp, project) this endpoint either returns an existing buildid (200) or creates a new one for you (201). -------------- next part -------------- An HTML attachment was scrubbed... URL: From zack.galbreath at kitware.com Thu Feb 21 20:26:47 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Thu, 21 Feb 2019 15:26:47 -0500 Subject: [CDash] Random questions In-Reply-To: References: Message-ID: On Thu, Feb 21, 2019 at 6:00 AM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > Hi, > > * What's a "track"? Does it refer to "Build Group"? > Yes, you're right. What CTest refers to as a "track" is a Build Group in CDash. * What are the CDash Append semantics? Is it configurable? > I'm not sure. I would have to do some manual experiments to see what effect (if any) the Append flag has on results submitted to CDash. * What are "Dynamic BuildGroups" > I have only seen them referenced in > http://mycdash.com/manageBuildGroup.php?projectid=1#!#dynamic You can use dynamic build groups to show the most recent results regardless of when they occurred. https://blog.kitware.com/dynamic-buildgroups-in-cdash/ An example of this is the "latest master" group on VTK's dashboard: https://open.cdash.org/index.php?project=VTK It shows the most recent results of building master across a variety of configurations. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zack.galbreath at kitware.com Thu Feb 21 20:39:30 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Thu, 21 Feb 2019 15:39:30 -0500 Subject: [CDash] Scripting project creation In-Reply-To: References: Message-ID: On Thu, Feb 21, 2019 at 7:15 AM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > We have lots of small independent projects instead of a few big ones > (in general the small ones are not really subprojects of others). So I > would like to script away the creation of projects in CDash. Can this > be done? > > In https://public.kitware.com/Wiki/CDash:API#List I see how to list > projects, but no how to create them. > Yes, this should be doable. When you click the "Create Project" button, CDash sends a POST request to /api/v1/project.php. An example of the JSON payload for this request is attached. In order to use this API endpoint you'll need to provide a bearer API token from a CDash user that has permission to create projects. You can create such a token on user.php after logging in. Include it in the header of your request like so: "Authorization: Bearer " -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: example.json Type: application/json Size: 700 bytes Desc: not available URL: From christian.morales.vega at gmail.com Fri Feb 22 09:54:54 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Fri, 22 Feb 2019 09:54:54 +0000 Subject: [CDash] Random questions In-Reply-To: References: Message-ID: On Thu, 21 Feb 2019 at 20:26, Zack Galbreath wrote: >> * What are the CDash Append semantics? Is it configurable? > > > I'm not sure. I would have to do some manual experiments to see what effect (if any) the Append flag has on results submitted to CDash. I'm guessing that puts it in the "nobody uses it -> is not really too tested -> avoid it" category? >> * What are "Dynamic BuildGroups" >> I have only seen them referenced in >> http://mycdash.com/manageBuildGroup.php?projectid=1#!#dynamic > > > You can use dynamic build groups to show the most recent results regardless of when they occurred. > > https://blog.kitware.com/dynamic-buildgroups-in-cdash/ > > An example of this is the "latest master" group on VTK's dashboard: > https://open.cdash.org/index.php?project=VTK > It shows the most recent results of building master across a variety of configurations. Thanks. FWIW I have already understood that half of the best documentation of CDash is in blog posts. And have read everything in https://blog.kitware.com/tag/cdash/. That specific post lacks the CDash tag! Probably I should have looked at https://blog.kitware.com/author/zack-galbreath/, though :-p I would look at https://blog.kitware.com/tag/software-process, but it's polluted with too many cmake X.Y.Z released/ready for testing posts. From christian.morales.vega at gmail.com Fri Feb 22 13:35:32 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Fri, 22 Feb 2019 13:35:32 +0000 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Thu, 21 Feb 2019 at 13:03, Zack Galbreath wrote: > Yes there is. > > https://github.com/Kitware/CDash/blob/master/public/api/v1/addBuild.php > > Provided the minimum details to define a build (site, name, stamp, project) this endpoint either returns an existing buildid (200) or creates a new one for you (201). Probably it would be useful to make ctest_submit() able to return this buildid? I can imagine this being call from Jenkins and wanting to store a reference to this buildid in the Jenkins logs. From zack.galbreath at kitware.com Fri Feb 22 14:15:28 2019 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Fri, 22 Feb 2019 09:15:28 -0500 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Fri, Feb 22, 2019 at 8:35 AM Cristian Morales Vega < christian.morales.vega at gmail.com> wrote: > Probably it would be useful to make ctest_submit() able to return this > buildid? > I can imagine this being call from Jenkins and wanting to store a > reference to this buildid in the Jenkins logs. > Good idea, I can definitely see that being a useful improvement to ctest_submit() -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.morales.vega at gmail.com Fri Feb 22 14:29:40 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Fri, 22 Feb 2019 14:29:40 +0000 Subject: [CDash] GitHub Status API support In-Reply-To: References: Message-ID: On Fri, 22 Feb 2019 at 14:15, Zack Galbreath wrote: > > On Fri, Feb 22, 2019 at 8:35 AM Cristian Morales Vega wrote: >> >> Probably it would be useful to make ctest_submit() able to return this buildid? >> I can imagine this being call from Jenkins and wanting to store a >> reference to this buildid in the Jenkins logs. > > > Good idea, I can definitely see that being a useful improvement to ctest_submit() So here it is -> https://gitlab.kitware.com/cmake/cmake/issues/18968 From christian.morales.vega at gmail.com Fri Feb 22 17:02:49 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Fri, 22 Feb 2019 17:02:49 +0000 Subject: [CDash] Coverage reports Message-ID: Hi, I'm finding the coverage report support confusing, mainly the default "Directories" view. Not sure if I'm finding bugs or if I just don't get the view intention. So in my http://cdash.samknows.com/viewCoverage.php?buildid= view I can see the coverage of my "include//" directory is 0%. 2 out of 2 lines inside are not covered. The "status" is "Zero" and "2/2" under "Lines not covered" is in green. Inside that directory there are actually two files, but since the other one has 100% line coverage it seems to get filtered out by the "Directories" view. This is specially confusing when, from the "Directories" view, you click on the directory and you start seeing both files. I'm missing something? Regards. From christian.morales.vega at gmail.com Mon Feb 25 17:53:47 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Mon, 25 Feb 2019 17:53:47 +0000 Subject: [CDash] Branch Coverage with GCov Message-ID: Hi, I have tried to reproduce https://blog.kitware.com/additional-coverage-features-in-cdash/ but I have had no luck. The gcov.tar file is being created. I guess uploaded, but it's not 100% clear from the ctest output. In CDash I can't see coverage information at all. And I'm wondering why the script uses ctest_coverage_collect_gcov() instead of ctest_coverage(). Before trying to reproduce with branch_coverage_example.tar.gz I was trying to do it in my project by using ctest_coverage and basically just trying to add -b or -c to CTEST_COVERAGE_EXTRA_FLAGS. There I got the coverage info, as always... but with no extra branch information I could see. Regression? There is something obvious I can't see? From christian.morales.vega at gmail.com Tue Feb 26 16:07:28 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Tue, 26 Feb 2019 16:07:28 +0000 Subject: [CDash] Submitted "files" presentation Message-ID: ctest_submit() let's you submit PARTS, but also FILES. I have submitted a file with the output of "llvm-cov" with Source-based Code Coverage. I know CDash only supports gcov coverage right now, I don't expect to get full support for it. But I was hoping to be able to access the submitted file somehow from the CDash web view, linked somehow to the build. Maybe some "Extra files" section or something. I can't see the file at all. I know it was "submitted" to CDash... and that's it. No idea if it was actually stored somewhere or anything. What's the purpose of ctest_submit(FILES) and, does CDash has any support for it? From christian.morales.vega at gmail.com Tue Feb 26 16:57:44 2019 From: christian.morales.vega at gmail.com (Cristian Morales Vega) Date: Tue, 26 Feb 2019 16:57:44 +0000 Subject: [CDash] Submitted "files" presentation In-Reply-To: References: Message-ID: On Tue, 26 Feb 2019 at 16:07, Cristian Morales Vega wrote: > ctest_submit() let's you submit PARTS, but also FILES. I have > submitted a file with the output of "llvm-cov" with Source-based Code > Coverage. I know CDash only supports gcov coverage right now, I don't > expect to get full support for it. But I was hoping to be able to > access the submitted file somehow from the CDash web view, linked > somehow to the build. Maybe some "Extra files" section or something. I have actually managed to get what I wanted with ctest_upload(). Not really sure what's supposed to be the difference, though.