[Cdash] tracing regressions in cdash
julien.jomier at kitware.com
Wed Jun 17 16:46:37 UTC 2009
This is an interesting point and definitely a nice feature to have. I
think we can create a new page showing the history of a given test (when
it fails/passes) and be able to go back to the corresponding checkin. We
can also improve the test page for a build showing if the status of a
test recently changed.
Could you log a feature request in the bug tracker (www.cdash.org/Bug)?
Thanks for the feedback,
Matthias Brantner wrote:
> with our open-source XQuery processor Zorba, we are a very happy user of
> CDash. Our projects submits a huge number of tests every day. Among
> these tests is a W3C conformance testsuite with approx. 15k tests.
> Currently, we do not pass all of these tests since we are still working
> on getting version 1.0 out (i.e. between 100-500 tests are still
> failing depending on the setup). As we are going along and trying to fix
> more and more of these tests, it is a big problem for us to trace
> regressions. For instance, we want to know if a commit makes particular
> tests fail. Although, CDash does report an de-/increase of
> failing/passing tests in the "Fail" and "Pass" columns this is
> unfortunately not good enough for us because CDash doesn't tell us if we
> had a regression. For instance, a commit could break one test but make
> two others pass.
> Does anybody else have this problem? Did you already think about
> providing a feature in CDash that allows for such a regression analysis?
> This would be something really helpful for us.
> Thanks for doing all this work.
> Best regards
> Cdash mailing list
> Cdash at public.kitware.com
More information about the CDash