[Cdash] tracing regressions in cdash

[Julien Jomier]

Hi Matthias,

This is an interesting point and definitely a nice feature to have. I 
think we can create a new page showing the history of a given test (when 
it fails/passes) and be able to go back to the corresponding checkin. We 
can also improve the test page for a build showing if the status of a 
test recently changed.

Could you log a feature request in the bug tracker (www.cdash.org/Bug)?

Thanks for the feedback,
Julien

Matthias Brantner wrote:
> Hello,
> 
> with our open-source XQuery processor Zorba, we are a very happy user of 
> CDash. Our projects submits a huge number of tests every day. Among 
> these tests is a W3C conformance testsuite with approx. 15k tests. 
> Currently, we do not pass all of these tests since we are still working 
> on getting version 1.0 out  (i.e. between 100-500 tests are still 
> failing depending on the setup). As we are going along and trying to fix 
> more and more of these tests, it is a big problem for us to trace 
> regressions. For instance, we want to know if a commit makes particular 
> tests fail. Although, CDash does report an de-/increase of 
> failing/passing tests in the "Fail" and "Pass" columns this is 
> unfortunately not good enough for us because CDash doesn't tell us if we 
> had a regression. For instance, a commit could break one test but make 
> two others pass.
> 
> Does anybody else have this problem? Did you already think about 
> providing a feature in CDash that allows for such a regression analysis? 
> This would be something really helpful for us.
> 
> Thanks for doing all this work.
> 
> Best regards
> 
> Matthias
> _______________________________________________
> Cdash mailing list
> Cdash at public.kitware.com
> http://public.kitware.com/cgi-bin/mailman/listinfo/cdash
>