[CDash] Acceptable range

[Zack Galbreath]

On Tue, Sep 29, 2020 at 12:31 PM Sébastien Dinot <sebastien.dinot at csgroup.eu>
wrote:

> Dear CDash community,
>
> Let's take the test result below as an example:
>
> https://open.cdash.org/test/252217127
>
> I have some questions related to the "Acceptable Range" serie, which
> appear on the "Test Time" graph:
>
>    1.
>
>    How is set exactly this "Acceptable Range", which seems to be a
>    dynamically calculated trend?
>    2.
>
>    Is it possible to change this threshold and to give it a fixed value?
>    3.
>
>    What happens when the "Execution Time" of a test exceeds the
>    "Acceptable Range"? Does CDash display any particular information on the
>    dashboard? Does it send a notification to developers?
>
> I did not find any information on this subject, neither in the project
> documentation, nor on any other web site.
>


There's a little bit of documentation for this feature here:
https://public.kitware.com/Wiki/CDash:Design#Test_Timing

The idea is that CDash maintains a rolling average of how long each test
takes to run on a per-site, per-build basis, and uses this to automatically
detect performance regressions.

To use a fixed value instead, the test timeout property
<https://cmake.org/cmake/help/latest/prop_test/TIMEOUT.html> is probably
more appropriate.

You can enable/disable this feature & tweak the numbers that go into this
formula by visiting the "Testing" tab on your project's Settings page. When
this feature is enabled, CDash will send emails about tests that violate
the "time status" check. They will also show up as red on the dashboard.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://public.kitware.com/pipermail/cdash/attachments/20200929/f3cd7241/attachment.htm>