[Cdash] [Insight-developers] Dashboard: Normalizing TIMEOUTS
bill.lorensen at gmail.com
Mon Jan 18 16:55:38 UTC 2010
It would need to be broken down by build type. For example a debug
build on some platforms can be 10-30 slower that a release build.
On Mon, Jan 18, 2010 at 11:32 AM, Luis Ibanez <luis.ibanez at kitware.com> wrote:
> As you may have noticed, the standard practice
> of using a single TIMEOUT number for all the
> ~1,700 test in ITK brings up the challenge of
> defining what a good timeout value is for each
> machine (and configuration: eg. Release/Debug).
> The following proposal was raised in the past,
> but we have not acted upon:
> 1) Add to ITK one or two test that can be considered
> a good benchmark for:
> a) computation power
> b) input / output speed
> 2) Run those tests and use their timings as a
> base value that characterize this machine.
> 3) Define timeout for all tests that are based
> on the values found in (2), multiplied by
> a factor.
> Let's say that the computation benchmark takes
> 2 seconds to run in the machine foobar.kitware,
> then we can tell that the DiffeomorphicDemons
> registration test in the same machine should take
> 153 x (time of benchmark1 )
> (where the number "153" is a factor that we
> will have to estimate for each test).
> CDash already does a similar thing with the
> historical record of the computation time that
> it takes to run every test on a given machine,
> although this is done on the CDash server,
> and therefore it happens too late to be used
> as a TIMEOUT mark.
> An interesting option as well, could be for
> a machine to get access to the historical
> record that CDash has computed, and then
> use those values as a base for computing
> TIMEOUT at the moment of running ctest.
> What do people think of these options ?
> Powered by www.kitware.com
> Visit other Kitware open-source projects at
> Kitware offers ITK Training Courses, for more information visit:
> Please keep messages on-topic and check the ITK FAQ at:
> Follow this link to subscribe/unsubscribe:
More information about the CDash