MantisBT - CDash
View Issue Details
0006384CDashpublic2008-02-19 14:002008-04-28 09:17
Bill Lorensen 
Julien Jomier 
normalfeatureN/A
closedfixed 
 
1.0 
0006384: Detecting timing defects
Changes that affect timing often go undetected. A defect should be noted if the timing of a test changes "significantly" on a given platform.

For example, looking at the VTK dashboard http://public.kitware.com/CDash/testDetails.php?test=390293&build=20282 [^]

something happened in mid January that caused this test to increase it's time by 300%! But, nothing on any of our dashboards (Classic Dart, Dart 2 or CDash) noted this!
In the pre-Dart days, our system at GE reported significant timing changes.
In the paper "Visualization Toolkit Extreme Testing," Computer Graphics, vol. 35, no. 3, pp. 8-11 (August 2001) http://marchingcubes.org/images/9/90/VTKExtremeTesting.pdf [^]
Jim Miller and I describe the pre-Dart testing system.
Regarding timing,
"A timing analysis is performed on each platform. A summary of tests that are faster or slower is created and links are made to plots that show the variation over the last 10 days and last 12 weeks."

BTW: I think this was Lisa's Greenbelt project. Ask her about it...

No tags attached.
Issue History
2008-02-19 14:00Bill LorensenNew Issue
2008-02-19 14:03Julien JomierNote Added: 0010593
2008-02-19 14:03Julien JomierStatusnew => assigned
2008-02-19 14:03Julien JomierAssigned To => Julien Jomier
2008-04-28 09:17Julien JomierStatusassigned => closed
2008-04-28 09:17Julien JomierNote Added: 0011567
2008-04-28 09:17Julien JomierResolutionopen => fixed
2008-04-28 09:17Julien JomierFixed in Version => 1.0

Notes
(0010593)
Julien Jomier   
2008-02-19 14:03   
This one is high priority on my list. We discussed about this with Luis. It should be implemented soon.
(0011567)
Julien Jomier   
2008-04-28 09:17   
This feature has been added in 1.0. There is probably room for improvement but so far it seems to work as expected.