View Issue Details [ Jump to Notes ]

[ Print ]

Project

Category

View Status

Date Submitted

Last Update

0006384

CDash

public

2008-02-19 14:00

2008-04-28 09:17

Reporter

Bill Lorensen

Assigned To

Julien Jomier

Priority

normal

Severity

feature

Reproducibility

N/A

Status

closed

Resolution

fixed

Platform

OS Version

Product Version

Target Version

Fixed in Version

1.0

Summary

0006384: Detecting timing defects

Description

Changes that affect timing often go undetected. A defect should be noted if the timing of a test changes "significantly" on a given platform.

For example, looking at the VTK dashboard http://public.kitware.com/CDash/testDetails.php?test=390293&build=20282 [^]

something happened in mid January that caused this test to increase it's time by 300%! But, nothing on any of our dashboards (Classic Dart, Dart 2 or CDash) noted this!

Additional Information

In the pre-Dart days, our system at GE reported significant timing changes.
In the paper "Visualization Toolkit Extreme Testing," Computer Graphics, vol. 35, no. 3, pp. 8-11 (August 2001) http://marchingcubes.org/images/9/90/VTKExtremeTesting.pdf [^]
Jim Miller and I describe the pre-Dart testing system.
Regarding timing,
"A timing analysis is performed on each platform. A summary of tests that are faster or slower is created and links are made to plots that show the variation over the last 10 days and last 12 weeks."

BTW: I think this was Lisa's Greenbelt project. Ask her about it...

Tags

No tags attached.

Attached Files

Relationships

Notes
(0010593) Julien Jomier (manager) 2008-02-19 14:03	This one is high priority on my list. We discussed about this with Luis. It should be implemented soon.

(0011567) Julien Jomier (manager) 2008-04-28 09:17	This feature has been added in 1.0. There is probably room for improvement but so far it seems to work as expected.

Issue History
Date Modified	Username	Field	Change
2008-02-19 14:00	Bill Lorensen	New Issue
2008-02-19 14:03	Julien Jomier	Note Added: 0010593
2008-02-19 14:03	Julien Jomier	Status	new => assigned
2008-02-19 14:03	Julien Jomier	Assigned To	=> Julien Jomier
2008-04-28 09:17	Julien Jomier	Status	assigned => closed
2008-04-28 09:17	Julien Jomier	Note Added: 0011567
2008-04-28 09:17	Julien Jomier	Resolution	open => fixed
2008-04-28 09:17	Julien Jomier	Fixed in Version	=> 1.0