View Issue Details Jump to Notes ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0007051CDashpublic2008-05-19 12:032008-07-29 13:26
ReporterBrad King 
Assigned ToJulien Jomier 
PrioritynormalSeverityminorReproducibilityN/A
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version1.2 
Target VersionFixed in Version1.2 
Summary0007051: Timing tests have too many false positives
DescriptionThe algorithm to detect time failures needs to be revised. One time sample lying outside the majority of the data is *not* a failure. Single outliers can occur from busy machines or any number of reasons. Only several samples can define a failure.

Statistically, the test would be something like deciding if the last N days are sampled from a distribution with a different mean than the distribution of the preceding M days (with M much bigger than N). Such a formal test would probably be too expensive. Instead we could use some kind of hack to reduce false positives by requiring that the last 3 or 4 days be outliers according to the current test before reporting the failure.
TagsNo tags attached.
Attached Files

 Relationships

  Notes
(0012861)
Julien Jomier (manager)
2008-07-29 13:26

There is now a "Test time # max failures before flag" variable per project. That means that it doesn't display the timing error if the error occurs less than the max number of testing time failure.

Thanks for the report.

 Issue History
Date Modified Username Field Change
2008-05-19 12:03 Brad King New Issue
2008-05-19 12:06 Julien Jomier Status new => assigned
2008-05-19 12:06 Julien Jomier Assigned To => Julien Jomier
2008-07-29 13:26 Julien Jomier Status assigned => closed
2008-07-29 13:26 Julien Jomier Note Added: 0012861
2008-07-29 13:26 Julien Jomier Resolution open => fixed
2008-07-29 13:26 Julien Jomier Fixed in Version => 1.2


Copyright © 2000 - 2018 MantisBT Team