[Dart] Dart 1.0.0 cruisecontrol support

Wed May 17 09:57:51 EDT 2006

Scott,

There are a series of "summarization" tasks that should propagate information from the leaf tests to the root test after a submission arrives. We need to check whether these summarization tasks are triggered on the new format.

"Execution Time" is used to measure the amount of time of a leaf test.  The "ElapsedTime" is used to measure the amount of time for the test hierarchy (.Build, .Test, etc.).  We probably could have used the same name for these but we didn't because they come from two completely different sections of the Dart Classic formats.

We should be able to generate a set of XML examples from the archiver.

Jim

-----Original Message-----
From: Scott W. Oster [mailto:oster at bmi.osu.edu]
Sent: Wednesday, May 17, 2006 9:33 AM
To: Miller, James V (GE, Research); Blezek, Daniel J (GE, Research); dart at public.kitware.com
Subject: RE: [Dart] Dart 1.0.0 cruisecontrol support

The "special measurements" that are used for most Test types aren't detailed, and it took some trial and error to figure them out.  For example, to show the build time, you need to set an "ElapsedTime" measurement, but the apparently same thing for the "Test" test type is "Execution Time".  Also, in addition to Note, I don't believe the manual even mentions the Update format.

I was able to figure most things out from here: http://www.na-mic.org:8000/websvn/filedetails.php?repname=Dart <http://www.na-mic.org:8000/websvn/filedetails.php?repname=Dart&path=%2Ftrunk%2FSource%2Fdart%2FResources%2FServer%2FTestProcessorRules.xml&rev=0&sc=0> &path=%2Ftrunk%2FSource%2Fdart%2FResources%2FServer%2FTestProcessorRules.xml&rev=0&sc=0

But the information that isn't available in that file is the business logic on how things are linked together (such as how error counts are rolled up, etc).  I am apparently not doing something right for errors, as when I click on one on the dartboard, it generates an exception: http://vandelay.bmi.ohio-state.edu:8081/caGrid-1.0/Dashboard/Build?submissionid=55

Also, the way build logs are used in practice isn't entirely clear from the manual; examples would help greatly.  I am currently just using "stages" with Log, not parsing into Lines.  Finally, my total test time is not rolling up on the dartboard, even though the execution time for each individual test is.  Not sure if I need to calculate the "ElapsedTime" my self, and submit it with the parent .Test.

Scott

Scott Oster

Senior Research Specialist
Biomedical Informatics Department
Ohio State University
 <mailto:oster at bmi.osu.edu> oster at bmi.osu.edu

lab: 614-292-9845
voice mail: 614-292-8680

  _____  

From: dart-bounces+oster=bmi.osu.edu at public.kitware.com [mailto:dart-bounces+oster=bmi.osu.edu at public.kitware.com] On Behalf Of Miller, James V (GE, Research)
Sent: Wednesday, May 17, 2006 9:17 AM
To: Scott W. Oster; Blezek, Daniel J (GE, Research); dart at public.kitware.com
Subject: RE: [Dart] Dart 1.0.0 cruisecontrol support

Scott,

We'll pull together some other examples.  Is there a specific question on the format we can answer? The manual briefly describes for each type of test (Test, Build, Coverage. Style, and DynamicAnalysis but Notes seems to be missing) what the (potentially optional) Measurements are.  But this will all be made clearer by an example per test type.

There are some things under the covers are not covered here that we need probably need to address.  As one example, we sometimes create a phantom test called _Properties to record some information from the Dart Classic format where there was no clean place to store it in the Dart2. 

We'll expand the documentation of the XML format.

Jim

-----Original Message-----
From: dart-bounces+millerjv=crd.ge.com at public.kitware.com [mailto:dart-bounces+millerjv=crd.ge.com at public.kitware.com]On Behalf Of Scott W. Oster
Sent: Tuesday, May 16, 2006 5:11 PM
To: Blezek, Daniel J (GE, Research); dart at public.kitware.com
Cc: Scott W. Oster
Subject: RE: [Dart] Dart 1.0.0 cruisecontrol support

I have an XSLT script (CC2DART.xsl) that seems to be doing a reasonable job converting the cruisecontrol log to the standard dart submission format.

This is the first XSLT I have ever written, so I'm sure there are much better ways to do some of what I am doing in it.

For those who are interested, it's available here:

http://gforge.nci.nih.gov/plugins/scmcvs/cvsweb.php/~checkout~/cagrid-1-0/caGrid/test/CC2DART.xsl?content-type=text%2Fplain;cvsroot=cagrid-1-0

I call this in my antpublisher task using the ant xslt task, before passing the result off to the existing publish task (as described in the manual).

I still have several questions about the expected format that the dart manual does not cover.  I am confident I could have this 100% working, if I knew exactly what XML I needed to produce.

Could someone please send me additional details on the submission format, or several full example files?  The examples should use all the test types (which the Dart manual example does not provide), such as Update, Build, Coverage, Note, etc.

Thanks,

Scott

Scott Oster

Senior Research Specialist
Biomedical Informatics Department
Ohio State University
 <mailto:oster at bmi.osu.edu> oster at bmi.osu.edu

lab: 614-292-9845
voice mail: 614-292-8680

  _____  

From: Blezek, Daniel J (GE, Research) [mailto:blezek at crd.ge.com] 
Sent: Monday, May 15, 2006 4:54 PM
To: Scott W. Oster; dart at public.kitware.com
Subject: RE: [Dart] Dart 1.0.0 cruisecontrol support

Scott,

  We always like to see people willing to help out with code!

  Glad things are reasonably working, I had thought the first item on the issues list was working, but perhaps not.  Build/Test times are not rolled up, and CVS remains a problem.  The best possible thing would be to write an XSLT that transforms the CruiseControl logs to something Dart likes, but that may be problematic.  Second best is to have a look at the Dart Digestor parsing (also problematic, it's a real mess).

  I'm open to any and all suggestions as to how to make Dart work better with CruiseControl.  The current method is somewhat of a hack and needs to be refactored.

  If you have more CC experience, put something a suggestion on the Wiki and we can start working towards that.

Thanks,

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://public.kitware.com/pipermail/dart/attachments/20060517/5a9ebb90/attachment-0001.htm