[Cdash] [cmake-developers] RFC: pretty/HTML output in test logs in ctest/cdash

[Zach Mullen]

I like the idea as long as it is well namespaced, e.g.:

<cdash:sometag>Value</cdash:sometag>

It wouldn't even need to use XML style, it could be more like handlebars or
something, but we'd need to be very sure it wouldn't mangle any output that
wasn't intended to be mangled by CDash.


On Fri, May 3, 2013 at 10:12 AM, Julien Jomier <julien.jomier at kitware.com>wrote:

> Alex,
>
> I'm very sorry for the late reply.
>
> I like this idea very much. I'm a little bit ambivalent about adopting
> HTML as the output as it could be tricky to standardize the output on the
> CDash side, but at the same time it gives flexibility. I'm wondering if
> CDash should do this automatically based on some pre-defined tags.
>
> Anyone has an opinion on this?
>
> Regarding the short lines, I would think that the test should be modified
> to produce the right output. We could implement something in CDash but it
> might be difficult to make it generic enough.
>
> Julien
>
>  We are using cdash a lot, and for many of our tests the log is quite
>>> verbose and somewhat hard to read. When looking for the reason why a test
>>> failed, it can be somewhat hard to find the line which contains "ERROR".
>>> Also, a lot of stuff is repeated every line, which also makes the log
>>> harder to read, e.g. the directory part of the file where the log came
>>> from:
>>>
>>> <timestamp> /some/where/deep/in/a/subdir/**test.py: Something...
>>> <timestamp> /some/where/deep/in/a/subdir/**test.py: happend...
>>> <timestamp> /some/where/deep/in/a/subdir/**test.py: here...
>>> <timestamp> /some/where/deep/in/a/subdir/**test.py: ...
>>>
>>> The directory part is actually so wide that we have to scroll to see
>>> actual
>>> text.
>>>
>>> So, to help with this, we came up with the idea to use HTML to format the
>>> log so that it becomes easier to read.
>>>
>>> The attached patch for ctest adds a new test property GENERATE_HTML, and
>>> if
>>> this is enabled, ctest generates HTML. It searches the lines which made
>>> the
>>> test fail and colors them red, and the lines which make a test pass are
>>> colored green.
>>> This makes it easier to find the fail/succeed lines.
>>>
>>> (In the patch the fail strings are still hardcoded, this would have to
>>> use
>>> the respective test property.)
>>>
>>> On the cdash side, if "</span>" is found in the log, it is considered
>>> HTML
>>> and not escaped. I guess this should be done in a different way, which
>>> does not involve parsing the log text.
>>> For making our log lines shorter, we are not sure yet how to do that.
>>> We could add support for squish XML to ctest, so that ctest reads the
>>> squish XML and converts it to HTML which is in some way shorter, maybe a
>>> HTML table, or something.
>>> Or we could add a test property with a regular expression, if that regexp
>>> matches, ctest replaces it with a shorter version or something.
>>> Or we could modify our test scripts so that they already generate the
>>> pretty HTML, which ctest then simply sends to cdash.
>>> That's the ideas we have so far, but it doesn't feel quite right yet.
>>>
>>> So, what do you think ?
>>>
>>
>> Any comments ?
>>
>> Alex
>>
>>  ______________________________**_________________
> Cdash mailing list
> Cdash at public.kitware.com
> http://public.kitware.com/cgi-**bin/mailman/listinfo/cdash<http://public.kitware.com/cgi-bin/mailman/listinfo/cdash>
>



-- 
Zach Mullen
R & D Engineer
Kitware Inc.
919-869-8858
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cdash/attachments/20130503/710c8341/attachment-0002.htm>