[Cdash] [vtk-developers] CDash query for all failing tests

[David E DeMarle]

The second script does essentially the same thing that Dave Cole did years ago
Ie:
http://www.cmake.org/Wiki/CMake_Useful_Variables/Get_Variables_From_CMake_Dashboards
I assume that was in the dart days as it didn't work for me right off
the bat, so I decided to roll my own.

David E DeMarle
Kitware, Inc.
R&D Engineer
21 Corporate Drive
Clifton Park, NY 12065-8662
Phone: 518-881-4909


On Tue, Feb 12, 2013 at 9:02 PM, David E DeMarle
<dave.demarle at kitware.com> wrote:
> If you run vtk_fail_summary.py it will summarize all the test failures
> on the nightly expected group, from there you have to dig deeper than
> the cdash web api will let you go.
>
> To go down that route, take a look at vtk_submitter_summary.py. It
> scrapes the raw html pages to get the results of a particular
> test on every machine. The script and the accompanying test (once I finish and
> commit them) will give us the details of each submitter's configuration, so
> that we can inspect the testing space and know what configurations we
> are testing and what we aren't.
>
> David E DeMarle
> Kitware, Inc.
> R&D Engineer
> 21 Corporate Drive
> Clifton Park, NY 12065-8662
> Phone: 518-881-4909
>
>
> On Tue, Feb 12, 2013 at 4:04 PM, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>> Bill,
>>
>> Thanks. I knew how to do that. I was hoping I could see all the
>> failing test details. I'll have to do it by hand. Or maybe I can
>> figure out a way to scrape the cdash pages.
>>
>> Currently VTK has 18 Nightly Expected builds with test failures
>> (actually that is every build in that group). I'd like to generate a
>> page (or spreadsheet) that shows every failing test (today there are
>> 123 total tests failing).
>>
>> Bill
>>
>> On Tue, Feb 12, 2013 at 3:23 PM, Bill Hoffman <bill.hoffman at kitware.com> wrote:
>>> On 2/12/2013 1:53 PM, Bill Lorensen wrote:
>>>>
>>>> Folks,
>>>>
>>>> Is there a cdash query that can provide, on a single page, links to
>>>> all failing tests for a given group, say "Nightly Expected"?
>>>>
>>>> Bill
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at
>>>> http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> http://www.vtk.org/mailman/listinfo/vtk-developers
>>>>
>>>>
>>>
>>> You can do something like this:
>>>
>>> http://open.cdash.org/index.php?project=CMake&date=2013-02-12&filtercount=2&showfilters=1&filtercombine=and&field1=groupname/string&compare1=63&value1=Nightly%20Expected&field2=testsfailed/number&compare2=43&value2=0
>>>
>>> If you create a filter you can match the group name.  Unfortunately, you can
>>> not show errors and test failures.  You can only match all or any of the
>>> filters.   I think CDash needs to be able to have the any or all be applied
>>> to each of the filters so you can create a filter that will show you any red
>>> in a group.  But you can show all failing tests for a group.  :)
>>>
>>> -Bill
>>>
>>>
>>> --
>>> Bill Hoffman
>>> Kitware, Inc.
>>> 28 Corporate Drive
>>> Clifton Park, NY 12065
>>> bill.hoffman at kitware.com
>>> http://www.kitware.com
>>> 518 881-4905 (Direct)
>>> 518 371-3971 x105
>>> Fax (518) 371-4573
>>
>>
>>
>> --
>> Unpaid intern in BillsBasement at noware dot com
>> _______________________________________________
>> Cdash mailing list
>> Cdash at public.kitware.com
>> http://public.kitware.com/cgi-bin/mailman/listinfo/cdash