[vtk-developers] cdash API and VTK's most wanted list

Andrew Maclean andrew.amaclean at gmail.com
Sat Oct 6 18:22:13 EDT 2012


David,

What a useful script! I will be using this after every merge, that's for
sure.
Thanks for your efforts.
If you get a testing space coverage done that will be really useful.

The useful thing about these scripts is they take such a short time to run.

Hey Bill, I remember the early dashboards, they were so concise and
succinct. I tried for a long time to convince my work to use a similar
process but it took a long time for people to realise the benefits of such
an approach.

Regards
   Andrew

---------- Forwarded message ----------

> From: David E DeMarle <dave.demarle at kitware.com>
> To: Bill Lorensen <bill.lorensen at gmail.com>
> Cc: vtkdev <vtk-developers at vtk.org>, Kitware Developers <
> kitware-developers at kitware.com>
> Date: Fri, 5 Oct 2012 15:49:10 -0400
> Subject: Re: [vtk-developers] cdash API and VTK's most wanted list
> Thanks for the slides they are quite informative.
>
> I'm hoping to find time to use cdash's web API to scrape out a testing
> space coverage view.
>
> That is I want to see what OS's, compilers and graphics drivers do we
> test on, as well as what wrapped languages, dependent libraries, and
> other cmake options do we exercise on them.
>
> >From a view of the test option space we can see what we cover and more
> importantly don't cover, and on what we cover, where we are having
> problems.
>
> Like the test view, you _can_ build up this information manually from
> cdash's pages, but it takes even longer to do so because the
> information is more widely scattered (at least for VTK).
>
> David E DeMarle
> Kitware, Inc.
> R&D Engineer
> 21 Corporate Drive
> Clifton Park, NY 12065-8662
> Phone: 518-881-4909
>
>
> On Fri, Oct 5, 2012 at 3:31 PM, Bill Lorensen <bill.lorensen at gmail.com>
> wrote:
> > Dave,
> >
> > Very nice. Back in the day, we used to generate a table with a row for
> each
> > test and a column for each build. A red/green box showed pass/fail. Made
> it
> > easy to see if it was a windows/linux/sun/hp/sgi... issue.
> >
> > I'll look into TestXMLFIleOutputWindow which seems to be failing
> strangely
> > on all windows platform (except cygwin).
> >
> > For a description of the eary origins of testing for vtk you might look
> at:
> > http://www.cdash.org/cdash/project/download/ExtremeTestingTalk.pdf
> >
> >
> > On Fri, Oct 5, 2012 at 2:38 PM, David E DeMarle <
> dave.demarle at kitware.com>
> > wrote:
> >>
> >> Howdy y'all,
> >>
> >> I've been working on cleaning the VTK dashboard lately and recently
> >> learned something that is really useful - that CDash's Web API is
> >> really powerful.
> >>
> >> See: http://public.kitware.com/Wiki/CDash:API
> >>
> >> Using it I wrote the attached python script which scrapes last night's
> >> VTK test results and presents them in a test centric view, rather than
> >> cdash's native submitter centric view.
> >>
> >> The test centric view is more useful I think because it helps you to
> >> prioritize what _code_ or project feature is worst and should be fixed
> >> soonest, as opposed to what machine is performing worst. It also gives
> >> you a more realistic view of the state of the project, because it
> >> won't make you falsely believe, for instance, that there are at most
> >> the same number of problems as the worst dashboard shows. (20 or so vs
> >> 70 today)
> >>
> >> Also, since it is a script it takes less than a second to generate the
> >> report, whereas if you build up the same list on your own manually by
> >> clicking on cdash's web pages it will take you about an hour on a
> >> project (and project state) like VTK today.
> >>
> >> If you run the script you will see that the worst tests are:
> >>
> >> vtkIOExportCxx-TestGL2PSExporterVolumeRaster-VerifyRasterizedPNG , 6 ,
> >> ['Mac10.7.3-clang-release-x86_64-nightly_master', 'Win32-vs9-Shared',
> >> 'Win64-VS10', 'Mac10.7.3-gcc-release-x86_64-nightly_master',
> >> 'Win32-vs9-Static', 'Ubuntu-Valgrind']
> >> vtkInteractionStyleTcl-TestInteractorStyleTerrain , 7 , [...]
> >> vtkRenderingOpenGLCxx-TestGaussianBlurPass , 7 , [...]
> >> vtkCommonCore-TestXMLFileOutputWindow , 7 , [...]
> >> vtkRenderingOpenGLCxx-TestTranslucentLUTDepthPeelingPass , 7 , [...]
> >> ! vtkRenderingVolumeTcl-gaussian , 10 , [...]
> >> ! vtkIOGeometryTcl-TestXYZMol , 10 , [...]
> >> vtkIOSQLCxx-SQLiteTableReadWrite , 10 , [...]
> >> vtkRenderingOpenGLCxx-TestOffAxisStereo , 14 , [...]
> >> ! vtkDomainsChemistryCxx-TestProteinRibbon , 19 , [...]
> >>
> >> Where the ones I've marked with ! have been fixed and thus should be
> >> off the most listed list tonight.
> >>
> >> David E DeMarle
> >> Kitware, Inc.
> >> R&D Engineer
> >> 21 Corporate Drive
> >> Clifton Park, NY 12065-8662
> >> Phone: 518-881-4909
> >>
> >> _______________________________________________
> >> Powered by www.kitware.com
> >>
> >> Visit other Kitware open-source projects at
> >> http://www.kitware.com/opensource/opensource.html
> >>
> >> Follow this link to subscribe/unsubscribe:
> >> http://www.vtk.org/mailman/listinfo/vtk-developers
> >>
> >>
> >
> >
> >
> > --
> > Unpaid intern in BillsBasement at noware dot com
> >
>
> --
___________________________________________
Andrew J. P. Maclean

___________________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20121007/c2d898ab/attachment.html>


More information about the vtk-developers mailing list