[vtk-developers] [ParaView3-Developer] Uploading full test images to CDash

Moreland, Kenneth kmorel at sandia.gov
Wed May 20 17:10:54 EDT 2009


OK.  I've made the suggested changes and a bit more cleanup.  The new patch is attached.  I engineered some failures and sent them to the dashboard.  You can take a look here:

http://www.cdash.org/CDash/viewTest.php?onlyfailed&buildid=336986

Unless there are further comments, I will check these in.

-Ken


On 5/20/09 7:31 AM, "Moreland, Kenneth" <kmorel at sandia.gov> wrote:

I made a few experimental dashboard runs with the previous patch yesterday.  Here are some dashboard entries for showing VTK Cxx failures, VTK Python failures, and ParaView failures, respectively.

http://www.cdash.org/CDash/testDetails.php?test=23401926&build=336271
http://www.cdash.org/CDash/testDetails.php?test=23402927&build=336276
http://www.cdash.org/CDash/viewTest.php?onlyfailed&buildid=336279

I'll make some changes when I get into the office today and set up some more dashboard runs.

-Ken


On 5/20/09 7:25 AM, "David Cole" <david.cole at kitware.com> wrote:

Let's do that and have you submit an Experimental (VTK or ParaView) dashboard with that change and give us something concrete to look at.

Maybe even force one of the tests on it to fail to see if you can then retrieve the test image from another machine through the CDash web interface...

"We have the technology."
:-)

Dave


On Wed, May 20, 2009 at 8:36 AM, Moreland, Kenneth <kmorel at sandia.gov> wrote:
I have not made all of the images full size, but that is an easy change to make.

-Ken



On 5/20/09 4:44 AM, "David Cole" <david.cole at kitware.com <http://david.cole@kitware.com> > wrote:

Go for it unless somebody has an objection. Since CDash hashes images, it will only store one copy of each image and many dashboards will reference each one. That eliminates the need for keeping the images sent to the server small.

The one comment I have is this: we may want to consider keeping all images the same size for a given test (valid, test, diff) so that your brain can visually compare them better. So we may want to make the other images full size as well. (Or does this change already do that? I don't think so, but I was only reading through the diff...)


Nice work...,
David


On Tue, May 19, 2009 at 7:08 PM, Moreland, Kenneth <kmorel at sandia.gov <http://kmorel@sandia.gov> > wrote:
Although I try to avoid it, occasionally I create or modify a test that requires multiple baseline images because different GPUs will generate images with enough differences.  Doing this is easy enough (just check in the extra baseline images), but I don't have access to most of the VTK and ParaView dashboards.  In order to get the image, I have to track down someone with access to the machine and ask them to check in the image for me.

In my mind it is backwards to have the dashboard maintainer making the decision on whether a test legitimately failed or has innocuous image differences.  Rather, the person creating the test should make the determination.  Thus, it makes sense for the image to be uploaded to the dashboard where everyone, most importantly the test coder, has access, can download a copy, and check in a new baseline.

I recently met some of the CMake/CDash developers and asked them for this feature.  They told me the functionality is already in CDash; I only need to instruct CTest to upload the files I want to show up.  As it turns out, with only a couple of small changes I can implement this in VTK (and by proxy in ParaView).  The attached VTK patch implements the full image upload.  Are there any objections to checking this in?

-Ken

   ****      Kenneth Moreland
    ***      Sandia National Laboratories
***********
*** *** ***  email: kmorel at sandia.gov <http://kmorel@sandia.gov>  <http://kmorel@sandia.gov>
**  ***  **  phone: (505) 844-8919
    ***      web:   http://www.cs.unm.edu/~kmorel <http://www.cs.unm.edu/%7Ekmorel>  <http://www.cs.unm.edu/%7Ekmorel>


_______________________________________________
ParaQ-Developer mailing list
ParaQ-Developer at paraview.org <http://ParaQ-Developer@paraview.org>
http://public.kitware.com/cgi-bin/mailman/listinfo/paraq-developer




   ****      Kenneth Moreland
    ***      Sandia National Laboratories
***********
*** *** ***  email: kmorel at sandia.gov <http://kmorel@sandia.gov>
**  ***  **  phone: (505) 844-8919
    ***      web:   http://www.cs.unm.edu/~kmorel <http://www.cs.unm.edu/%7Ekmorel>





   ****      Kenneth Moreland
    ***      Sandia National Laboratories
***********
*** *** ***  email: kmorel at sandia.gov
**  ***  **  phone: (505) 844-8919
    ***      web:   http://www.cs.unm.edu/~kmorel




   ****      Kenneth Moreland
    ***      Sandia National Laboratories
***********
*** *** ***  email: kmorel at sandia.gov
**  ***  **  phone: (505) 844-8919
    ***      web:   http://www.cs.unm.edu/~kmorel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20090520/13d8bc18/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CDashFullTestImages2.patch
Type: application/octet-stream
Size: 12003 bytes
Desc: CDashFullTestImages2.patch
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20090520/13d8bc18/attachment-0001.obj>


More information about the vtk-developers mailing list