[Insight-developers] [ITK Community] Script for Remote Module testing

Bradley Lowekamp blowekamp at mail.nih.gov
Thu Jan 30 12:56:21 EST 2014 

I just tried out Brian's script. I added some untested IO modules too:

http://open.cdash.org/viewNotes.php?buildid=3195689

With the new feature to not compile dependent modules' tests, it's very fast to configure an build on my system.

To run valgrind/coverage I suppose I could make it so it turns on all of these modules, to just bulk test them.

Brad

On Jan 30, 2014, at 12:43 PM, Bill Lorensen <bill.lorensen at gmail.com> wrote:

> Of course you can always set up cdash filters to present customized
> views and save those as bookmarks,
> 
> Bill
> 
> 
> On Thu, Jan 30, 2014 at 12:17 PM, Matt McCormick
> <matt.mccormick at kitware.com> wrote:
>> The ITKGroup_Remote would be nice to have.  Another idea is to use
>> similar code in itk_common.cmake and after the initial configure, and
>> submit with each module turned on.
>> 
>> -----
>> 
>> In terms of the modular dashboard, I agree that keeping an eye on the
>> goals is a good idea.  Example goals:
>> 
>>  1. People look at the dashboard.
>>  2. People can quickly see if anything went wrong, what went wrong,
>> and if they need to pay attention.
>> 
>> In practice, I think the previous modular dashboard hurt more than
>> helped with these goals.
>> 
>> Part of the problem was that it took too many clicks and too many long
>> CDash page loads to investigate the issues.
>> 
>> The other problem was that the hierarchical view made it difficult to
>> see what went wrong.  We have been working had to clean up the
>> dashboard so there is not so much noise that it is difficult to see
>> when any new problems occur.  It takes a very consistently clean
>> dashboard for the hierarchical view to add value.
>> 
>> The current page is getting somewhat long, but I personally don't have
>> much difficulty scrolling through.  If it starts to get worse, which
>> is a valid concern, I think something like javascript
>> hiding/re-presentation of the data, which is done locally, would be a
>> better solution than something that requires repeated CDash database
>> lookups.
>> 
>> My 2 cents,
>> Matt
>> 
>> On Wed, Jan 29, 2014 at 2:17 PM, Xiaoxiao Liu <xiaoxiao.liu at kitware.com> wrote:
>>> As a temporary solution, how about creating a CMake variable
>>> "ITKGroup_Remote" to turn on
>>> all remote modules for dashboard testing purpose?:
>>>   http://review.source.kitware.com/#/c/14201/  (a quick prototype just to
>>> show the idea)
>>> 
>>> If you turn ITK_BUILD_DEFAULT_MODULES OFF and turn this ITKGroup_Remote ON,
>>> all
>>> remote modules would be ON.
>>> 
>>> This will make things easier to setup remote dashboards, with all the remote
>>> module tests
>>> included in a single dashboard (along with all the depending modules'
>>> tests).
>>> 
>>> 
>>> 
>>> On Wed, Jan 29, 2014 at 11:50 AM, Julien Jomier <julien.jomier at kitware.com>
>>> wrote:
>>>> 
>>>> Bill, Brad,
>>>> 
>>>> I'm also happy to "tune" CDash if necessary.
>>>> 
>>>> Julien
>>>> 
>>>> 
>>>> On 29/01/2014 17:29, Bill Lorensen wrote:
>>>>> 
>>>>> One alternative is to submit to a different (non-modular) dashboard.
>>>>> 
>>>>> 
>>>>> On Wed, Jan 29, 2014 at 11:28 AM, Bill Lorensen <bill.lorensen at gmail.com>
>>>>> wrote:
>>>>>> 
>>>>>> I agree we need something better. I also believe it is not the modular
>>>>>> dashboard.
>>>>>> 
>>>>>> Maybe we should establish requirements and see how best to meet them.
>>>>>> 
>>>>>> 
>>>>>> On Wed, Jan 29, 2014 at 10:53 AM, Bradley Lowekamp
>>>>>> <blowekamp at mail.nih.gov> wrote:
>>>>>>> 
>>>>>>> Bill,
>>>>>>> 
>>>>>>> If I add 3 machines to loop through the additional modules, do you
>>>>>>> think it would get too cluttered and un-organized to find the modules with
>>>>>>> problems or the modules you are interested in? How do you find the Slicer
>>>>>>> dashboard for it's modules/extensions?
>>>>>>> 
>>>>>>> Brad
>>>>>>> 
>>>>>>> 
>>>>>>> On Jan 29, 2014, at 10:23 AM, Bill Lorensen <bill.lorensen at gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> I prefer keeping the remote module where they are.
>>>>>>>> 
>>>>>>>> On Wed, Jan 29, 2014 at 10:15 AM, Jean-Christophe Fillion-Robin
>>>>>>>> <jchris.fillionr at kitware.com> wrote:
>>>>>>>>> 
>>>>>>>>> +1
>>>>>>>>> 
>>>>>>>>> As a side note, we gonna also look at the modular dashboard for
>>>>>>>>> Slicer
>>>>>>>>> extension. I think it makes complete sense from an extension or ITK
>>>>>>>>> module
>>>>>>>>> developer perspective.
>>>>>>>>> 
>>>>>>>>> Please, keep us posted of your findings..
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Wed, Jan 29, 2014 at 8:58 AM, Bradley Lowekamp
>>>>>>>>> <blowekamp at mail.nih.gov>
>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Bill,
>>>>>>>>>> 
>>>>>>>>>> That is what I recall from it too. But I was looking at it from an
>>>>>>>>>> ITK
>>>>>>>>>> developers perspective. I wanted to get the whole ITK view the way I
>>>>>>>>>> was use
>>>>>>>>>> to from the normal dashboard. I am not suggesting replacing the
>>>>>>>>>> current
>>>>>>>>>> view, well just the moving the "Expected Nightly Remote Modules"
>>>>>>>>>> there as I
>>>>>>>>>> think the current layout won't scale.
>>>>>>>>>> 
>>>>>>>>>> As a modular developer the modular dashboard might be the right
>>>>>>>>>> view. I
>>>>>>>>>> would get a line for my module and I may be able to drill down for
>>>>>>>>>> that
>>>>>>>>>> module. I don't see a live version of it anymore to check though.
>>>>>>>>>> 
>>>>>>>>>> I was just thinking that they might compliment each other for these
>>>>>>>>>> two
>>>>>>>>>> cases.
>>>>>>>>>> 
>>>>>>>>>> Brad
>>>>>>>>>> 
>>>>>>>>>> On Jan 29, 2014, at 8:44 AM, Bill Lorensen <bill.lorensen at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> I do not recommend the modular dashboard. It has severe navigation
>>>>>>>>>>> issues.
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, Jan 29, 2014 at 8:36 AM, Bradley Lowekamp
>>>>>>>>>>> <blowekamp at mail.nih.gov> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Hello,
>>>>>>>>>>>> 
>>>>>>>>>>>> I'm lazy and want to reduce the maintenance burden on creating a
>>>>>>>>>>>> nightly build to test the remote modules.
>>>>>>>>>>>> 
>>>>>>>>>>>> It looks like the script running on "dash5.kitware" is nicely
>>>>>>>>>>>> grinding
>>>>>>>>>>>> through some remote modules ( and potentially other off by default
>>>>>>>>>>>> modules
>>>>>>>>>>>> like DCMTK, MINC). I was about to suggest this be some how added
>>>>>>>>>>>> into the
>>>>>>>>>>>> "dashboard" branch, but now I am seeing the
>>>>>>>>>>>> itkmodular_common.cmake script
>>>>>>>>>>>> there, which really has a lot of the functionality need such as
>>>>>>>>>>>> some support
>>>>>>>>>>>> for valgrind and coverage.
>>>>>>>>>>>> 
>>>>>>>>>>>> Does it make since to bring back the modular dashboard for these
>>>>>>>>>>>> other
>>>>>>>>>>>> modules?
>>>>>>>>>>>> 
>>>>>>>>>>>> I really just want to be able to specify a list for modules to be
>>>>>>>>>>>> modularly tested on a a couple builds and try to re-use a script
>>>>>>>>>>>> to do this.
>>>>>>>>>>>> I am trying to figure out what's the easiest way to do this for
>>>>>>>>>>>> the long
>>>>>>>>>>>> run.
>>>>>>>>>>>> 
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Brad
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Powered by www.kitware.com
>>>>>>>>>>>> 
>>>>>>>>>>>> Visit other Kitware open-source projects at
>>>>>>>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>>>>>>> 
>>>>>>>>>>>> Kitware offers ITK Training Courses, for more information visit:
>>>>>>>>>>>> http://kitware.com/products/protraining.php
>>>>>>>>>>>> 
>>>>>>>>>>>> Please keep messages on-topic and check the ITK FAQ at:
>>>>>>>>>>>> http://www.itk.org/Wiki/ITK_FAQ
>>>>>>>>>>>> 
>>>>>>>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>>>>>>>> http://www.itk.org/mailman/listinfo/insight-developers
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Community mailing list
>>>>>>>>>>>> Community at itk.org
>>>>>>>>>>>> http://public.kitware.com/cgi-bin/mailman/listinfo/community
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> Unpaid intern in BillsBasement at noware dot com
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Powered by www.kitware.com
>>>>>>>>>> 
>>>>>>>>>> Visit other Kitware open-source projects at
>>>>>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>>>>> 
>>>>>>>>>> Kitware offers ITK Training Courses, for more information visit:
>>>>>>>>>> http://kitware.com/products/protraining.php
>>>>>>>>>> 
>>>>>>>>>> Please keep messages on-topic and check the ITK FAQ at:
>>>>>>>>>> http://www.itk.org/Wiki/ITK_FAQ
>>>>>>>>>> 
>>>>>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>>>>>> http://www.itk.org/mailman/listinfo/insight-developers
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> +1 919 869 8849
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Unpaid intern in BillsBasement at noware dot com
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Unpaid intern in BillsBasement at noware dot com
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>> 
>>>> Visit other Kitware open-source projects at
>>>> http://www.kitware.com/opensource/opensource.html
>>>> 
>>>> Kitware offers ITK Training Courses, for more information visit:
>>>> http://kitware.com/products/protraining.php
>>>> 
>>>> Please keep messages on-topic and check the ITK FAQ at:
>>>> http://www.itk.org/Wiki/ITK_FAQ
>>>> 
>>>> Follow this link to subscribe/unsubscribe:
>>>> http://www.itk.org/mailman/listinfo/insight-developers
>>>> _______________________________________________
>>>> Community mailing list
>>>> Community at itk.org
>>>> http://public.kitware.com/cgi-bin/mailman/listinfo/community
>>> 
>>> 
>>> 
>>> 
>>> --
>>> 
>>> 
>>> ---------------------------------------------
>>> Xiaoxiao Liu, Ph.D.
>>> R & D Engineer
>>> Kitware Inc.
>>> Clifton Park, NY
>>> Phone: (518) 881-4924  or  (518) 371-3971 x124
>>> 
>>> 
>>> _______________________________________________
>>> Powered by www.kitware.com
>>> 
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>> 
>>> Kitware offers ITK Training Courses, for more information visit:
>>> http://kitware.com/products/protraining.php
>>> 
>>> Please keep messages on-topic and check the ITK FAQ at:
>>> http://www.itk.org/Wiki/ITK_FAQ
>>> 
>>> Follow this link to subscribe/unsubscribe:
>>> http://www.itk.org/mailman/listinfo/insight-developers
>>> 
>> _______________________________________________
>> Powered by www.kitware.com
>> 
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>> 
>> Kitware offers ITK Training Courses, for more information visit:
>> http://kitware.com/products/protraining.php
>> 
>> Please keep messages on-topic and check the ITK FAQ at:
>> http://www.itk.org/Wiki/ITK_FAQ
>> 
>> Follow this link to subscribe/unsubscribe:
>> http://www.itk.org/mailman/listinfo/insight-developers
> 
> 
> 
> -- 
> Unpaid intern in BillsBasement at noware dot com
> _______________________________________________
> Powered by www.kitware.com
> 
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
> 
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.php
> 
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
> 
> Follow this link to subscribe/unsubscribe:
> http://www.itk.org/mailman/listinfo/insight-developers

More information about the Insight-developers mailing list