[Insight-developers] [ITK Community] Script for Remote Module testing

Thu Jan 30 12:17:20 EST 2014

The ITKGroup_Remote would be nice to have.  Another idea is to use
similar code in itk_common.cmake and after the initial configure, and
submit with each module turned on.

-----

In terms of the modular dashboard, I agree that keeping an eye on the
goals is a good idea.  Example goals:

  1. People look at the dashboard.
  2. People can quickly see if anything went wrong, what went wrong,
and if they need to pay attention.

In practice, I think the previous modular dashboard hurt more than
helped with these goals.

Part of the problem was that it took too many clicks and too many long
CDash page loads to investigate the issues.

The other problem was that the hierarchical view made it difficult to
see what went wrong.  We have been working had to clean up the
dashboard so there is not so much noise that it is difficult to see
when any new problems occur.  It takes a very consistently clean
dashboard for the hierarchical view to add value.

The current page is getting somewhat long, but I personally don't have
much difficulty scrolling through.  If it starts to get worse, which
is a valid concern, I think something like javascript
hiding/re-presentation of the data, which is done locally, would be a
better solution than something that requires repeated CDash database
lookups.

My 2 cents,
Matt

On Wed, Jan 29, 2014 at 2:17 PM, Xiaoxiao Liu <xiaoxiao.liu at kitware.com> wrote:
> As a temporary solution, how about creating a CMake variable
> "ITKGroup_Remote" to turn on
> all remote modules for dashboard testing purpose?:
>    http://review.source.kitware.com/#/c/14201/  (a quick prototype just to
> show the idea)
>
> If you turn ITK_BUILD_DEFAULT_MODULES OFF and turn this ITKGroup_Remote ON,
> all
> remote modules would be ON.
>
> This will make things easier to setup remote dashboards, with all the remote
> module tests
> included in a single dashboard (along with all the depending modules'
> tests).
>
>
>
> On Wed, Jan 29, 2014 at 11:50 AM, Julien Jomier <julien.jomier at kitware.com>
> wrote:
>>
>> Bill, Brad,
>>
>> I'm also happy to "tune" CDash if necessary.
>>
>> Julien
>>
>>
>> On 29/01/2014 17:29, Bill Lorensen wrote:
>>>
>>> One alternative is to submit to a different (non-modular) dashboard.
>>>
>>>
>>> On Wed, Jan 29, 2014 at 11:28 AM, Bill Lorensen <bill.lorensen at gmail.com>
>>> wrote:
>>>>
>>>> I agree we need something better. I also believe it is not the modular
>>>> dashboard.
>>>>
>>>> Maybe we should establish requirements and see how best to meet them.
>>>>
>>>>
>>>> On Wed, Jan 29, 2014 at 10:53 AM, Bradley Lowekamp
>>>> <blowekamp at mail.nih.gov> wrote:
>>>>>
>>>>> Bill,
>>>>>
>>>>> If I add 3 machines to loop through the additional modules, do you
>>>>> think it would get too cluttered and un-organized to find the modules with
>>>>> problems or the modules you are interested in? How do you find the Slicer
>>>>> dashboard for it's modules/extensions?
>>>>>
>>>>> Brad
>>>>>
>>>>>
>>>>> On Jan 29, 2014, at 10:23 AM, Bill Lorensen <bill.lorensen at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I prefer keeping the remote module where they are.
>>>>>>
>>>>>> On Wed, Jan 29, 2014 at 10:15 AM, Jean-Christophe Fillion-Robin
>>>>>> <jchris.fillionr at kitware.com> wrote:
>>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>> As a side note, we gonna also look at the modular dashboard for
>>>>>>> Slicer
>>>>>>> extension. I think it makes complete sense from an extension or ITK
>>>>>>> module
>>>>>>> developer perspective.
>>>>>>>
>>>>>>> Please, keep us posted of your findings..
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jan 29, 2014 at 8:58 AM, Bradley Lowekamp
>>>>>>> <blowekamp at mail.nih.gov>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> Bill,
>>>>>>>>
>>>>>>>> That is what I recall from it too. But I was looking at it from an
>>>>>>>> ITK
>>>>>>>> developers perspective. I wanted to get the whole ITK view the way I
>>>>>>>> was use
>>>>>>>> to from the normal dashboard. I am not suggesting replacing the
>>>>>>>> current
>>>>>>>> view, well just the moving the "Expected Nightly Remote Modules"
>>>>>>>> there as I
>>>>>>>> think the current layout won't scale.
>>>>>>>>
>>>>>>>> As a modular developer the modular dashboard might be the right
>>>>>>>> view. I
>>>>>>>> would get a line for my module and I may be able to drill down for
>>>>>>>> that
>>>>>>>> module. I don't see a live version of it anymore to check though.
>>>>>>>>
>>>>>>>> I was just thinking that they might compliment each other for these
>>>>>>>> two
>>>>>>>> cases.
>>>>>>>>
>>>>>>>> Brad
>>>>>>>>
>>>>>>>> On Jan 29, 2014, at 8:44 AM, Bill Lorensen <bill.lorensen at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I do not recommend the modular dashboard. It has severe navigation
>>>>>>>>> issues.
>>>>>>>>>
>>>>>>>>> On Wed, Jan 29, 2014 at 8:36 AM, Bradley Lowekamp
>>>>>>>>> <blowekamp at mail.nih.gov> wrote:
>>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> I'm lazy and want to reduce the maintenance burden on creating a
>>>>>>>>>> nightly build to test the remote modules.
>>>>>>>>>>
>>>>>>>>>> It looks like the script running on "dash5.kitware" is nicely
>>>>>>>>>> grinding
>>>>>>>>>> through some remote modules ( and potentially other off by default
>>>>>>>>>> modules
>>>>>>>>>> like DCMTK, MINC). I was about to suggest this be some how added
>>>>>>>>>> into the
>>>>>>>>>> "dashboard" branch, but now I am seeing the
>>>>>>>>>> itkmodular_common.cmake script
>>>>>>>>>> there, which really has a lot of the functionality need such as
>>>>>>>>>> some support
>>>>>>>>>> for valgrind and coverage.
>>>>>>>>>>
>>>>>>>>>> Does it make since to bring back the modular dashboard for these
>>>>>>>>>> other
>>>>>>>>>> modules?
>>>>>>>>>>
>>>>>>>>>> I really just want to be able to specify a list for modules to be
>>>>>>>>>> modularly tested on a a couple builds and try to re-use a script
>>>>>>>>>> to do this.
>>>>>>>>>> I am trying to figure out what's the easiest way to do this for
>>>>>>>>>> the long
>>>>>>>>>> run.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Brad
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Powered by www.kitware.com
>>>>>>>>>>
>>>>>>>>>> Visit other Kitware open-source projects at
>>>>>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>>>>>
>>>>>>>>>> Kitware offers ITK Training Courses, for more information visit:
>>>>>>>>>> http://kitware.com/products/protraining.php
>>>>>>>>>>
>>>>>>>>>> Please keep messages on-topic and check the ITK FAQ at:
>>>>>>>>>> http://www.itk.org/Wiki/ITK_FAQ
>>>>>>>>>>
>>>>>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>>>>>> http://www.itk.org/mailman/listinfo/insight-developers
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Community mailing list
>>>>>>>>>> Community at itk.org
>>>>>>>>>> http://public.kitware.com/cgi-bin/mailman/listinfo/community
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Unpaid intern in BillsBasement at noware dot com
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Powered by www.kitware.com
>>>>>>>>
>>>>>>>> Visit other Kitware open-source projects at
>>>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>>>
>>>>>>>> Kitware offers ITK Training Courses, for more information visit:
>>>>>>>> http://kitware.com/products/protraining.php
>>>>>>>>
>>>>>>>> Please keep messages on-topic and check the ITK FAQ at:
>>>>>>>> http://www.itk.org/Wiki/ITK_FAQ
>>>>>>>>
>>>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>>>> http://www.itk.org/mailman/listinfo/insight-developers
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> +1 919 869 8849
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Unpaid intern in BillsBasement at noware dot com
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Unpaid intern in BillsBasement at noware dot com
>>>
>>>
>>>
>>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Kitware offers ITK Training Courses, for more information visit:
>> http://kitware.com/products/protraining.php
>>
>> Please keep messages on-topic and check the ITK FAQ at:
>> http://www.itk.org/Wiki/ITK_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.itk.org/mailman/listinfo/insight-developers
>> _______________________________________________
>> Community mailing list
>> Community at itk.org
>> http://public.kitware.com/cgi-bin/mailman/listinfo/community
>
>
>
>
> --
>
>
> ---------------------------------------------
> Xiaoxiao Liu, Ph.D.
> R & D Engineer
> Kitware Inc.
> Clifton Park, NY
> Phone: (518) 881-4924  or  (518) 371-3971 x124
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.php
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.itk.org/mailman/listinfo/insight-developers
>