[Insight-developers] Testing data addition guidelines

[Brad King]

On 9/12/2011 3:06 PM, Xiaoxiao Liu wrote:
> Hi Michael,
> You could find the new test data submission work flow at http://www.vtk.org/Wiki/ITK/Git/Develop/Data#Workflow.
> As specified in the wiki, the new testing submission framework allows arbitrary large
> data submissions and removals via the data content links (the hash files).

Xiaoxiao, I think Michael was asking about whether use of reference images
is appropriate for his tests, not about how to actually add the image data.

> On Mon, Sep 12, 2011 at 1:13 PM, M Stauffer (V) <mstauff at verizon.net <mailto:mstauff at verizon.net>> wrote:
>
>     With the new testing data framework, what are the guidelines now for
>     submitting test data?

The new framework overcomes technical limitations that previously prevented
data from being added in certain cases (e.g. very large files).  With those
limitations out of the way we have not written any new guidelines AFAIK.

 >     If I remember right, part of the goal of the new
>     framework was to allow more testing data submission?

Yes.

>     We'll be putting
>     together a number of tests that could use reference images for comparing
>     against test output, or they could use a hard-coded image metric value
>     to evaluate the output. The latter would use less testing data memory,
>     but sacrifice a central historical record of the reference image itself.

Many of our tests use manually verified "baseline" images.  This is encouraged.
If you think they are of value to your test then do not hesitate to add them.

-Brad