[vtk-developers] VTK image error 0 despite differences - false pass

[Marcus D. Hanwell]

On Fri, May 4, 2018 at 1:59 PM, Marcus D. Hanwell <
marcus.hanwell at kitware.com> wrote:

> On Fri, May 4, 2018 at 1:24 PM, Marcus D. Hanwell <
> marcus.hanwell at kitware.com> wrote:
>
>> On Fri, May 4, 2018 at 9:34 AM, Marcus D. Hanwell <
>> marcus.hanwell at kitware.com> wrote:
>>
>>> A while ago I opened issue https://gitlab.kitware.com/vtk
>>> /vtk/issues/17201 that talked about a bug on Windows where setting the
>>> precision to 0 resulted in 6 precision on Windows. There was a test, but it
>>> was passing even on Windows, this week we had a new employee start and I
>>> thought it would be a good first bug to create a failing test, and then
>>> make it pass by addressing the issue.
>>>
>>> I thought that TestAxes was a little crowded, and that had perhaps been
>>> enough to throw the image diff off. So he wrote a test that just had this
>>> axis in, made a baseline for what was expected, and then changed it so that
>>> we could see the test fail. Changing to 6 or 8 is still showing passing
>>> test:
>>>
>>>
>>>
>>> This looks like a good difference to me (not sure if we are allowed
>>> attachments, but I tried). The ImageError is being shown as 0. We could
>>> make the text bigger, but there are a few chart tests where we use text of
>>> this size and I thought they would fail with extra numbers like this.
>>>
>>
>> Digging into this for anyone following, I can change vtkTesting.cxx to
>> use 'rtId->SetAllowShift(0);' which removes the single pixel shift logic,
>> and I get a diff showing my extra digits, as I would expect. We likely have
>> this on for a reason, and the text is pretty tiny, so I could see a nearest
>> neighbor search turning up no difference for every character added.
>>
>
> Lots of tests fail when this is turned off, we can actually make the text
> larger and see the expected failure in the axis test. We could redo the
> TestAxes to use a larger font, or perhaps add an option to disable the
> pixel shift on a per test basis (I didn't see anything exposing that in the
> test driver.
>

For anyone interested, a merge request with dashboard results looking at
the impact of not allowing shifts in image diffs globally,

https://gitlab.kitware.com/vtk/vtk/merge_requests/4261
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://vtk.org/pipermail/vtk-developers/attachments/20180504/87660989/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: TestAxes2.png
Type: image/png
Size: 1770 bytes
Desc: not available
URL: <https://vtk.org/pipermail/vtk-developers/attachments/20180504/87660989/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: TestAxes22.png
Type: image/png
Size: 1876 bytes
Desc: not available
URL: <https://vtk.org/pipermail/vtk-developers/attachments/20180504/87660989/attachment-0003.png>