[vtk-developers] vtkRenderingOpenGL2Cxx-TestDepthOfFieldPass ERRORs

Andras Lasso lasso at queensu.ca
Mon Oct 26 23:42:20 EDT 2015


The limitation of using FAIL_REGULAR_EXPRESSION CTest property for catching errors is that it's very difficult (if at all possible) to catch unexpected errors when there are expected errors as well.

When you use vtkTestingOutputWindow class then the default behavior is to return with failure at the end of the test if warnings/errors have been logged; but you can easily mark sections of code that are expected/allowed to log errors. See some examples below.

Andras


// the test driver calls TESTING_OUTPUT_INIT(), which instantiates vtkTestingOutputWindow to capture all log messages
int simpleTestThatShouldNotLogErrors(...)
{
...perform operations that must_not log error or warning messages - no changes in the test are necessary, initialization and returning with failure are provided by the test driver
}
// the test driver calls TESTING_OUTPUT_ASSERT_WARNINGS_ERRORS(0), which returns with EXIT_FAILURE in case there were warnings or errors


int complexTestThatHasExpectedAnUnexpectedErrors(...)
{
...perform operations that must_not log error messages

TESTING_OUTPUT_IGNORE_WARNINGS_ERRORS_BEGIN();
...perform operations that are allowed to log errors and warnings
TESTING_OUTPUT_IGNORE_WARNINGS_ERRORS_END();

...perform operations that must_not log error messages

TESTING_OUTPUT_ASSERT_WARNINGS_BEGIN();
...perform operations that must log at least one warning message
TESTING_OUTPUT_ASSERT_WARNINGS_END();

...perform operations that must_not log error messages
}

-----Original Message-----
From: Ben Boeckel [mailto:ben.boeckel at kitware.com]
Sent: October 26, 2015 10:33 PM
To: Andras Lasso <lasso at queensu.ca>
Cc: Bill Lorensen <bill.lorensen at gmail.com>; VTK Developers <vtk-developers at vtk.org>
Subject: Re: [vtk-developers] vtkRenderingOpenGL2Cxx-TestDepthOfFieldPass ERRORs

On Tue, Oct 27, 2015 at 01:43:59 +0000, Andras Lasso wrote:
> The mechanism that I wrote about is useful for catching all
> errors/warnings logged by any class during any test. It can detect
> real errors that might otherwise go unnoticed.

The branch handles this by failing the test (via a CTest property) if ERROR is in its output.

> Grabbing errors with observer works well for verifying that expected
> error events are invoked, but it will not detect unexpected errors.
> Parsing the test output with a regexp works for catching any
> unexpected errors, but it is difficult to manage when there are
> expected errors, too.

The idea is to capture expected errors via vtkErrorObserver and verify that they are the expected error. Anything not expected there or not caught triggers the CTest property to have it fail on ERROR output. Is this not the same outcome?

--Ben

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20151027/ff358586/attachment.html>


More information about the vtk-developers mailing list