<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from rtf -->
<style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<font face="Calibri" size="2"><span style="font-size:11pt;">
<div>The limitation of using FAIL_REGULAR_EXPRESSION CTest property for catching errors is that it's very difficult (if at all possible) to catch unexpected errors when there are expected errors as well. </div>
<div> </div>
<div>When you use vtkTestingOutputWindow class then the default behavior is to return with failure at the end of the test if warnings/errors have been logged; but you can easily mark sections of code that are expected/allowed to log errors. See some examples
below.</div>
<div> </div>
<div>Andras</div>
<div> </div>
<div> </div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">// the test driver calls TESTING_OUTPUT_INIT(), which instantiates vtkTestingOutputWindow to capture all log messages</span></font></div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">int simpleTestThatShouldNotLogErrors(...)</span></font></div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">{</span></font></div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">...perform operations that must_not log error or warning messages - no changes in the test are necessary, initialization and returning with failure are provided by the test driver</span></font></div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">}</span></font></div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">// the test driver calls TESTING_OUTPUT_ASSERT_WARNINGS_ERRORS(0), which returns with EXIT_FAILURE in case there were warnings or errors</span></font></div>
<div> </div>
<div> </div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">int complexTestThatHasExpectedAnUnexpectedErrors(...)</span></font></div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">{</span></font></div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">...perform operations that must_not log error messages</span></font></div>
<div> </div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">TESTING_OUTPUT_IGNORE_WARNINGS_ERRORS_BEGIN();</span></font></div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">...perform operations that are allowed to log errors and warnings</span></font></div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">TESTING_OUTPUT_IGNORE_WARNINGS_ERRORS_END();</span></font></div>
<div> </div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">...perform operations that must_not log error messages</span></font></div>
<div> </div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">TESTING_OUTPUT_ASSERT_WARNINGS_BEGIN();</span></font></div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">...perform operations that must log at least one warning message</span></font></div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">TESTING_OUTPUT_ASSERT_WARNINGS_END();</span></font></div>
<div> </div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">...perform operations that must_not log error messages</span></font></div>
<div><font face="Courier New" size="2"><span style="font-size:9pt;">}</span></font></div>
<div> </div>
<div>-----Original Message-----<br>
From: Ben Boeckel [<a href="mailto:ben.boeckel@kitware.com">mailto:ben.boeckel@kitware.com</a>]
<br>
Sent: October 26, 2015 10:33 PM<br>
To: Andras Lasso <lasso@queensu.ca><br>
Cc: Bill Lorensen <bill.lorensen@gmail.com>; VTK Developers <vtk-developers@vtk.org><br>
Subject: Re: [vtk-developers] vtkRenderingOpenGL2Cxx-TestDepthOfFieldPass ERRORs</div>
<div> </div>
<div>On Tue, Oct 27, 2015 at 01:43:59 +0000, Andras Lasso wrote:</div>
<div>> The mechanism that I wrote about is useful for catching all </div>
<div>> errors/warnings logged by any class during any test. It can detect </div>
<div>> real errors that might otherwise go unnoticed.</div>
<div> </div>
<div>The branch handles this by failing the test (via a CTest property) if ERROR is in its output.</div>
<div> </div>
<div>> Grabbing errors with observer works well for verifying that expected </div>
<div>> error events are invoked, but it will not detect unexpected errors.</div>
<div>> Parsing the test output with a regexp works for catching any </div>
<div>> unexpected errors, but it is difficult to manage when there are </div>
<div>> expected errors, too.</div>
<div> </div>
<div>The idea is to capture expected errors via vtkErrorObserver and verify that they are the expected error. Anything not expected there or not caught triggers the CTest property to have it fail on ERROR output. Is this not the same outcome?</div>
<div> </div>
<div>--Ben</div>
<div> </div>
</span></font>
</body>
</html>