[Insight-developers] Change in ITK[master]: COMP: Intermittent test failure

[Bill Lorensen]

Ziv,

The problem with the test was that sometimes it was terminating on
iterations. When this happened, the tolerance was not always satisfied.

I upped the number of max iterations and I think the test should pass
conosistently.

We'll see.

Usually for non-deterministic algorithms the test seeds the random number
generator with a fixed number.

On Thu, Sep 15, 2011 at 8:43 PM, Yaniv, Ziv <ZYaniv at childrensnational.org>wrote:

> **
>
> Hi Bill,
>
> I was traveling and did not have internet connection for a day. Back
> online. The PSO family of algorithms is stochastic and does not ensure
> convergence to the local minimum. My initial implementation of the tests
> dealt with this by running them multiple times and if a certain percentage
> was successful then the test was declared successful. Perhaps this is not
> the best way to test such algorithms but I do not have a better idea. The
> nice feature of PSO is that it efficiently explores a rugged parameter
> space. A common practice is to run the PSO and then initialize a standard
> optimizer with the previous result, ensuring converges to the local minimum.
>
> What approach does ITK use to tests non-deterministic algorithms?
>
>               regards
>                 Ziv
>
> --
> Ziv Yaniv, PhD.,
> Principal Investigator,
> The Sheikh Zayed Institute for Pediatric Surgical Innovation
> Children's National Medical Center
> 6th Floor Main Hospital, Room M7740
> 111 Michigan Ave., N.W
> Washington, D.C 20010
> Phone: 202-476-1288
> email: zyaniv at childrensnational.org
>
>
>
>
>
> -----Original Message-----
> From: Code Review [mailto:review at kitware.com <review at kitware.com>]
> Sent: Thu 9/15/2011 6:25 PM
> To: Bill Lorensen
> Cc: Matt McCormick; Hans J. Johnson; Yaniv, Ziv
> Subject: Change in ITK[master]: COMP: Intermittent test failure
>
> From Kitware Robot <kwrobot at kitware.com>:
>
> Kitware Robot has submitted this change and it was merged.
>
> Change subject: COMP: Intermittent test failure
> ......................................................................
>
>
> COMP: Intermittent test failure
>
> itkInitializationBiasedParticleSwarmOptimizerTest is failing
> intermittenly. This patch enables more verbose (but not too verbose)
> output to help track down the problem.
>
> It seems the test was failing a tolerance check on the results of the
> optimization. This was due to hitting the max number of iterations
> stop condition rather than the parameter tolerance stop
> condition. This patch increases the max number of iterations, forcing
> a parameter tolerance stop condition.
>
> It is not clear whether this is a deterministic algorithm. Should we
> expect consistent answers for each run?
>
> This patch does not address the deterministic issue.
>
> Change-Id: Ie2110c51639f491cb7b8a6287b0eaeb1370c50bd
> ---
> M
> Modules/Numerics/Optimizers/test/itkInitializationBiasedParticleSwarmOptimizerTest.cxx
> 1 file changed, 45 insertions(+), 90 deletions(-)
>
> Approvals:
>   Hans J. Johnson: Verified; Looks good to me, approved
>   Bill Lorensen: Verified; Looks good to me, but someone else must approve
>
>
> --
> To view, visit http://review.source.kitware.com/2793
> To unsubscribe, visit http://review.source.kitware.com/settings
>
> Gerrit-MessageType: merged
> Gerrit-Change-Id: Ie2110c51639f491cb7b8a6287b0eaeb1370c50bd
> Gerrit-PatchSet: 2
> Gerrit-Project: ITK
> Gerrit-Branch: master
> Gerrit-Owner: Bill Lorensen <bill.lorensen at gmail.com>
> Gerrit-Reviewer: Bill Lorensen <bill.lorensen at gmail.com>
> Gerrit-Reviewer: Hans J. Johnson <hans-johnson at uiowa.edu>
> Gerrit-Reviewer: Matt McCormick <matt.mccormick at kitware.com>
> Gerrit-Reviewer: Ziv Yaniv <zyaniv at childrensnational.org>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/mailman/private/insight-developers/attachments/20110915/289e8129/attachment.htm>