[Insight-developers] Optimizer termination criteria

[Stephen R. Aylward]

I think this should be a part of a broader effort to standardize the 
optimizers like we standardized the transforms.   There seem to be 
general categories of optimizers whose re-use would benefit from 
subclassing the optimizer hierarchy.

On 6/20, Simon Warfield sent a great email to the developers that 
summarizes numerous inconsistencies with the optimizers.   I've got a 
copy somewhere....it should also be in the archives.

We've been having trouble with optimizers because some problems must 
avoid overfitting by using termination criterion based on a system's 
performance on a second set of data.  Your change would make this MUCH 
easier - so we wouldn't have to keep changing ITK methods in a problem 
specific way.

How about putting something up on the wiki to let people know what you 
are doing....

THANKS!
Stephen


Zachary Pincus wrote:
> Hello folks,
> 
> I've been chatting with the folks who contributed the SPSA optimizer to 
> ITK (on the users list), partially about the termination criteria for 
> that class. Initially we thought that adding a few different criteria 
> would be good, but on reflection (a) there are simply too many 
> reasonable options for termination criteria to support all of them in 
> one class, (b) this would contribute even more to the balkanization of 
> the optimizer API that had been previously mentioned, and (c) this 
> wouldn't do any good for any other optimizers.
> 
> So the suggestion that we came up with was that in general optimizers 
> might be best to not implement any fancy termination criteria,* and 
> instead rely on user-provided callbacks to terminate optimization. Then, 
> provided with a prefab set of callbacks for most reasonable types of 
> termination criteria (size of parameter change, size of objective 
> function value change, size of gradient), the user could simply plug in 
> the appropriate criteria for their task.
> 
> So: Is this a good idea? Or is there enough of a performance penalty 
> from adding callbacks that requiring a callback for every optimizer 
> would be infeasible? Or some other objection?
> 
> If this is a worthy goal, I think it would be possible to refactor 
> existing optimizers so that their "termination criteria API" stays the 
> same, while they, behind the scenes, just use callbacks. This way, 
> backwards compatibility could be maintained, while still allowing users 
> to remove these criteria and add their own.
> 
> What do you guys think? Does this suggestion even approach a reasonable 
> work-to-reward tradeoff, compared to some of the other stuff on the ITK 
> roadmap?
> 
> Thanks,
> 
> Zach Pincus
> 
> Department of Biochemistry and Program in Biomedical Informatics
> Stanford University School of Medicine
> 
> 
> * Or some optimizer-specific criteria that is best evaluated in the 
> optimizer itself, e.g. the OnePlusOne optimizer's 
> frobenius-norm-of-the-A-matrix criteria.
> 
> _______________________________________________
> Insight-developers mailing list
> Insight-developers at itk.org
> http://www.itk.org/mailman/listinfo/insight-developers

-- 
===========================================================
Dr. Stephen R. Aylward
Associate Professor of Radiology
Adjunct Associate Professor of Computer Science and Surgery
http://caddlab.rad.unc.edu
aylward at unc.edu
(919) 966-9695