[Insight-developers] single valued cost function

[Ziv Yaniv]

  Hi Stefan,

The solution you are advocating indeed reduces the number of required 
changes. On the other hand the proposed hierarchy will force the user to 
think about his/her choices. If we supply a default GetDerivative() 
method that is based on finite differences I am certain that many users 
will use this class with optimizers that require the gradient and never 
think about providing an analytic derivative even when it is available. 
This is human nature, we have a tendency to accept default 
values/implementations set by others, simply out of convenience.

Personally, I believe it is preferable to have users think about their 
choices and not use implicit default values/implementations.

                                 regards
                                       Ziv

On 9/7/2010 7:34 AM, Stefan Klein wrote:
> Hi Ziv, Karthik,
>
> I just noticed the following wiki:
>
> http://www.itk.org/Wiki/Proposals:Refactoring_of_optimization_framework
>
> and took a look again at the emails that you sent on the mailing list 
> about this topic.
>
> I was wondering what you think about the following alternative option:
>
> Implement a finite difference derivative in the GetDerivative method 
> of the SingleValuedCostFunction (instead of making it pure virtual).
>
> In that way all cost functions will have a valid implementation of the 
> GetDerivative. It seems more straightforward than adding a new class 
> (which implies also modifications in the optimizer classes).
>
> Kind regards,
> Stefan
>
>
>
>
>
> On 15/07/2010 21:51, Ziv Yaniv wrote:
>> Karthik,
>>
>> You are obviously correct. Either the optimizer hierarchy should reflect
>> this, or the type of the cost function passed to optimizers that require
>> derivatives should be changed to the proposed
>> SingleValuedCostFunctionWithDerivative.
>>
>> I believe the only itk optimizer that doesn't require a derivative is
>> the Amoeba optimizer?
>>
>> In any case, a nice design of an optimization hierarchy is described in:
>>
>> J. C. Meza, R. A. Oliva, P. D. Hough, and P. J. Williams, "OPT++: An
>> Object Oriented Toolkit for Nonlinear Optimization", ACM Transactions on
>> Mathematical Software, Volume 33, Number 2, June 2007.
>>
>> The toolkit's site is https://software.sandia.gov/opt++/index.html
>>
>> Ziv
>>
>>
>> On 7/15/2010 2:52 PM, Karthik Krishnan wrote:
>>> On Fri, Jul 16, 2010 at 12:02 AM, Ziv Yaniv
>>> <zivy at isis.imac.georgetown.edu <mailto:zivy at isis.imac.georgetown.edu>>
>>> wrote:
>>>
>>> What was the reasoning behind having the GetValueAndDerivative and
>>> the pure virtual GetDerivative method in the
>>> SingleValuedCostFunction class? This does not accommodate non
>>> differentiable functions in a natural way.
>>>
>>> I see two options:
>>>
>>> Slight modification of the current cost function hierarchy,
>>> 1. Remove both methods from the SingleValuedCostFunction class.
>>> 2. Add a class SingleValuedCostFunctionWithDerivative adding the two
>>> methods and inheriting from SingleValuedCostFunction
>>> 3. Modify the inheritance of the direct subclasses (7 according to
>>> doxygen) from the old class to the new one.
>>>
>>>
>>> Don't you also need to to include a similar additional hierarchy in the
>>> optimizers, since the hierarchy of cost functions should be matched by
>>> the space of optimizers.
>>>
>>> There are only two kinds of cost functions : (a) single valued
>>> multivariate (b) Multivalued multivariate cost functions. Likewise 
>>> there
>>> are two kinds of optimizers as well.
>>>
>>> thx
>>> -- 
>>> karthik
>>>
>>>
>>> No change to code base,
>>> 1. Implementation of GetDerivative in a subclass that cannot compute
>>> one just throws an excpetion. Ugly but guaranteed to get the
>>> developers attention.
>>>
>>> Am I missing something? If not, I would be happy if itk-v4 adopts
>>> the first option.
>>>
>>> Ziv
>>>
>>>
>>> -- 
>>> Ziv Yaniv, PhD., Research Assistant Professor
>>> Imaging Science and Information Systems (ISIS) Center
>>> Department of Radiology
>>> Georgetown University Medical Center
>>> 2115 Wisconsin Avenue, Suite 603
>>> Washington, DC, 20007,
>>>
>>> Phone: +1-202-687-7286
>>> Fax: +1-202-784-3479
>>> email: zivy at isis.georgetown.edu <mailto:zivy at isis.georgetown.edu>
>>> web: http://isiswiki.georgetown.edu/zivy/
>>> _______________________________________________
>>> Powered by www.kitware.com <http://www.kitware.com>
>>>
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>>
>>> Kitware offers ITK Training Courses, for more information visit:
>>> http://kitware.com/products/protraining.html
>>>
>>> Please keep messages on-topic and check the ITK FAQ at:
>>> http://www.itk.org/Wiki/ITK_FAQ
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://www.itk.org/mailman/listinfo/insight-developers
>>>
>>>
>>
>>
>


-- 
Ziv Yaniv, PhD., Research Assistant Professor
Imaging Science and Information Systems (ISIS) Center
Department of Radiology
Georgetown University Medical Center
2115 Wisconsin Avenue, Suite 603
Washington, DC, 20007,

Phone: +1-202-6877286
Fax: +1-202-784-3479
email: zivy at isis.georgetown.edu
web: http://isiswiki.georgetown.edu/zivy/