<html>
<head>
<style>
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
FONT-SIZE: 10pt;
FONT-FAMILY:Tahoma
}
</style>
</head>
<body class='hmmessage'><div style="text-align: left;"><br></div>Stefan,<br><br>Your description clears up the ambiguity. Since the Levenberg-Marquadt and the quasi-Newton optimisers are comparable methods, the quasi-Newton method could be implemented to be a multi-valued multi-variate optimiser just like the Levenberg-Marquadt implementation is. Therefore, this is not an implementation issue of VNL or ITK, it's just that the implementation of quasi-Newton in LBFGC is not multi-valued comparing with the implementation of the Levenberg-Marquadt method in MINPACK, where VNL is just a wrapper for both of them.<br><blockquote><br><br>
<font size="3">Hi Ali,<br><br>
The itk::LBFGS(B)Optimizer is single-VALUED and multi-VARIATE. Their
implementation is correct, both in itk and vnl. <br><br>
the optimisaton problem is:<br><br>
min_x f(x)<br><br>
f = the cost function to be minimised<br>
x = the input arguments/parameters<br><br>
single-valued: the cost function f is a scalar (dim(f)=1)<br>
multiple-valued: the cost function f is a vector (dim(f)>1)<br><br>
univariate: the input argument x is a scalar (dim(x)=1)<br>
multivariate: the input argument x is a vector (dim(x)>1)<br><br>
The LBFGS(B)Optimizer can handle: single-valued f, both univariate x and
multivariate x<br>
This is correctly implemented in itk/vnl<br><br>
The LBFGS(B)Optimizer can NOT handle: multiple-valued f. <br>
The itk implementation should therefore not be modified to accept
MultipleValuedCostFunctions.<br><br>
Sincerely,<br>
Stefan<br><br>
<br>
At 14:40 7-11-2007, Ali - wrote:<br>
</font><blockquote class="cite"><font size="3">I guess Stefan is right. There is
no such thing as vnl_multiple_valued_cost_function, hence,
itk::LBFGSBOptimizer and itk::LBFGSOptimizer shouldn't have been
classified as single-variant optimisers in ITK and they should be
modified for accepting itk::MultipleValuedCostFunction.<br><br>
<br>
>Hi,<br><br>
>The itk/vxl::LBFGS(B)Optimizer implementation IS multivariate. The
'multivariate' label refers to the number of variables (input arguments
of the cost function) over which is optimised. <br><br>
>the optimisaton problem is:<br><br>
>min_x f(x)<br><br>
> On the site
<a href="http://www.alglib.net/optimization/%A0" target="_blank">http://www.alglib.net/optimization/
</a> a multivariate optimisation method means that dim(x) >1 is
allowed.<br><br>
> The LBFGSOptimizer is not suitable for MultipleValuedCostFucnction,
as far as I know, only for SingleValuedCostFunctions.<br><br>
> singlevalued means: dim(f) = 1<br>
> multiplevalued means: dim(f) > 1<br><br>
> The vnl_cost_function indeed inherits from the vnl_unary_function,
but the vnl_unary_function is templated over a vector of doubles, which
contains the input argument x:<br><br>
> class vnl_cost_function : public vnl_unary_function<double,
vnl_vector<double> ><br><br>
> with kind regards,<br>
> stefan<br><br>
<br>
> <br>
> <br>
> <br>
> Hi Ali,<br>
> <br>
> <br>
> 1) It is not a "problem", it is an "implementation
feature". :-)<br>
> <br>
> <br>
> <br>
> 2) It is not in ITK, it is in VXL,<br>
> <br>
> Please look at the implementation in the file:<br>
> <br>
> Insight/Utilities/vxl/core/vnl/vnl_lbfgs.h<br>
> <br>
> Where the optimizer takes as input a vnl_cost_function<br>
> object, that derives from the vnl_unary_function class.<br>
> <br>
> <br>
> <br>
> 3) It will be great to have multi-variate implementations<br>
> of these optimizers. If you are motivated to write one,<br>
> it will make a great contribution to the Insight Journal.<br>
> <br>
> <br>
> <br>
> Regards,<br>
> <br>
> <br>
> Luis<br>
> <br>
> <br>
> <br>
> -------------<br>
> Ali - wrote:<br>
> > Hi,<br>
> > <br>
> > The VNL wrappers itk::LBFGSBOptimizer and itk::LBFGSOptimizer
are <br>
> > classified as single-variant optimiser, while being based on
the <br>
> > quasi-Newton method, they are multi-variant optimisers by
nature -- see <br>
> >
<a href="http://www.alglib.net/optimization/" target="_blank">http://www.alglib.net/optimization/</a>
. Is this the ITK wrapper problem or <br>
> > a VNL issue?<br>
> > <br>
> > _______________________________________________<br>
> > Insight-users mailing list<br>
> > Insight-users@itk.org<br>
> > <a href="http://www.itk.org/mailman/listinfo/insight-users" target="_blank">http://www.itk.org/mailman/listinfo/insight-users</a><br><br>
</font><hr>
<font size="3">Do you know a place like the back of your hand? Share local knowledge with <a target="_blank">BackOfMyHand.com</a> <br>
_______________________________________________<br>
Insight-users mailing list<br>
Insight-users@itk.org<br>
<a href="http://www.itk.org/mailman/listinfo/insight-users" target="_blank">http://www.itk.org/mailman/listinfo/insight-users</a></font></blockquote>
<br>
</blockquote><br /><hr />Do you know a place like the back of your hand? Share local knowledge with <a href='http://www.backofmyhand.com' target='_new'>BackOfMyHand.com</a></body>
</html>