[Insight-developers] NeuralNetworks: BatchSupervisedTrainingFunction copying vectors over and over?

[kent williams]

I'm trying to figure out why our code using the ITK NeuralNetworks isn't
working right. As a result, I'm spending my days in GDB stepping through the
code.

So here is the scenario:

BatchSupervisedTrainingFunction::Train is stepping through the training
vector, calling OneHiddenLayerBackPropagationNeuralNetwork::GenerateOutput.

That function calls MultilayerNeuralNetworkBase:GenerateOutput.

That method calls BackPropagationLayer::ForwardPropagate.

Each of the called functions takes a sample vector as a value parameter. So
the vector passed from Train gets copied 3 times as a parameter before it's
even used.

Then in ForwardPropagate it uses the samplevector destructively.  In this
particular case, it¹s completely unnecessary because a single scalar
variable could be used.

    template<class TMeasurementVector, class TTargetVector>
      void
      BackPropagationLayer<TMeasurementVector,TTargetVector>
      ::ForwardPropagate(TMeasurementVector samplevector)
        {
        typename Superclass::TransferFunctionInterfaceType::Pointer
transferfunction;
        transferfunction = this->GetActivationFunction();

        for (unsigned int i = 0; i < samplevector.Size(); i++)
          {
          samplevector[i] = transferfunction->Evaluate(samplevector[i]);
          m_NodeOutputValues.put(i, samplevector[i]);
          }
        }

My question is this: was this a design decision, or did it just happen this
way. When you¹re training a neural network, this call tree is invoked many
thousands or millions of times, and the input vectors are copied
unnecessarily 3 times each time through the call tree. Pretty soon that adds
up to some significant wasted cycles.

Since this code is in Review, should this be re-evaluated for efficiency?



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.itk.org/mailman/private/insight-developers/attachments/20070828/0d5b0026/attachment.htm