OpenNN::NewtonMethod Class Reference

#include <newton_method.h>

Inheritance diagram for OpenNN::NewtonMethod:

OpenNN::TrainingAlgorithm

List of all members.

Classes

struct  NewtonMethodResults

Public Member Functions

 NewtonMethod (void)
 NewtonMethod (PerformanceFunctional *)
 NewtonMethod (TiXmlElement *)
virtual ~NewtonMethod (void)
const TrainingRateAlgorithmget_training_rate_algorithm (void) const
TrainingRateAlgorithmget_training_rate_algorithm_pointer (void)
const double & get_warning_parameters_norm (void) const
const double & get_warning_gradient_norm (void) const
const double & get_warning_training_rate (void) const
const double & get_error_parameters_norm (void) const
const double & get_error_gradient_norm (void) const
const double & get_error_training_rate (void) const
const double & get_minimum_parameters_increment_norm (void) const
const double & get_minimum_performance_increase (void) const
const double & get_performance_goal (void) const
const double & get_gradient_norm_goal (void) const
const unsigned int & get_maximum_generalization_evaluation_decreases (void) const
const unsigned int & get_maximum_epochs_number (void) const
const double & get_maximum_time (void) const
const bool & get_reserve_parameters_history (void) const
const bool & get_reserve_parameters_norm_history (void) const
const bool & get_reserve_evaluation_history (void) const
const bool & get_reserve_gradient_history (void) const
const bool & get_reserve_gradient_norm_history (void) const
const bool & get_reserve_inverse_Hessian_history (void) const
const bool & get_reserve_generalization_evaluation_history (void) const
const bool & get_reserve_training_direction_history (void) const
const bool & get_reserve_training_rate_history (void) const
const bool & get_reserve_elapsed_time_history (void) const
const unsigned int & get_display_period (void) const
void set_default (void)
void set_warning_parameters_norm (const double &)
void set_warning_gradient_norm (const double &)
void set_warning_training_rate (const double &)
void set_error_parameters_norm (const double &)
void set_error_gradient_norm (const double &)
void set_error_training_rate (const double &)
void set_minimum_parameters_increment_norm (const double &)
void set_minimum_performance_increase (const double &)
void set_performance_goal (const double &)
void set_gradient_norm_goal (const double &)
void set_maximum_generalization_evaluation_decreases (const unsigned int &)
void set_maximum_epochs_number (const unsigned int &)
void set_maximum_time (const double &)
void set_reserve_parameters_history (const bool &)
void set_reserve_parameters_norm_history (const bool &)
void set_reserve_evaluation_history (const bool &)
void set_reserve_gradient_history (const bool &)
void set_reserve_gradient_norm_history (const bool &)
void set_reserve_inverse_Hessian_history (const bool &)
void set_reserve_generalization_evaluation_history (const bool &)
void set_reserve_training_direction_history (const bool &)
void set_reserve_training_rate_history (const bool &)
void set_reserve_elapsed_time_history (const bool &)
void set_reserve_all_training_history (const bool &)
void set_display_period (const unsigned int &)
Vector< double > calculate_gradient_descent_training_direction (const Vector< double > &) const
Vector< double > calculate_training_direction (const Vector< double > &, const Matrix< double > &) const
NewtonMethodResultsperform_training (void)
std::string write_training_algorithm_type (void) const
TiXmlElement * to_XML (void) const
void from_XML (TiXmlElement *)


Detailed Description

This concrete class represents the Newton method training algorithm for a performance functional of a neural network.

Definition at line 37 of file newton_method.h.


Constructor & Destructor Documentation

OpenNN::NewtonMethod::NewtonMethod ( void   )  [explicit]

Default constructor. It creates a Newton method training algorithm object not associated to any performance functional object. It also initializes the class members to their default values.

Definition at line 43 of file newton_method.cpp.

OpenNN::NewtonMethod::NewtonMethod ( PerformanceFunctional new_performance_functional_pointer  )  [explicit]

General constructor. It creates a Newton method training algorithm object associated to a performance functional object. It also initializes the class members to their default values.

Parameters:
new_performance_functional_pointer Pointer to a performance functional object.

Definition at line 57 of file newton_method.cpp.

OpenNN::NewtonMethod::NewtonMethod ( TiXmlElement *  Newton_method_element  )  [explicit]

XML Constructor. This method creates a Newton method object, and loads its members from a XML element.

Parameters:
Newton_method_element Pointer to a TinyXML element containing the Newton method data.

Definition at line 72 of file newton_method.cpp.

OpenNN::NewtonMethod::~NewtonMethod ( void   )  [virtual]

Destructor.

Definition at line 85 of file newton_method.cpp.


Member Function Documentation

const TrainingRateAlgorithm & OpenNN::NewtonMethod::get_training_rate_algorithm ( void   )  const

This method returns a constant reference to the training rate algorithm object inside the Newton method object.

Definition at line 96 of file newton_method.cpp.

TrainingRateAlgorithm * OpenNN::NewtonMethod::get_training_rate_algorithm_pointer ( void   ) 

This method returns a pointer to the training rate algorithm object inside the Newton method object.

Definition at line 106 of file newton_method.cpp.

const double & OpenNN::NewtonMethod::get_warning_parameters_norm ( void   )  const

This method returns the minimum value for the norm of the parameters vector at wich a warning message is written to the screen.

Definition at line 117 of file newton_method.cpp.

const double & OpenNN::NewtonMethod::get_warning_gradient_norm ( void   )  const

This method returns the minimum value for the norm of the gradient vector at wich a warning message is written to the screen.

Definition at line 128 of file newton_method.cpp.

const double & OpenNN::NewtonMethod::get_warning_training_rate ( void   )  const

This method returns the training rate value at wich a warning message is written to the screen during line minimization.

Definition at line 139 of file newton_method.cpp.

const double & OpenNN::NewtonMethod::get_error_parameters_norm ( void   )  const

This method returns the value for the norm of the parameters vector at wich an error message is written to the screen and the program exits.

Definition at line 150 of file newton_method.cpp.

const double & OpenNN::NewtonMethod::get_error_gradient_norm ( void   )  const

This method returns the value for the norm of the gradient vector at wich an error message is written to the screen and the program exits.

Definition at line 161 of file newton_method.cpp.

const double & OpenNN::NewtonMethod::get_error_training_rate ( void   )  const

This method returns the training rate value at wich the line minimization algorithm is assumed to fail when bracketing a minimum.

Definition at line 172 of file newton_method.cpp.

const double & OpenNN::NewtonMethod::get_minimum_parameters_increment_norm ( void   )  const

This method returns the minimum norm of the parameter increment vector used as a stopping criteria when training.

Definition at line 182 of file newton_method.cpp.

const double & OpenNN::NewtonMethod::get_minimum_performance_increase ( void   )  const

This method returns the minimum performance improvement during training.

Definition at line 192 of file newton_method.cpp.

const double & OpenNN::NewtonMethod::get_performance_goal ( void   )  const

This method returns the goal value for the performance. This is used as a stopping criterium when training a multilayer perceptron

Definition at line 203 of file newton_method.cpp.

const double & OpenNN::NewtonMethod::get_gradient_norm_goal ( void   )  const

This method returns the goal value for the norm of the objective function gradient. This is used as a stopping criterium when training a multilayer perceptron

Definition at line 214 of file newton_method.cpp.

const unsigned int & OpenNN::NewtonMethod::get_maximum_generalization_evaluation_decreases ( void   )  const

This method returns the maximum number of generalization failures during the training process.

Definition at line 224 of file newton_method.cpp.

const unsigned int & OpenNN::NewtonMethod::get_maximum_epochs_number ( void   )  const

This method returns the maximum number of epochs for training.

Definition at line 234 of file newton_method.cpp.

const double & OpenNN::NewtonMethod::get_maximum_time ( void   )  const

This method returns the maximum training time.

Definition at line 244 of file newton_method.cpp.

const bool & OpenNN::NewtonMethod::get_reserve_parameters_history ( void   )  const

This method returns true if the parameters history matrix is to be reserved, and false otherwise.

Definition at line 254 of file newton_method.cpp.

const bool & OpenNN::NewtonMethod::get_reserve_parameters_norm_history ( void   )  const

This method returns true if the parameters norm history vector is to be reserved, and false otherwise.

Definition at line 264 of file newton_method.cpp.

const bool & OpenNN::NewtonMethod::get_reserve_evaluation_history ( void   )  const

This method returns true if the evaluation history vector is to be reserved, and false otherwise.

Definition at line 274 of file newton_method.cpp.

const bool & OpenNN::NewtonMethod::get_reserve_gradient_history ( void   )  const

This method returns true if the gradient history vector of vectors is to be reserved, and false otherwise.

Definition at line 284 of file newton_method.cpp.

const bool & OpenNN::NewtonMethod::get_reserve_gradient_norm_history ( void   )  const

This method returns true if the gradient norm history vector is to be reserved, and false otherwise.

Definition at line 294 of file newton_method.cpp.

const bool & OpenNN::NewtonMethod::get_reserve_inverse_Hessian_history ( void   )  const

This method returns true if the inverse Hessian history vector of matrices is to be reserved, and false otherwise.

Definition at line 304 of file newton_method.cpp.

const bool & OpenNN::NewtonMethod::get_reserve_generalization_evaluation_history ( void   )  const

This method returns true if the Generalization evaluation history vector is to be reserved, and false otherwise.

Definition at line 344 of file newton_method.cpp.

const bool & OpenNN::NewtonMethod::get_reserve_training_direction_history ( void   )  const

This method returns true if the training direction history matrix is to be reserved, and false otherwise.

Definition at line 314 of file newton_method.cpp.

const bool & OpenNN::NewtonMethod::get_reserve_training_rate_history ( void   )  const

This method returns true if the training rate history vector is to be reserved, and false otherwise.

Definition at line 324 of file newton_method.cpp.

const bool & OpenNN::NewtonMethod::get_reserve_elapsed_time_history ( void   )  const

This method returns true if the elapsed time history vector is to be reserved, and false otherwise.

Definition at line 334 of file newton_method.cpp.

const unsigned int & OpenNN::NewtonMethod::get_display_period ( void   )  const

This method returns the number of epochs between the training showing progress.

Definition at line 354 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_default ( void   )  [virtual]

This method sets the members of the training algorithm object to their default values.

Reimplemented from OpenNN::TrainingAlgorithm.

Definition at line 362 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_warning_parameters_norm ( const double &  new_warning_parameters_norm  ) 

This method sets a new value for the parameters vector norm at which a warning message is written to the screen.

Parameters:
new_warning_parameters_norm Warning norm of parameters vector value.

Definition at line 414 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_warning_gradient_norm ( const double &  new_warning_gradient_norm  ) 

This method sets a new value for the gradient vector norm at which a warning message is written to the screen.

Parameters:
new_warning_gradient_norm Warning norm of gradient vector value.

Definition at line 445 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_warning_training_rate ( const double &  new_warning_training_rate  ) 

This method sets a new training rate value at wich a warning message is written to the screen during line minimization.

Parameters:
new_warning_training_rate Warning training rate value.

Definition at line 476 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_error_parameters_norm ( const double &  new_error_parameters_norm  ) 

This method sets a new value for the parameters vector norm at which an error message is written to the screen and the program exits.

Parameters:
new_error_parameters_norm Error norm of parameters vector value.

Definition at line 505 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_error_gradient_norm ( const double &  new_error_gradient_norm  ) 

This method sets a new value for the gradient vector norm at which an error message is written to the screen and the program exits.

Parameters:
new_error_gradient_norm Error norm of gradient vector value.

Definition at line 536 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_error_training_rate ( const double &  new_error_training_rate  ) 

This method sets a new training rate value at wich a the line minimization algorithm is assumed to fail when bracketing a minimum.

Parameters:
new_error_training_rate Error training rate value.

Definition at line 567 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_minimum_parameters_increment_norm ( const double &  new_minimum_parameters_increment_norm  ) 

This method sets a new value for the minimum parameters increment norm stopping criterium.

Parameters:
new_minimum_parameters_increment_norm Value of norm of parameters increment norm used to stop training.

Definition at line 597 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_minimum_performance_increase ( const double &  new_minimum_performance_increase  ) 

This method sets a new minimum performance improvement during training.

Parameters:
new_minimum_performance_increase Minimum improvement in the performance between two epochs.

Definition at line 627 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_performance_goal ( const double &  new_performance_goal  ) 

This method sets a new goal value for the performance. This is used as a stopping criterium when training a multilayer perceptron

Parameters:
new_performance_goal Goal value for the performance.

Definition at line 658 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_gradient_norm_goal ( const double &  new_gradient_norm_goal  ) 

This method sets a new the goal value for the norm of the objective function gradient. This is used as a stopping criterium when training a multilayer perceptron

Parameters:
new_gradient_norm_goal Goal value for the norm of the objective function gradient.

Definition at line 670 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_maximum_generalization_evaluation_decreases ( const unsigned int &  new_maximum_generalization_evaluation_decreases  ) 

This method sets a new maximum number of generalization failures.

Parameters:
new_maximum_generalization_evaluation_decreases Maximum number of epochs in which the generalization evalutation decreases.

Definition at line 700 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_maximum_epochs_number ( const unsigned int &  new_maximum_epochs_number  ) 

This method sets a maximum number of epochs for training.

Parameters:
new_maximum_epochs_number Maximum number of epochs for training.

Definition at line 730 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_maximum_time ( const double &  new_maximum_time  ) 

This method sets a new maximum training time.

Parameters:
new_maximum_time Maximum training time.

Definition at line 760 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_reserve_parameters_history ( const bool &  new_reserve_parameters_history  ) 

This method makes the parameters history vector of vectors to be reseved or not in memory.

Parameters:
new_reserve_parameters_history True if the parameters history vector of vectors is to be reserved, false otherwise.

Definition at line 790 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_reserve_parameters_norm_history ( const bool &  new_reserve_parameters_norm_history  ) 

This method makes the parameters norm history vector to be reseved or not in memory.

Parameters:
new_reserve_parameters_norm_history True if the parameters norm history vector is to be reserved, false otherwise.

Definition at line 801 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_reserve_evaluation_history ( const bool &  new_reserve_evaluation_history  ) 

This method makes the evaluation history vector to be reseved or not in memory.

Parameters:
new_reserve_evaluation_history True if the evaluation history vector is to be reserved, false otherwise.

Definition at line 812 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_reserve_gradient_history ( const bool &  new_reserve_gradient_history  ) 

This method makes the gradient history vector of vectors to be reseved or not in memory.

Parameters:
new_reserve_gradient_history True if the gradient history matrix is to be reserved, false otherwise.

Definition at line 823 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_reserve_gradient_norm_history ( const bool &  new_reserve_gradient_norm_history  ) 

This method makes the gradient norm history vector to be reseved or not in memory.

Parameters:
new_reserve_gradient_norm_history True if the gradient norm history matrix is to be reserved, false otherwise.

Definition at line 835 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_reserve_inverse_Hessian_history ( const bool &  new_reserve_inverse_Hessian_history  ) 

This method sets the history of the inverse of the Hessian matrix to be reserved or not in memory. This is a vector of matrices.

Parameters:
new_reserve_inverse_Hessian_history True if the inverse Hessian history is to be reserved, false otherwise.

Definition at line 847 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_reserve_generalization_evaluation_history ( const bool &  new_reserve_generalization_evaluation_history  ) 

This method makes the Generalization evaluation history to be reserved or not in memory. This is a vector.

Parameters:
new_reserve_generalization_evaluation_history True if the Generalization evaluation history is to be reserved, false otherwise.

Definition at line 895 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_reserve_training_direction_history ( const bool &  new_reserve_training_direction_history  ) 

This method makes the training direction history vector of vectors to be reseved or not in memory.

Parameters:
new_reserve_training_direction_history True if the training direction history matrix is to be reserved, false otherwise.

Definition at line 859 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_reserve_training_rate_history ( const bool &  new_reserve_training_rate_history  ) 

This method makes the training rate history vector to be reseved or not in memory.

Parameters:
new_reserve_training_rate_history True if the training rate history vector is to be reserved, false otherwise.

Definition at line 871 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_reserve_elapsed_time_history ( const bool &  new_reserve_elapsed_time_history  ) 

This method makes the elapsed time over the epochs to be reseved or not in memory. This is a vector.

Parameters:
new_reserve_elapsed_time_history True if the elapsed time history vector is to be reserved, false otherwise.

Definition at line 883 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_reserve_all_training_history ( const bool &  new_reserve_all_training_history  ) 

This method makes the training history of all variables to be reseved or not in memory.

Definition at line 2240 of file newton_method.cpp.

void OpenNN::NewtonMethod::set_display_period ( const unsigned int &  new_display_period  ) 

This method sets a new number of epochs between the training showing progress.

Parameters:
new_display_period Number of epochs between the training showing progress.

Definition at line 907 of file newton_method.cpp.

Vector< double > OpenNN::NewtonMethod::calculate_gradient_descent_training_direction ( const Vector< double > &  gradient  )  const

This method returns the gradient descent training direction, which is the negative of the normalized gradient.

Parameters:
gradient Gradient vector.

Definition at line 936 of file newton_method.cpp.

Vector< double > OpenNN::NewtonMethod::calculate_training_direction ( const Vector< double > &  gradient,
const Matrix< double > &  inverse_Hessian 
) const

This method returns the Newton method training direction, which has been previously normalized.

Parameters:
gradient Gradient vector.
inverse_Hessian Inverse Hessian matrix.

Definition at line 951 of file newton_method.cpp.

NewtonMethod::NewtonMethodResults * OpenNN::NewtonMethod::perform_training ( void   )  [virtual]

This method trains a neural network with an associated performance functional according to the Newton method algorithm. Training occurs according to the training operators, the training parameters and the stopping criteria.

Todo:

Implements OpenNN::TrainingAlgorithm.

Definition at line 1093 of file newton_method.cpp.

std::string OpenNN::NewtonMethod::write_training_algorithm_type ( void   )  const [virtual]

This method writes a string with the type of training algoritm.

Reimplemented from OpenNN::TrainingAlgorithm.

Definition at line 1467 of file newton_method.cpp.

TiXmlElement * OpenNN::NewtonMethod::to_XML ( void   )  const [virtual]

This method prints to the screen the training parameters, the stopping criteria and other user stuff concerning the Newton's method object: Stopping criteria:

  • Performance goal.
  • Gradient norm goal.
  • Maximum training time.
  • Maximum number of epochs.
User stuff:
  • Display.
  • Display period.

Reimplemented from OpenNN::TrainingAlgorithm.

Definition at line 1490 of file newton_method.cpp.

void OpenNN::NewtonMethod::from_XML ( TiXmlElement *  training_algorithm_element  )  [virtual]

This method loads a training algorithm object from a XML element.

Parameters:
training_algorithm_element Pointer to a Tiny XML element containing the training algorithm members.

Reimplemented from OpenNN::TrainingAlgorithm.

Definition at line 1785 of file newton_method.cpp.


The documentation for this class was generated from the following files:

Generated on Sun Aug 26 11:58:20 2012 for OpenNN by  doxygen 1.5.9