#include <quasi_newton_method.h>
Definition at line 37 of file quasi_newton_method.h.
Enumeration of the available training operators for obtaining the approximation to the inverse Hessian.
Definition at line 46 of file quasi_newton_method.h.
OpenNN::QuasiNewtonMethod::QuasiNewtonMethod | ( | void | ) | [explicit] |
Default constructor. It creates a quasi-Newton method training algorithm not associated to any performance functional. It also initializes the class members to their default values.
Definition at line 41 of file quasi_newton_method.cpp.
OpenNN::QuasiNewtonMethod::QuasiNewtonMethod | ( | PerformanceFunctional * | new_performance_functional_pointer | ) | [explicit] |
Performance functional constructor. It creates a quasi-Newton method training algorithm associated to a performance functional. It also initializes the class members to their default values.
new_performance_functional_pointer | Pointer to a performance functional object. |
Definition at line 55 of file quasi_newton_method.cpp.
OpenNN::QuasiNewtonMethod::QuasiNewtonMethod | ( | TiXmlElement * | quasi_Newton_method_element | ) | [explicit] |
XML constructor. It creates a quasi-Newton method training algorithm not associated to any performance functional. It also initializes the class members to their default values.
Definition at line 70 of file quasi_newton_method.cpp.
OpenNN::QuasiNewtonMethod::~QuasiNewtonMethod | ( | void | ) | [virtual] |
Destructor. It does not delete any object.
Definition at line 82 of file quasi_newton_method.cpp.
const TrainingRateAlgorithm & OpenNN::QuasiNewtonMethod::get_training_rate_algorithm | ( | void | ) | const |
This method returns a constant reference to the training rate algorithm object inside the quasi-Newton method object.
Definition at line 95 of file quasi_newton_method.cpp.
TrainingRateAlgorithm * OpenNN::QuasiNewtonMethod::get_training_rate_algorithm_pointer | ( | void | ) |
This method returns a pointer to the training rate algorithm object inside the quasi-Newton method object.
Definition at line 105 of file quasi_newton_method.cpp.
const QuasiNewtonMethod::InverseHessianApproximationMethod & OpenNN::QuasiNewtonMethod::get_inverse_Hessian_approximation_method | ( | void | ) | const |
This method returns the method for approximating the inverse Hessian matrix to be used when training.
Definition at line 115 of file quasi_newton_method.cpp.
std::string OpenNN::QuasiNewtonMethod::write_inverse_Hessian_approximation_method | ( | void | ) | const |
This method returns the name of the method for the approximation of the inverse Hessian.
Definition at line 125 of file quasi_newton_method.cpp.
const double & OpenNN::QuasiNewtonMethod::get_warning_parameters_norm | ( | void | ) | const |
This method returns the minimum value for the norm of the parameters vector at wich a warning message is written to the screen.
Definition at line 160 of file quasi_newton_method.cpp.
const double & OpenNN::QuasiNewtonMethod::get_warning_gradient_norm | ( | void | ) | const |
This method returns the minimum value for the norm of the gradient vector at wich a warning message is written to the screen.
Definition at line 170 of file quasi_newton_method.cpp.
const double & OpenNN::QuasiNewtonMethod::get_warning_training_rate | ( | void | ) | const |
This method returns the training rate value at wich a warning message is written to the screen during line minimization.
Definition at line 180 of file quasi_newton_method.cpp.
const double & OpenNN::QuasiNewtonMethod::get_error_parameters_norm | ( | void | ) | const |
This method returns the value for the norm of the parameters vector at wich an error message is written to the screen and the program exits.
Definition at line 190 of file quasi_newton_method.cpp.
const double & OpenNN::QuasiNewtonMethod::get_error_gradient_norm | ( | void | ) | const |
This method returns the value for the norm of the gradient vector at wich an error message is written to the screen and the program exits.
Definition at line 201 of file quasi_newton_method.cpp.
const double & OpenNN::QuasiNewtonMethod::get_error_training_rate | ( | void | ) | const |
This method returns the training rate value at wich the line minimization algorithm is assumed to fail when bracketing a minimum.
Definition at line 212 of file quasi_newton_method.cpp.
const double & OpenNN::QuasiNewtonMethod::get_minimum_parameters_increment_norm | ( | void | ) | const |
This method returns the minimum norm of the parameter increment vector used as a stopping criteria when training.
Definition at line 222 of file quasi_newton_method.cpp.
const double & OpenNN::QuasiNewtonMethod::get_minimum_performance_increase | ( | void | ) | const |
This method returns the minimum performance improvement during training.
Definition at line 232 of file quasi_newton_method.cpp.
const double & OpenNN::QuasiNewtonMethod::get_performance_goal | ( | void | ) | const |
This method returns the goal value for the performance. This is used as a stopping criterium when training a multilayer perceptron
Definition at line 243 of file quasi_newton_method.cpp.
const double & OpenNN::QuasiNewtonMethod::get_gradient_norm_goal | ( | void | ) | const |
This method returns the goal value for the norm of the objective function gradient. This is used as a stopping criterium when training a multilayer perceptron
Definition at line 254 of file quasi_newton_method.cpp.
const unsigned int & OpenNN::QuasiNewtonMethod::get_maximum_generalization_evaluation_decreases | ( | void | ) | const |
This method returns the maximum number of generalization failures during the training process.
Definition at line 264 of file quasi_newton_method.cpp.
const unsigned int & OpenNN::QuasiNewtonMethod::get_maximum_epochs_number | ( | void | ) | const |
This method returns the maximum number of epochs for training.
Definition at line 274 of file quasi_newton_method.cpp.
const double & OpenNN::QuasiNewtonMethod::get_maximum_time | ( | void | ) | const |
This method returns the maximum training time.
Definition at line 284 of file quasi_newton_method.cpp.
const bool & OpenNN::QuasiNewtonMethod::get_reserve_parameters_history | ( | void | ) | const |
This method returns true if the parameters history matrix is to be reserved, and false otherwise.
Definition at line 295 of file quasi_newton_method.cpp.
const bool & OpenNN::QuasiNewtonMethod::get_reserve_parameters_norm_history | ( | void | ) | const |
This method returns true if the parameters norm history vector is to be reserved, and false otherwise.
Definition at line 305 of file quasi_newton_method.cpp.
const bool & OpenNN::QuasiNewtonMethod::get_reserve_evaluation_history | ( | void | ) | const |
This method returns true if the evaluation history vector is to be reserved, and false otherwise.
Definition at line 315 of file quasi_newton_method.cpp.
const bool & OpenNN::QuasiNewtonMethod::get_reserve_gradient_history | ( | void | ) | const |
This method returns true if the gradient history vector of vectors is to be reserved, and false otherwise.
Definition at line 325 of file quasi_newton_method.cpp.
const bool & OpenNN::QuasiNewtonMethod::get_reserve_gradient_norm_history | ( | void | ) | const |
This method returns true if the gradient norm history vector is to be reserved, and false otherwise.
Definition at line 335 of file quasi_newton_method.cpp.
const bool & OpenNN::QuasiNewtonMethod::get_reserve_inverse_Hessian_history | ( | void | ) | const |
This method returns true if the inverse Hessian history is to be reserved, and false otherwise.
Definition at line 376 of file quasi_newton_method.cpp.
const bool & OpenNN::QuasiNewtonMethod::get_reserve_generalization_evaluation_history | ( | void | ) | const |
This method returns true if the Generalization evaluation history vector is to be reserved, and false otherwise.
Definition at line 386 of file quasi_newton_method.cpp.
const bool & OpenNN::QuasiNewtonMethod::get_reserve_training_direction_history | ( | void | ) | const |
This method returns true if the training direction history matrix is to be reserved, and false otherwise.
Definition at line 346 of file quasi_newton_method.cpp.
const bool & OpenNN::QuasiNewtonMethod::get_reserve_training_rate_history | ( | void | ) | const |
This method returns true if the training rate history vector is to be reserved, and false otherwise.
Definition at line 356 of file quasi_newton_method.cpp.
const bool & OpenNN::QuasiNewtonMethod::get_reserve_elapsed_time_history | ( | void | ) | const |
This method returns true if the elapsed time history vector is to be reserved, and false otherwise.
Definition at line 366 of file quasi_newton_method.cpp.
const unsigned int & OpenNN::QuasiNewtonMethod::get_display_period | ( | void | ) | const |
This method returns the number of epochs between the training showing progress.
Definition at line 396 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_inverse_Hessian_approximation_method | ( | const InverseHessianApproximationMethod & | new_inverse_Hessian_approximation_method | ) |
This method sets a new inverse Hessian approximatation method value.
new_inverse_Hessian_approximation_method | Inverse Hessian approximation method value. |
Definition at line 407 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_inverse_Hessian_approximation_method | ( | const std::string & | new_inverse_Hessian_approximation_method_name | ) |
This method sets a new method for approximating the inverse of the Hessian matrix from a string containing the name. Possible values are:
new_inverse_Hessian_approximation_method_name | Name of inverse Hessian approximation method. |
Definition at line 424 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_default | ( | void | ) | [virtual] |
This method sets the members of the training algorithm object to their default values.
Reimplemented from OpenNN::TrainingAlgorithm.
Definition at line 468 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_warning_parameters_norm | ( | const double & | new_warning_parameters_norm | ) |
This method sets a new value for the parameters vector norm at which a warning message is written to the screen.
new_warning_parameters_norm | Warning norm of parameters vector value. |
Definition at line 524 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_warning_gradient_norm | ( | const double & | new_warning_gradient_norm | ) |
This method sets a new value for the gradient vector norm at which a warning message is written to the screen.
new_warning_gradient_norm | Warning norm of gradient vector value. |
Definition at line 555 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_warning_training_rate | ( | const double & | new_warning_training_rate | ) |
This method sets a new training rate value at wich a warning message is written to the screen during line minimization.
new_warning_training_rate | Warning training rate value. |
Definition at line 586 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_error_parameters_norm | ( | const double & | new_error_parameters_norm | ) |
This method sets a new value for the parameters vector norm at which an error message is written to the screen and the program exits.
new_error_parameters_norm | Error norm of parameters vector value. |
Definition at line 615 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_error_gradient_norm | ( | const double & | new_error_gradient_norm | ) |
This method sets a new value for the gradient vector norm at which an error message is written to the screen and the program exits.
new_error_gradient_norm | Error norm of gradient vector value. |
Definition at line 646 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_error_training_rate | ( | const double & | new_error_training_rate | ) |
This method sets a new training rate value at wich a the line minimization algorithm is assumed to fail when bracketing a minimum.
new_error_training_rate | Error training rate value. |
Definition at line 677 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_minimum_parameters_increment_norm | ( | const double & | new_minimum_parameters_increment_norm | ) |
This method sets a new value for the minimum parameters increment norm stopping criterium.
new_minimum_parameters_increment_norm | Value of norm of parameters increment norm used to stop training. |
Definition at line 707 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_minimum_performance_increase | ( | const double & | new_minimum_performance_increase | ) |
This method sets a new minimum performance improvement during training.
new_minimum_performance_increase | Minimum improvement in the performance between two epochs. |
Definition at line 737 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_performance_goal | ( | const double & | new_performance_goal | ) |
This method sets a new goal value for the performance. This is used as a stopping criterium when training a multilayer perceptron
new_performance_goal | Goal value for the performance. |
Definition at line 768 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_gradient_norm_goal | ( | const double & | new_gradient_norm_goal | ) |
This method sets a new the goal value for the norm of the objective function gradient. This is used as a stopping criterium when training a multilayer perceptron
new_gradient_norm_goal | Goal value for the norm of the objective function gradient. |
Definition at line 780 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_maximum_generalization_evaluation_decreases | ( | const unsigned int & | new_maximum_generalization_evaluation_decreases | ) |
This method sets a new maximum number of generalization failures.
new_maximum_generalization_evaluation_decreases | Maximum number of epochs in which the generalization evalutation decreases. |
Definition at line 810 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_maximum_epochs_number | ( | const unsigned int & | new_maximum_epochs_number | ) |
This method sets a maximum number of epochs for training.
new_maximum_epochs_number | Maximum number of epochs for training. |
Definition at line 840 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_maximum_time | ( | const double & | new_maximum_time | ) |
This method sets a new maximum training time.
new_maximum_time | Maximum training time. |
Definition at line 870 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_reserve_parameters_history | ( | const bool & | new_reserve_parameters_history | ) |
This method makes the parameters history vector of vectors to be reseved or not in memory.
new_reserve_parameters_history | True if the parameters history vector of vectors is to be reserved, false otherwise. |
Definition at line 900 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_reserve_parameters_norm_history | ( | const bool & | new_reserve_parameters_norm_history | ) |
This method makes the parameters norm history vector to be reseved or not in memory.
new_reserve_parameters_norm_history | True if the parameters norm history vector is to be reserved, false otherwise. |
Definition at line 911 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_reserve_evaluation_history | ( | const bool & | new_reserve_evaluation_history | ) |
This method makes the evaluation history vector to be reseved or not in memory.
new_reserve_evaluation_history | True if the evaluation history vector is to be reserved, false otherwise. |
Definition at line 922 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_reserve_gradient_history | ( | const bool & | new_reserve_gradient_history | ) |
This method makes the gradient history vector of vectors to be reseved or not in memory.
new_reserve_gradient_history | True if the gradient history matrix is to be reserved, false otherwise. |
Definition at line 933 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_reserve_gradient_norm_history | ( | const bool & | new_reserve_gradient_norm_history | ) |
This method makes the gradient norm history vector to be reseved or not in memory.
new_reserve_gradient_norm_history | True if the gradient norm history matrix is to be reserved, false otherwise. |
Definition at line 945 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_reserve_inverse_Hessian_history | ( | const bool & | new_reserve_inverse_Hessian_history | ) |
This method sets the history of the inverse of the Hessian matrix to be reserved or not in memory. This is a vector of matrices.
new_reserve_inverse_Hessian_history | True if the inverse Hessian history is to be reserved, false otherwise. |
Definition at line 957 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_reserve_generalization_evaluation_history | ( | const bool & | new_reserve_generalization_evaluation_history | ) |
This method makes the Generalization evaluation history to be reserved or not in memory. This is a vector.
new_reserve_generalization_evaluation_history | True if the Generalization evaluation history is to be reserved, false otherwise. |
Definition at line 1005 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_reserve_training_direction_history | ( | const bool & | new_reserve_training_direction_history | ) |
This method makes the training direction history vector of vectors to be reseved or not in memory.
new_reserve_training_direction_history | True if the training direction history matrix is to be reserved, false otherwise. |
Definition at line 969 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_reserve_training_rate_history | ( | const bool & | new_reserve_training_rate_history | ) |
This method makes the training rate history vector to be reseved or not in memory.
new_reserve_training_rate_history | True if the training rate history vector is to be reserved, false otherwise. |
Definition at line 981 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_reserve_elapsed_time_history | ( | const bool & | new_reserve_elapsed_time_history | ) |
This method makes the elapsed time over the epochs to be reseved or not in memory. This is a vector.
new_reserve_elapsed_time_history | True if the elapsed time history vector is to be reserved, false otherwise. |
Definition at line 993 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_display_period | ( | const unsigned int & | new_display_period | ) |
This method sets a new number of epochs between the training showing progress.
new_display_period | Number of epochs between the training showing progress. |
Definition at line 1017 of file quasi_newton_method.cpp.
Vector< double > OpenNN::QuasiNewtonMethod::calculate_training_direction | ( | const Vector< double > & | gradient, | |
const Matrix< double > & | inverse_Hessian_approximation | |||
) | const |
This method returns the quasi-Newton method training direction, which has been previously normalized.
gradient | Gradient vector. | |
inverse_Hessian_approximation | Inverse Hessian approximation matrix. |
Definition at line 1092 of file quasi_newton_method.cpp.
Matrix< double > OpenNN::QuasiNewtonMethod::calculate_inverse_Hessian_approximation | ( | const Vector< double > & | old_parameters, | |
const Vector< double > & | parameters, | |||
const Vector< double > & | old_gradient, | |||
const Vector< double > & | gradient, | |||
const Matrix< double > & | old_inverse_Hessian | |||
) | const |
This method calculates an approximation of the inverse Hessian, accoring to the method used.
old_parameters | Another point of the objective function. | |
parameters | Current point of the objective function | |
old_gradient | Gradient at the other point. | |
gradient | Gradient at the current point. | |
old_inverse_Hessian | Inverse Hessian at the other point of the objective function. |
Definition at line 1052 of file quasi_newton_method.cpp.
Vector< double > OpenNN::QuasiNewtonMethod::calculate_gradient_descent_training_direction | ( | const Vector< double > & | gradient | ) | const |
This method returns the gradient descent training direction, which is the negative of the normalized gradient.
gradient | Gradient vector. |
Definition at line 1107 of file quasi_newton_method.cpp.
Matrix< double > OpenNN::QuasiNewtonMethod::calculate_DFP_inverse_Hessian | ( | const Vector< double > & | old_parameters, | |
const Vector< double > & | parameters, | |||
const Vector< double > & | old_gradient, | |||
const Vector< double > & | gradient, | |||
const Matrix< double > & | old_inverse_Hessian | |||
) | const |
This method returns an approximation of the inverse Hessian matrix according to the Davidon-Fletcher-Powel (DFP) algorithm.
old_parameters | A previous set of parameters. | |
old_gradient | The gradient of the objective function for that previous set of parameters. | |
old_inverse_Hessian | The Hessian of the objective function for that previous set of parameters. | |
parameters | Actual set of parameters. | |
gradient | The gradient of the objective function for the actual set of parameters. |
Definition at line 1125 of file quasi_newton_method.cpp.
Matrix< double > OpenNN::QuasiNewtonMethod::calculate_BFGS_inverse_Hessian | ( | const Vector< double > & | old_parameters, | |
const Vector< double > & | parameters, | |||
const Vector< double > & | old_gradient, | |||
const Vector< double > & | gradient, | |||
const Matrix< double > & | old_inverse_Hessian | |||
) | const |
This method returns an approximation of the inverse Hessian matrix according to the Broyden-Fletcher-Goldfarb-Shanno (BGFS) algorithm.
old_parameters | A previous set of parameters. | |
old_gradient | The gradient of the objective function for that previous set of parameters. | |
old_inverse_Hessian | The Hessian of the objective function for that previous set of parameters. | |
parameters | Actual set of parameters. | |
gradient | The gradient of the objective function for the actual set of parameters. |
Definition at line 1311 of file quasi_newton_method.cpp.
QuasiNewtonMethod::QuasiNetwonMethodResults * OpenNN::QuasiNewtonMethod::perform_training | ( | void | ) | [virtual] |
This method trains a neural network with an associated performance functional according to the quasi-Newton method. Training occurs according to the training operators, training parameters and stopping criteria.
Implements OpenNN::TrainingAlgorithm.
Definition at line 1646 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::set_reserve_all_training_history | ( | const bool & | new_reserve_all_training_history | ) |
This method makes the training history of all variables to reseved or not in memory.
new_reserve_all_training_history | True if the training history of all variables is to be reserved, false otherwise. |
Definition at line 453 of file quasi_newton_method.cpp.
std::string OpenNN::QuasiNewtonMethod::write_training_algorithm_type | ( | void | ) | const [virtual] |
This method writes a string with the type of training algoritm.
Reimplemented from OpenNN::TrainingAlgorithm.
Definition at line 2071 of file quasi_newton_method.cpp.
TiXmlElement * OpenNN::QuasiNewtonMethod::to_XML | ( | void | ) | const [virtual] |
This method returns a XML-type string representation of this quasi-Newton method object. It contains the training methods and parameters chosen, as well as the stopping criteria and other user stuff concerning the quasi-Newton method object.
Reimplemented from OpenNN::TrainingAlgorithm.
Definition at line 2083 of file quasi_newton_method.cpp.
void OpenNN::QuasiNewtonMethod::from_XML | ( | TiXmlElement * | training_algorithm_element | ) | [virtual] |
This method loads a training algorithm object from a XML element.
training_algorithm_element | Pointer to a Tiny XML element containing the training algorithm members. |
Reimplemented from OpenNN::TrainingAlgorithm.
Definition at line 2378 of file quasi_newton_method.cpp.