OpenNN::GradientDescent Class Reference

#include <gradient_descent.h>

Inheritance diagram for OpenNN::GradientDescent:

OpenNN::TrainingAlgorithm

List of all members.

Classes

struct  GradientDescentResults

Public Member Functions

 GradientDescent (void)
 GradientDescent (PerformanceFunctional *)
 GradientDescent (TiXmlElement *)
virtual ~GradientDescent (void)
const TrainingRateAlgorithmget_training_rate_algorithm (void) const
TrainingRateAlgorithmget_training_rate_algorithm_pointer (void)
const double & get_warning_parameters_norm (void) const
const double & get_warning_gradient_norm (void) const
const double & get_warning_training_rate (void) const
const double & get_error_parameters_norm (void) const
const double & get_error_gradient_norm (void) const
const double & get_error_training_rate (void) const
const double & get_minimum_parameters_increment_norm (void) const
const double & get_minimum_performance_increase (void) const
const double & get_performance_goal (void) const
const double & get_gradient_norm_goal (void) const
const unsigned int & get_maximum_generalization_evaluation_decreases (void) const
const unsigned int & get_maximum_epochs_number (void) const
const double & get_maximum_time (void) const
const bool & get_reserve_parameters_history (void) const
const bool & get_reserve_parameters_norm_history (void) const
const bool & get_reserve_evaluation_history (void) const
const bool & get_reserve_gradient_history (void) const
const bool & get_reserve_gradient_norm_history (void) const
const bool & get_reserve_generalization_evaluation_history (void) const
const bool & get_reserve_training_direction_history (void) const
const bool & get_reserve_training_rate_history (void) const
const bool & get_reserve_elapsed_time_history (void) const
const unsigned int & get_display_period (void) const
void set_training_rate_algorithm (const TrainingRateAlgorithm &)
void set_default (void)
void set_reserve_all_training_history (const bool &)
void set_warning_parameters_norm (const double &)
void set_warning_gradient_norm (const double &)
void set_warning_training_rate (const double &)
void set_error_parameters_norm (const double &)
void set_error_gradient_norm (const double &)
void set_error_training_rate (const double &)
void set_minimum_parameters_increment_norm (const double &)
void set_minimum_performance_increase (const double &)
void set_performance_goal (const double &)
void set_gradient_norm_goal (const double &)
void set_maximum_generalization_evaluation_decreases (const unsigned int &)
void set_maximum_epochs_number (const unsigned int &)
void set_maximum_time (const double &)
void set_reserve_parameters_history (const bool &)
void set_reserve_parameters_norm_history (const bool &)
void set_reserve_evaluation_history (const bool &)
void set_reserve_gradient_history (const bool &)
void set_reserve_gradient_norm_history (const bool &)
void set_reserve_generalization_evaluation_history (const bool &)
void set_reserve_training_direction_history (const bool &)
void set_reserve_training_rate_history (const bool &)
void set_reserve_elapsed_time_history (const bool &)
void set_display_period (const unsigned int &)
Vector< double > calculate_training_direction (const Vector< double > &) const
GradientDescentResultsperform_training (void)
std::string write_training_algorithm_type (void) const
TiXmlElement * to_XML (void) const
void from_XML (TiXmlElement *)


Detailed Description

This concrete class represents the gradient descent training algorithm for a performance functional of a multilayer perceptron.

Definition at line 33 of file gradient_descent.h.


Constructor & Destructor Documentation

OpenNN::GradientDescent::GradientDescent ( void   )  [explicit]

Default constructor. It creates a gradient descent training algorithm not associated to any performance functional object. It also initializes the class members to their default values.

Definition at line 41 of file gradient_descent.cpp.

OpenNN::GradientDescent::GradientDescent ( PerformanceFunctional new_performance_functional_pointer  )  [explicit]

Performance functional constructor. It creates a gradient descent training algorithm associated to a performance functional. It also initializes the class members to their default values.

Parameters:
new_performance_functional_pointer Pointer to a performance functional object.

Definition at line 55 of file gradient_descent.cpp.

OpenNN::GradientDescent::GradientDescent ( TiXmlElement *  gradient_descent_element  )  [explicit]

XML constructor. It creates a gradient descent training algorithm not associated to any performance functional object. It also loads the class members from a XML element.

Parameters:
gradient_descent_element Tiny XML element with the members of a gradient descent object.

Definition at line 71 of file gradient_descent.cpp.

OpenNN::GradientDescent::~GradientDescent ( void   )  [virtual]

Destructor.

Definition at line 83 of file gradient_descent.cpp.


Member Function Documentation

const TrainingRateAlgorithm & OpenNN::GradientDescent::get_training_rate_algorithm ( void   )  const

This method returns a constant reference to the training rate algorithm object inside the gradient descent object.

Definition at line 94 of file gradient_descent.cpp.

TrainingRateAlgorithm * OpenNN::GradientDescent::get_training_rate_algorithm_pointer ( void   ) 

This method returns a pointer to the training rate algorithm object inside the gradient descent object.

Definition at line 104 of file gradient_descent.cpp.

const double & OpenNN::GradientDescent::get_warning_parameters_norm ( void   )  const

This method returns the minimum value for the norm of the parameters vector at wich a warning message is written to the screen.

Definition at line 115 of file gradient_descent.cpp.

const double & OpenNN::GradientDescent::get_warning_gradient_norm ( void   )  const

This method returns the minimum value for the norm of the gradient vector at wich a warning message is written to the screen.

Definition at line 126 of file gradient_descent.cpp.

const double & OpenNN::GradientDescent::get_warning_training_rate ( void   )  const

This method returns the training rate value at wich a warning message is written to the screen during line minimization.

Definition at line 137 of file gradient_descent.cpp.

const double & OpenNN::GradientDescent::get_error_parameters_norm ( void   )  const

This method returns the value for the norm of the parameters vector at wich an error message is written to the screen and the program exits.

Definition at line 148 of file gradient_descent.cpp.

const double & OpenNN::GradientDescent::get_error_gradient_norm ( void   )  const

This method returns the value for the norm of the gradient vector at wich an error message is written to the screen and the program exits.

Definition at line 159 of file gradient_descent.cpp.

const double & OpenNN::GradientDescent::get_error_training_rate ( void   )  const

This method returns the training rate value at wich the line minimization algorithm is assumed to fail when bracketing a minimum.

Definition at line 170 of file gradient_descent.cpp.

const double & OpenNN::GradientDescent::get_minimum_parameters_increment_norm ( void   )  const

This method returns the minimum norm of the parameter increment vector used as a stopping criteria when training.

Definition at line 180 of file gradient_descent.cpp.

const double & OpenNN::GradientDescent::get_minimum_performance_increase ( void   )  const

This method returns the minimum performance improvement during training.

Definition at line 190 of file gradient_descent.cpp.

const double & OpenNN::GradientDescent::get_performance_goal ( void   )  const

This method returns the goal value for the performance. This is used as a stopping criterium when training a multilayer perceptron

Definition at line 201 of file gradient_descent.cpp.

const double & OpenNN::GradientDescent::get_gradient_norm_goal ( void   )  const

This method returns the goal value for the norm of the objective function gradient. This is used as a stopping criterium when training a multilayer perceptron

Definition at line 212 of file gradient_descent.cpp.

const unsigned int & OpenNN::GradientDescent::get_maximum_generalization_evaluation_decreases ( void   )  const

This method returns the maximum number of generalization failures during the training process.

Definition at line 222 of file gradient_descent.cpp.

const unsigned int & OpenNN::GradientDescent::get_maximum_epochs_number ( void   )  const

This method returns the maximum number of epochs for training.

Definition at line 232 of file gradient_descent.cpp.

const double & OpenNN::GradientDescent::get_maximum_time ( void   )  const

This method returns the maximum training time.

Definition at line 242 of file gradient_descent.cpp.

const bool & OpenNN::GradientDescent::get_reserve_parameters_history ( void   )  const

This method returns true if the parameters history matrix is to be reserved, and false otherwise.

Definition at line 252 of file gradient_descent.cpp.

const bool & OpenNN::GradientDescent::get_reserve_parameters_norm_history ( void   )  const

This method returns true if the parameters norm history vector is to be reserved, and false otherwise.

Definition at line 262 of file gradient_descent.cpp.

const bool & OpenNN::GradientDescent::get_reserve_evaluation_history ( void   )  const

This method returns true if the evaluation history vector is to be reserved, and false otherwise.

Definition at line 272 of file gradient_descent.cpp.

const bool & OpenNN::GradientDescent::get_reserve_gradient_history ( void   )  const

This method returns true if the gradient history vector of vectors is to be reserved, and false otherwise.

Definition at line 282 of file gradient_descent.cpp.

const bool & OpenNN::GradientDescent::get_reserve_gradient_norm_history ( void   )  const

This method returns true if the gradient norm history vector is to be reserved, and false otherwise.

Definition at line 292 of file gradient_descent.cpp.

const bool & OpenNN::GradientDescent::get_reserve_generalization_evaluation_history ( void   )  const

This method returns true if the Generalization evaluation history vector is to be reserved, and false otherwise.

Definition at line 332 of file gradient_descent.cpp.

const bool & OpenNN::GradientDescent::get_reserve_training_direction_history ( void   )  const

This method returns true if the training direction history matrix is to be reserved, and false otherwise.

Definition at line 302 of file gradient_descent.cpp.

const bool & OpenNN::GradientDescent::get_reserve_training_rate_history ( void   )  const

This method returns true if the training rate history vector is to be reserved, and false otherwise.

Definition at line 312 of file gradient_descent.cpp.

const bool & OpenNN::GradientDescent::get_reserve_elapsed_time_history ( void   )  const

This method returns true if the elapsed time history vector is to be reserved, and false otherwise.

Definition at line 322 of file gradient_descent.cpp.

const unsigned int & OpenNN::GradientDescent::get_display_period ( void   )  const

This method returns the number of epochs between the training showing progress.

Definition at line 342 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_training_rate_algorithm ( const TrainingRateAlgorithm new_training_rate_algorithm  ) 

This method sets a new training rate algorithm object into the gradient descent object.

Parameters:
new_training_rate_algorithm Object of the class TrainingRateAlgorithm

Definition at line 353 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_default ( void   )  [virtual]

This method sets the members of the training algorithm object to their default values.

Reimplemented from OpenNN::TrainingAlgorithm.

Definition at line 361 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_reserve_all_training_history ( const bool &  new_reserve_all_training_history  ) 

This method makes the training history of all variables to reseved or not in memory:

  • Parameters.
  • Parameters norm.
  • Performance.
  • Gradient.
  • Gradient norm.
  • Generalization performance.
  • Training direction.
  • Training direction norm.
  • Training rate.
Parameters:
new_reserve_all_training_history True if the training history of all variables is to be reserved, false otherwise.

Definition at line 422 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_warning_parameters_norm ( const double &  new_warning_parameters_norm  ) 

This method sets a new value for the parameters vector norm at which a warning message is written to the screen.

Parameters:
new_warning_parameters_norm Warning norm of parameters vector value.

Definition at line 452 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_warning_gradient_norm ( const double &  new_warning_gradient_norm  ) 

This method sets a new value for the gradient vector norm at which a warning message is written to the screen.

Parameters:
new_warning_gradient_norm Warning norm of gradient vector value.

Definition at line 483 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_warning_training_rate ( const double &  new_warning_training_rate  ) 

This method sets a new training rate value at wich a warning message is written to the screen during line minimization.

Parameters:
new_warning_training_rate Warning training rate value.

Definition at line 514 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_error_parameters_norm ( const double &  new_error_parameters_norm  ) 

This method sets a new value for the parameters vector norm at which an error message is written to the screen and the program exits.

Parameters:
new_error_parameters_norm Error norm of parameters vector value.

Definition at line 543 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_error_gradient_norm ( const double &  new_error_gradient_norm  ) 

This method sets a new value for the gradient vector norm at which an error message is written to the screen and the program exits.

Parameters:
new_error_gradient_norm Error norm of gradient vector value.

Definition at line 574 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_error_training_rate ( const double &  new_error_training_rate  ) 

This method sets a new training rate value at wich a the line minimization algorithm is assumed to fail when bracketing a minimum.

Parameters:
new_error_training_rate Error training rate value.

Definition at line 605 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_minimum_parameters_increment_norm ( const double &  new_minimum_parameters_increment_norm  ) 

This method sets a new value for the minimum parameters increment norm stopping criterium.

Parameters:
new_minimum_parameters_increment_norm Value of norm of parameters increment norm used to stop training.

Definition at line 635 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_minimum_performance_increase ( const double &  new_minimum_performance_increase  ) 

This method sets a new minimum performance improvement during training.

Parameters:
new_minimum_performance_increase Minimum improvement in the performance between two epochs.

Definition at line 665 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_performance_goal ( const double &  new_performance_goal  ) 

This method sets a new goal value for the performance. This is used as a stopping criterium when training a multilayer perceptron

Parameters:
new_performance_goal Goal value for the performance.

Definition at line 696 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_gradient_norm_goal ( const double &  new_gradient_norm_goal  ) 

This method sets a new the goal value for the norm of the objective function gradient. This is used as a stopping criterium when training a multilayer perceptron

Parameters:
new_gradient_norm_goal Goal value for the norm of the objective function gradient.

Definition at line 708 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_maximum_generalization_evaluation_decreases ( const unsigned int &  new_maximum_generalization_evaluation_decreases  ) 

This method sets a new maximum number of generalization failures.

Parameters:
new_maximum_generalization_evaluation_decreases Maximum number of epochs in which the generalization evalutation decreases.

Definition at line 738 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_maximum_epochs_number ( const unsigned int &  new_maximum_epochs_number  ) 

This method sets a maximum number of epochs for training.

Parameters:
new_maximum_epochs_number Maximum number of epochs for training.

Definition at line 768 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_maximum_time ( const double &  new_maximum_time  ) 

This method sets a new maximum training time.

Parameters:
new_maximum_time Maximum training time.

Definition at line 798 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_reserve_parameters_history ( const bool &  new_reserve_parameters_history  ) 

This method makes the parameters history vector of vectors to be reseved or not in memory.

Parameters:
new_reserve_parameters_history True if the parameters history vector of vectors is to be reserved, false otherwise.

Definition at line 828 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_reserve_parameters_norm_history ( const bool &  new_reserve_parameters_norm_history  ) 

This method makes the parameters norm history vector to be reseved or not in memory.

Parameters:
new_reserve_parameters_norm_history True if the parameters norm history vector is to be reserved, false otherwise.

Definition at line 839 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_reserve_evaluation_history ( const bool &  new_reserve_evaluation_history  ) 

This method makes the evaluation history vector to be reseved or not in memory.

Parameters:
new_reserve_evaluation_history True if the evaluation history vector is to be reserved, false otherwise.

Definition at line 850 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_reserve_gradient_history ( const bool &  new_reserve_gradient_history  ) 

This method makes the gradient history vector of vectors to be reseved or not in memory.

Parameters:
new_reserve_gradient_history True if the gradient history matrix is to be reserved, false otherwise.

Definition at line 861 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_reserve_gradient_norm_history ( const bool &  new_reserve_gradient_norm_history  ) 

This method makes the gradient norm history vector to be reseved or not in memory.

Parameters:
new_reserve_gradient_norm_history True if the gradient norm history matrix is to be reserved, false otherwise.

Definition at line 873 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_reserve_generalization_evaluation_history ( const bool &  new_reserve_generalization_evaluation_history  ) 

This method makes the Generalization evaluation history to be reserved or not in memory. This is a vector.

Parameters:
new_reserve_generalization_evaluation_history True if the Generalization evaluation history is to be reserved, false otherwise.

Definition at line 921 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_reserve_training_direction_history ( const bool &  new_reserve_training_direction_history  ) 

This method makes the training direction history vector of vectors to be reseved or not in memory.

Parameters:
new_reserve_training_direction_history True if the training direction history matrix is to be reserved, false otherwise.

Definition at line 885 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_reserve_training_rate_history ( const bool &  new_reserve_training_rate_history  ) 

This method makes the training rate history vector to be reseved or not in memory.

Parameters:
new_reserve_training_rate_history True if the training rate history vector is to be reserved, false otherwise.

Definition at line 897 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_reserve_elapsed_time_history ( const bool &  new_reserve_elapsed_time_history  ) 

This method makes the elapsed time over the epochs to be reseved or not in memory. This is a vector.

Parameters:
new_reserve_elapsed_time_history True if the elapsed time history vector is to be reserved, false otherwise.

Definition at line 909 of file gradient_descent.cpp.

void OpenNN::GradientDescent::set_display_period ( const unsigned int &  new_display_period  ) 

This method sets a new number of epochs between the training showing progress.

Parameters:
new_display_period Number of epochs between the training showing progress.

Definition at line 933 of file gradient_descent.cpp.

Vector< double > OpenNN::GradientDescent::calculate_training_direction ( const Vector< double > &  gradient  )  const

This method returns the gradient descent training direction, which is the negative of the normalized gradient.

Definition at line 962 of file gradient_descent.cpp.

GradientDescent::GradientDescentResults * OpenNN::GradientDescent::perform_training ( void   )  [virtual]

Todo:

Implements OpenNN::TrainingAlgorithm.

Definition at line 1086 of file gradient_descent.cpp.

std::string OpenNN::GradientDescent::write_training_algorithm_type ( void   )  const [virtual]

This method writes a string with the type of training algoritm.

Reimplemented from OpenNN::TrainingAlgorithm.

Definition at line 1408 of file gradient_descent.cpp.

TiXmlElement * OpenNN::GradientDescent::to_XML ( void   )  const [virtual]

This method prints to the screen the training parameters, the stopping criteria and other user stuff concerning the gradient descent object.

Reimplemented from OpenNN::TrainingAlgorithm.

Definition at line 1419 of file gradient_descent.cpp.

void OpenNN::GradientDescent::from_XML ( TiXmlElement *  training_algorithm_element  )  [virtual]

This method loads a training algorithm object from a XML element.

Parameters:
training_algorithm_element Pointer to a Tiny XML element containing the training algorithm members.

Reimplemented from OpenNN::TrainingAlgorithm.

Definition at line 1699 of file gradient_descent.cpp.


The documentation for this class was generated from the following files:

Generated on Sun Aug 26 11:58:20 2012 for OpenNN by  doxygen 1.5.9