Overview

This volume aims to develop understanding of theoretical and practical issues involved in the development of efficient MLP training strategies, and to describe and evaluate the performance of a wide range of specific training algorithm. Particular emphasis is given to the development of methods which have a strong theoretical foundation, rather than heuristic, ""rule of thumb"" training strategies. ""Second order methods for neural networks"" should be of interest to academic researchers and postgraduate students working with neural networks (especially supervised learning with multi-layer perceptrons), industrial researchers and programmers developing neural network software, and professionals using neural networks as optimisation tools.

Full Product Details

Author:   Adrian J. Shepherd
Publisher:   Springer-Verlag Berlin and Heidelberg GmbH & Co. KG
Imprint:   Springer-Verlag Berlin and Heidelberg GmbH & Co. K
Edition:   1997 ed.
Dimensions:   Width: 15.50cm , Height: 0.90cm , Length: 23.50cm
Weight:   0.255kg
ISBN:  

9783540761006

ISBN 10:   3540761004
Pages:   145
Publication Date:   28 April 1997
Audience:   College/higher education ,  Professional and scholarly ,  Postgraduate, Research & Scholarly ,  Professional & Vocational
Format:   Paperback
Publisher's Status:   Active
Availability:   In Print  
This item will be ordered in for you from one of our suppliers. Upon receipt, we will promptly dispatch it out to you. For in store availability, please contact us.

Table of Contents

1 Multi-Layer Perceptron Training.- 1.1 Introduction to MLPs.- 1.1.1 The MLP Architecture.- 1.1.2 MLP Training.- 1.2 Error Surfaces and Local Minima.- 1.2.1 Error Surface Fundamentals.- 1.2.2 MLP Error Surfaces.- 1.3 Backpropagation.- 1.3.1 An Introduction to Backpropagation.- 1.3.2 The Bold Driver Method.- 1.3.3 Backpropagation with Momentum.- 1.3.4 On-Line Backpropagation.- 1.3.5 The Delta-Bar-Delta Method.- 2 Classical Optimisation.- 2.1 Introduction to Classical Methods.- 2.1.1 The Linear Model and Steepest Descent.- 2.1.2 The Quadratic Model and Newton's Method.- 2.1.3 Line-Search Methods vs. Model-Trust Region Methods.- 2.2 General Numerical Considerations.- 2.2.1 Finite-Precision Arithmetic and Computational Errors.- 2.2.2 Positive Definiteness and the Model Hessian.- 2.2.3 Scaling and Preconditioning.- 3 Second-Order Optimisation Methods.- 3.1 Line-Search Strategies.- 3.1.1 Line Minimisation Fundamentals.- 3.1.2 Inaccurate Line Searches.- 3.1.3 Backtracking Line Searches.- 3.2 Model-Trust Region Strategies.- 3.2.1 A Simple Model-Trust Region Algorithm.- 3.2.2 Fletcher's Method.- 3.2.3 Modern Model-Trust Region Algorithms.- 3.2.4 Moller's `Scaled' Model-Trust Region Strategy.- 3.3 Multivariate Methods for General Nonlinear Optimisation.- 3.3.1 Finite-Difference Newton's Method.- 3.3.2 Quasi-Newton Methods.- 3.3.3 The `Memoryless' Quasi-Newton Method.- 3.3.4 Conjugate Gradient Methods.- 3.4 Special Methods for Nonlinear Least Squares.- 3.4.1 The Gauss-Newton Method.- 3.4.2 The Levenberg-Marquardt Method.- 3.5 Comparison of Methods.- 4 Second-Order Training Methods for MLPs.- 4.1 The Calculation of Second Derivatives.- 4.1.1 Exact Evaluation of the Hessian Matrix.- 4.1.2 Exact Evaluation of the Hessian Times a Vector.- 4.1.3 Exact Evaluation of the Jacobian Matrix.- 4.2 Reducing Storage and Computational Costs.- 4.2.1 Diagonal Approximations of the Hessian Matrix.- 4.2.2 Reduced Function and Gradient Evaluations.- 4.3 Second-Order On-Line Training.- 4.3.1 An Introduction to Second-Order On-Line Training Strategies.- 4.3.2 `Noise-Free' On-Line Training Schemes.- 4.4 Conclusion.- 5 An Experimental Comparison of MLP Training Methods.- 5.1 Benchmark Training Tasks.- 5.1.1 N-Parity.- 5.1.2 The sin(x) Problem.- 5.1.3 The sin(x)cos(2x) Problem.- 5.1.4 The tan(x) Problem.- 5.2 Initial Training Conditions.- 5.2.1 MLP Architectures.- 5.2.2 Training Algorithms.- 5.2.3 Termination Conditions.- 5.3 Experimental Results.- 5.3.1 `Global Reliability'.- 5.3.2 Training Speed Metrics.- 5.3.3 Training Speed Results.- 5.3.4 Conclusions.- 6 Global Optimisation.- 6.1 Introduction to Global Methods.- 6.1.1 Stochastic Methods.- 6.1.2 Deterministic Methods.- 6.2 Expanded Range Approximation (ERA).- 6.2.1 An Introduction to ERA.- 6.2.2 The ERA Method in Practice.- 6.3 The TRUST Method.- 6.3.1 The Tunnelling Function.- 6.3.2 The Tunnelling Phase.- 6.3.3 An Evaluation of the TRUST Method.

Reviews

Author Information

Tab Content 6

Author Website:  

Customer Reviews

Recent Reviews

No review item found!

Add your own review!

Countries Available

All regions