TITLE: Adaptive Regularization of Neural Networks using Conjugate Gradient
AUTHOR:
Cyril Goutte and
Jan Larsen
Department of Mathematical Modelling, Building 321
Technical University of Denmark, DK-2800 Lyngby, Denmark
emails: cg,jl@imm.dtu.dk
www: http://eivind.imm.dtu.dk
ABSTRACT:
Recently we suggested a regularization scheme which iteratively adapts
regularization parameters by minimizing validation error using simple
gradient descent. In this contribution we present an improved
algorithm based on the conjugate gradient technique. Numerical
experiments with feed-forward neural networks successfully demonstrate
improved generalization ability and lower computational cost.
Preprint, appears in Proceedings of the Intl. Conference on Acoustics,
Speech and Signal Processing (ICASSP'98, Seattle), vol. 2, pp. 1201-1204.
Download:
abstract,
Compressed
Postscript.