TITLE: Adaptive Regularization of Neural Networks using Conjugate Gradient

AUTHOR: Cyril Goutte and Jan Larsen
Department of Mathematical Modelling, Building 321
Technical University of Denmark, DK-2800 Lyngby, Denmark
emails: cg,jl@imm.dtu.dk
www: http://eivind.imm.dtu.dk


Recently we suggested a regularization scheme which iteratively adapts regularization parameters by minimizing validation error using simple gradient descent. In this contribution we present an improved algorithm based on the conjugate gradient technique. Numerical experiments with feed-forward neural networks successfully demonstrate improved generalization ability and lower computational cost.

Preprint, appears in Proceedings of the Intl. Conference on Acoustics, Speech and Signal Processing (ICASSP'98, Seattle), vol. 2, pp. 1201-1204.

Download: abstract, Compressed Postscript.