Title: Adaptive Regularization of Neural Classifiers


Authors: L. Nonboe Andersen, J. Larsen, L.K. Hansen & M. Hintz-Madsen
         CONNECT, Department of Mathematical Modelling, Building 321
         Technical University of Denmark
         DK-2800 Lyngby, Denmark
         Phones: +45  4525+ext. 3899,3923,3889,3894
         Fax: +45 45872599
         emails: lna,jl,lkhansen,mhm@imm.dtu.dk

Abstract:
In this paper we present a regularization scheme which iteratively 
adapts the regularization parameters by minimizing the validation 
error. It is suggested to use the adaptive regularization scheme in
conjunction with Optimal Brain Damage pruning to optimize the 
architecture and to avoid overfitting. Furthermore, we propose an 
improved neural classification architecture eliminating an inherent 
redundancy in the widely used SoftMax classification network.
Numerical results demonstrate the viability of the method.


In proccedings of the IEEE Workshop on Neural Networks for Signal
Processing VII, IEEE Press, 1997. NNSP'97 is taking palce in  
Florida, September, 1997.