TITLE:   Linear Unlearning for Cross-Validation
         Preprint, accepted for Advances in Computational Mathematics 1996

AUTHORS: Lars Kai Hansen and Jan Larsen
         CONNECT, Electronics Institute B349
         Technical University of Denmark
         DK-2800 Lyngby, Denmark
         Phones: (+45) 45253889, (+45) 45253923
         Fax: (+45) 45880117
         Emails: lkhansen,jlarsen@ei.dtu.dk

ABSTRACT:
The leave-one-out cross-validation scheme for generalization assessment of
neural network models is computationally expensive due to replicated training
sessions. In this paper we suggest linear unlearning of examples
as an approach to approximative cross-validation. Further, we discuss the
possibility of exploiting the ensemble of networks offered by leave-one-out
for performing ensemble predictions. We show that the generalization
performance of the equally weighted ensemble predictor is identical to that
of the network trained on the whole training set.

Numerical experiments on the sunspot time series prediction benchmark
demonstrates the potential of the linear unlearning technique.