Regularization with a pruning prior

Cyril Goutte

      [ Request for reprint should be sent to
        Cyril Goutte, Electronics Institute, Bygn. 349, Technical
        University of Denmark, DK-2800 Lyngby, Denmark. Phone: +45 4525
        5738. Fax: +45 4288 0117. Email: goutte@ei.dtu.dk. Cyril
        Goutte is a visiting student, 1995-1996, 
        with a grant from the Danish Rector's Conference.
        Permanent adress: Equipe connexionniste - LAFORIA,
        Universit\'e Paris VI, 4 place Jussieu,
        75252 Paris Cedex 05, France.
        Email: goutte@laforia.ibp.fr]
        
        
Lars Kai Hansen

      [ This research is supported by the Danish
        National Councils for the Natural and Technical Sciences through the
        Danish National Network Center. Both authors wish to
        acknowledge stimulating discussions with Jan Larsen.]


We investigate the use of a regularization prior that we show has
pruning properties.
Analyses are conducted both using a Bayesian framework and with the
generalization method, on a simple toy problem.
Results are thoroughly compared with those obtained with a traditional
weight decay.

Keywords: Regularization, Generalization method, Bayesian
learning, Evidence framework, Laplace prior, Comparison with weight decay.