Finally!! What you have all been waiting for during the last three years (at least I have) is now available: My thesis! (2.7 Mb gzip'ed file, 15 Mb uncompressed, 330 pages). Here is the abstract if you prefer a small preview before you download. The title of the thesis is Optimization of Recurrent Neural Networks for Time Series Modeling.

My remaining list of publications includes:

Conference contribution to ICANN 94, Controlled Growth of the Cascade Correlation Algorithm where I have used the OBS pruning scheme to limit the number of weights in a Cascade Correlation net classifying the two-spirals problem.

Conference contribution to NIPS 94, Recurrent Networks: Second Order Properties and Pruning where I derive the second derivatives of the quadratic cost function and prune a recurrent net using the OBS pruning scheme.

Conference contribution to NIPS 95, Pruning with generalization based weight saliencies: gammaOBD, gammaOBS in which we suggest to calcutate the weight saliencies for pruning algorithms as the change in the estimated test error instead of the usual change in training error. An experiment based on the Mackey-Glass chaotic time series is presented.

Conference contribution to NNSP'96, Design and Evaluation of Neural Classifiers.
Abstract:
In this paper we propose a method for design of feed-forward neural classifiers based on regularization and adaptive architectures. Using a penalized maximum likelihood scheme we derive a modified form of the entropic error measure and an algebraic estimate of the test error. In conjunction with Optimal Brain Damage pruning the test error estimate is used to optimize the network architecture. The scheme is evaluated on an artificial and a real world problem.
Also available in an extended version submitted for IEEE Transactions on Signal Processing.

Conference contribution to Asilomar 96, Pruning Boltzmann Networks and Hidden Markov Models where we derive the second derivative of the Kullback-Leibler measure, or relative entropy, for general Boltzmann networks for training using second-order methods and pruning. We discuss how to prune HMMs by converting these into equivalent Boltzmann networks called Boltzmann chains, and illustate pruning on a Boltzmann structure called Boltzmann zippers.

Conference contribution for NNSP'97, Training Recurrent Networks.
Abstract:
Training recurrent networks is generally believed to be a difficult task. Excessive training times and lack of convergence to an acceptable solution are frequently reported. In this paper we seek to explain the reason for this from a numerical point of view and show how to avoid problems when training. In particular we investigate ill-conditioning, the need for and effect of regularization and illustrate the superiority of second-order methods for training.

Conference contribution for NNSP'97, Interpretation of Recurrent Neural Networks.
Abstract:
This paper addresses techniques for interpretation and characterization of trained recurrent nets for time series problems. In particular, we focus on assessment of memory and suggest an operational definition of memory. Further we discuss the evaluation of learning curves. Various numerical experiments on a time series prediction problem is used to illustrate the potential of the suggested methods.

Conference submission to NIPS*97,
Second-Order Methods in Boltzmann Learning: An Application to Speechreading.
Abstract:
We introduce second-order methods for training and pruning of general Boltzmann networks trained with cross-entropy error. In particular, we derive the second derivatives for the entropic cost function. We illustrate pruning on Boltzmann zippers, applied to real-world data --- a speechreading (lipreading) problem.

In addition, I have written a note on how to invoke MATLAB from your C-programs (in danish).