Abstract: Deep neural networks (DNNs) with the flexibility to learn good top-layer representations have eclipsed shallow kernel methods without that flexibility. Here, we take inspiration from DNNs to develop the deep kernel machine. Optimizing the deep kernel machine objective is equivalent to exact inference in an infinitely wide Bayesian neural network or deep Gaussian process, which has been scaled carefully to retain representation learning. We conjecture that the deep kernel machine objective is unimodal, and give a proof of unimodality for linear kernels. We describe a fast optimizer that uses the Continuous Algebraic Ricatti Equations from control theory to converge in ~10 steps, and we show superior predictive performance over alternative approaches to learning the kernel.

Speakers:  Laurence Aitchison is a Senior Lecturer in Computer Science at the University of Bristol's Computational Neuroscience Unit. His group is on a mission to "Make Bayes Great Again" by showing that many of the things done in deep learning are Bayesian, and developing new Bayesian models with the power and flexibility of neural networks.

Affiliation: University of Bristol

Place of Seminar:  Zoom
Meeting ID: 639 8752 5937
Passcode: 170532