December 7th, 2011 @ 1130am Youngmin Cho (UCSD)

Kernel Methods for Deep Learning

Abstract: We introduce a new family of positive-definite kernels that mimic the computation in large neural networks. We explore various types of these kernels by using different activation functions in the neural networks. We also show how to derive new kernels, by recursive composition, that may be viewed as mapping their inputs through a series of nonlinear feature spaces. These recursively derived kernels mimic the computation in deep networks with multiple hidden layers. These kernel functions can be used in shallow architectures, such as support vector machines (SVMs), or in deep kernel-based architectures that we call multilayer kernel machines (MKMs). We evaluate SVMs and MKMs with these kernel functions on problems designed to illustrate the advantages of deep architectures. On several problems, we obtain better results than previous, leading benchmarks from both SVMs with Gaussian kernels as well as deep belief nets.

seminars/seminaritems/2011-12-07.txt · Last modified: 2011/12/05 16:20 by silberman