Research Projects, Past and Present

Unsupervised Deep Learning

  • Time Period: September 2004 - present
  • Participants: Marc'Aurelio Ranzato, Koray Kavukcuoglu, Y-Lan Boureau, Yann LeCun (Courant Institute/CBLL).
  • Sponsors: ONR, NSF
  • Description: Animals and humans can learn to see, perceive, act, and communicate with an efficiency that no Machine Learning method can approach. The brains of humans and animals are “deep”, in the sense that each action is the result of a long chain of synaptic communications (many layers of processing). We are currently researching efficient learning algorithms for such “deep architectures”. We are currently concentrating on unsupervised learning algorithms that can be used to produce deep hierarchies of features for visual recognition. We surmise that understanding deep learning will not only enable us to build more intelligent machines, but will also help us understand human intelligence and the mechanisms of human learning.


Relational Regression

  • Time Period: September 2006 - present.
  • Participants: Sumit Chopra, Yann LeCun (Courant Institute/CBLL), Trivikraman Thampy, John Leahy, Andrew Caplin (Economics Dept, NYU).
  • Sponsors: NYU
  • Description: We are developing a new type of relational graphical models that can be applied to “structured regression problem”. A prime example of structured regression problem is the prediction of house prices. The price of a house depends not only on the characteristics of the house, but also of the prices of similar houses in the neighborhood, or perhaps on hidden features of the neighborhood that influence them. Our relational regression model infers a hidden “desirability sruface” from which house prices are predicted.


Energy-Based Models

  • Time Period: September 2003 - present.
  • Participants: Yann LeCun, Sumit Chopra, Marc'Aurelio Ranzato, Y-Lan Boureau, Fu-Jie Huang (Courant Institute/CBLL)
  • Sponsors: NSF.
  • Description: Probabilistic graphical models associate a probability to each configuration of the relevant variables. Energy-based models (EBGM) associate an energy to those configurations, eliminating the need for proper normalization of probability distributions. Making a decision (an inference) with an EBM consists in comparing the energies associated with various configurations of the variable to be predicted, and choosing the one with the smallest energy. Such systems must be trained discriminatively to associate low energies to the desired configurations and higher energies to undesired configurations. A wide variety of loss function can be usedo for this purpose. We give sufficient conditions that a loss function should satisfy so that its minimization will cause the system to approach to desired behavior.
  • Latest Publication:
    • [LeCun et al 2006]. A Tutorial on Energy-Based Learning, in Bakir et al. (eds) “Predicting Structured Outputs”, MIT Press 2006.


LAGR: Learning Applied to Ground Robots



DAVE: Learning Vision-Based Obstacle Avoidance for Mobile Robots


  • Time Period: September 2003 - June 2004.
  • Participants: Yann LeCun (Courant Institute/CBLL), Eric Cosatto, Jan Ben, Urs Muller, Beat Flepp (Net-Scale Technologies).
  • Sponser: DARPA
  • Description: We built a small off-road robot that uses an end-to-end learning system to avoid obstacles solely from visual input. The robot transmits analog video from two video cameras to a remote computer, which drives the robot through remote control. The driving software is built around a convolutional network which is trained to produce the steering angle of a human driver from raw video frames.

This project served as a proof-of-concept for the LAGR project.


NORB: Generic Object Recognition in Images

  • Time Period: September 2003 - present.
  • Participants: Fu Jie Huang, Yann LeCun (Courant Institute/CBLL), Leon Bottou (NEC Labs).
  • Sponsers: NSF.
  • Description: The recognition of generic object categories with invariance to pose, lighting, diverse backgrounds, and the presence of clutter is one of the major challenges of Computer Vision. We are developing learning systems that can recognize generic object purely from their shape, independently of pose, illumination, and surrounding clutter.
  • Publications:


Automatic Phenotyping

  • Time Period: January 2004 - present.
  • Participants: Feng Ning, Yann LeCun (Courant Institute/CBLL), Leon Bottou (NEC Labs), Fabio Piano (NYU, Biology Dept), Paolo Barbano (Yale University).
  • Description: We are using convolutional networks and conditional random fields to automatically segment movies of developing C. Elegans into regions: cell nucleus, nucleus membrane, cytoplasm, cell membrane.

Simultaneous Face Detection and Pose Estimation


  • Time Period: September 2003-June 2004.
  • Participants: Margarita Osadchy (NEC Labs, Technion), Matt Miller (NEC Labs), Yann LeCun (Courant Institute/CBLL).
  • Description: We developed a novel method for real-time, simultaneous multi-view face detection and facial pose estimation. The method employs a convolutional network that maps face images to points on a manifold, parameterized by pose, and non-face images to points far from that manifold. This system is trained as an “Energy-Based Model” with a discriminative loss function.
  • Publications:


research/research.txt · Last modified: 2009/02/08 22:28 by koray