From Camera Array to CrowdCam

Abstract:

Multi camera systems significantly evolved over time and have been used to solve a variety of problems in computer vision. We will consider a particular use of a camera array to recover accurate 3D structure and 3D motion of a dynamic scene. Traditionally, camera arrays are carefully assembled in the lab, and are controlled by a single user - the photographer. But recently the way we capture images changed before our eyes. One can often see a group of people, armed with smartphones, huddling together to take pictures of some exciting dynamic event. The data obtained this way can be regarded as the output of a new type of an ad-hoc camera array, which we call a crowd-based camera (or CrowdCam). Different from traditional camera array, CrowdCam is operated by multiple photographers, and there is no single moment of capture. Moreover, the data obtained by CrowdCam lacks accurate temporal information since the cameras cannot be assumed to be calibrated or synchronized. We are interested in developing tools that analyze, explore and visualize CrowdCamimages and a first step in this direction is to recover the temporal order of the images. We term this problem photo sequencing and present a geometry-based solution to it. Finally, future applications and following challenges will be presented.

Continuous Graphical Models for Protein Conformation Spaces

Generative models of protein structure enable researchers to predict the behavior of proteins under different conditions. These models, for instance, can help with the discovery of important pathways in proteins involved in virus infection process, and provide researchers with target regions that may be better candidates for drug development.

In this talk, we will focus on a series of graphical models developed to handle the challenges specific to the protein conformation modeling, which include ability to handle angular, multi-modal variables in very large datasets. We will first present von Mises graphical models which can handle angular variables. We then describe non-paranormal graphical models, also known as Gaussian copula, which can model arbitrary distributions and in theory, handle multi-modality. Finally we will present non-parametric kernel space embedded graphical models, which can handle multi-modal, arbitrary distributions via kernel density estimation. We will also briefly mention our solutions to make the kernel embedded models more scalable.

Better generalization with forecasts

Abstract:

Prediction has been shown to be a promising principle on which an autonomous reinforcement learning agent might base its knowledge of the world. One particularly powerful predictive representational mechanism introduced recently is the general value function (GVF), which has greater flexibility than previous predictive methods for capturing the regularities the agent may encounter in its environment. We investigate the generalization abilities of these GVFs (or “forecasts”) by generating a focused subclass and testing them within a set of environments in which generalization can be readily measured. We compare the results with a closely related predictive method (PSRs) already shown to have good generalization ability. Our results show that GVFs provide a substantial improvement in generalization, producing features that lead to better value-function approximation than PSRs and better generalization to as-yet-unseen parts of the state space.

(This is joint work with Mark Ring)

Temporal Dynamics and Information Retrieval

Abstract: Many digital resources, like the Web, are dynamic and ever-changing collections of information. However, most information retrieval tools developed for interacting with Web content, such as browsers and search engines, focus on a single static snapshot of the information. In this talk, I will present analyses of how Web content changes over time, how people re-visit Web pages over time, and how re-visitation patterns are influenced by changes in user intent and content. These results have implications for many aspects of information retrieval and management including crawling policy, ranking and information extraction algorithms, result presentation, and systems evaluation. I will describe a prototype that supports people in understanding how the information they interact with changes over time, and new information retrieval models that incorporate features about the temporal evolution of content to improve core ranking. Finally, I will conclude with an overview of some general challenges that need to be addressed to fully incorporate temporal dynamics in information retrieval and information management systems.

Bio: Susan Dumais is a Principal Researcher and manager of the Context, Learning and User Experience for Search (CLUES) Group at Microsoft Research. Prior to joining Microsoft Research, she was at Bellcore and Bell Labs for many years, where she worked on Latent Semantic Indexing (a statistical method for concept-based retrieval), interfaces for combining search and navigation, and organizational impacts of new technology. Her current research focuses on user modeling and personalization, context and information retrieval, temporal dynamics of information, interactive retrieval, and novel evaluation methods. She has worked closely with several Microsoft groups (Bing, Windows Desktop Search, SharePoint Portal Server, and Office Online Help) on search-related innovations. Susan has published more than 200 articles in the fields of information science, human-computer interaction, and cognitive science, and holds several patents on novel retrieval algorithms and interfaces. Susan is also an adjunct professor in the Information School at the University of Washington. She is Past-Chair of ACM's Special Interest Group in Information Retrieval (SIGIR), and serves on several editorial boards, technical program committees, and government panels. She was elected to the CHI Academy in 2005, an ACM Fellow in 2006, received the SIGIR Gerard Salton Award for Lifetime Achievement in 2009, and was elected to the National Academy of Engineering (NAE) in 2011.

Single-Loop Surrogate Likelihood Training of Graphical Models

This talk will describe a faster “single-loop” method for training Conditional Random Fields (CRFs). CRFs are used to model problems in computer vision, natural language processing and computational biology. In general, exact learning and inference in such models is intractable. As such, when learning CRFs, it is common to use approximate message-passing inference algorithms to estimate gradients of a “surrogate likelihood” objective. However, this is a rather slow double-loop optimization, where message-passing must be iterated to convergence to estimate each likelihood gradient. I will show how, with tree-reweighted belief propagation, one may reformulate the learning objective into an equivalent single-loop optimization, where inference and learning are solved simultaneously. Experiments on computer vision and multi-label prediction suggest that this approach is faster and more reliable than double-loop learning.