On Some Connections between Nonlinear Filtering, Information Theory, & Statistical Mechanics

ECE Seminar: On Some Connections between Nonlinear Filtering, Information Theory, & Statistical Mechanics

Starts at: April 25, 2013 4:30 PM

Ends at: 5:30 PM

Location: Scaife Hall Auditorium Room 125 at 4:30 p.m. Refreshments at 4:00 p.m.

Speaker: Sanjoy Mitter

Affiliation: Massachusetts Institute of Technology

Link to Abstract

Link to Audio

Link to Video (1)



In this talk I discuss the problem of recursively estimating the state of a noisy dynamical system form noisy observations of the state. When the noise is addiitive and Gaussian the estimator is the celebrated Kalman Filter. I analyze the Filter from an Information Theory viewpoint and show that the filter stores the "right" amount of information and dissipates historical information at a rate governed by the Fisher Information. These ideas are related to the well known result in Statistical Mechanics where the state of a thermodynamic system in equilibrium with a heat bath is given by a Gibbs distribution. In turn, the Gibbs distribution satisfies a Variational Principle, namely it corresponds to the state which minimizes the Free Enrgy . The Free Energy has the interpretation of the energy which is available for doing work . These ideas can be generalized to the case where the state evolution is described by a nonlinear noisy dynamical system and the observations are noisy nonlinear functions of the state. Joint work with Nigel Newton, University of Essex.


Sanjoy K. Mitter received his Ph.D. degree from the Imperial College of Science and Technology in 1965. He taught at Case Western Reserve University from 1965 to 1969. He joined MIT in 1969 where he has been a Professor of Electrical Engineering since 1973. He was the Director of the MIT Laboratory for Information and Decision Systems from 1981 to 1999. He has also been a Professor of Mathematics at the Scuola Normale, Pisa, Italy from 1986 to 1996. He has held visiting positions at Imperial College, London; University of Groningen, Holland; INRIA, France; Tata Institute of Fundamental Research, India and ETH, Zürich, Switzerland; and several American universities. Professor Mitter was the Ulam Scholar at Los Alamos National Laboratories in April 2012 and the John von Neumann Visiting Professor in Mathematics at the Technical University of Munich, Germany from May-June 2012. He was awarded the AACC Richard E. Bellman Control Heritage Award for 2007. He was the McKay Professor at the University of California, Berkeley in March 2000, and held the RussellSeverance-Springer Chair in Fall 2003. He is a Fellow of the IEEE and a Member of the National Academy of Engineering. He is the winner of the 2000 IEEE Control Systems Award. He was elected a Foreign Member of Istituto Veneto di Scienze, Lettere ed Arti in 2003. In 1988, he was elected to the National Academy of Engineering. His current research interests are Communication and Control in a Networked Environment, the relationship of Statistical and Quantum Physics to Information Theory.