Information theory is a fascinating topic that informs many of the ways we think about structure in data. This video provides a brief overview of some of the concepts, inspired by the lectures of my PhD advisor, the late David MacKay. This is part of a series of videos for COS 302: Mathematics for Numerical Computation and Machine Learning, replacing lectures after the course went remote due to the COVID-19 pandemic.
Laboratory for Intelligent Probabilistic Systems
Princeton University Department of Computer Science