Lecture 5: Entropy and Data Compression (IV): Shannon's Source Coding Theorem, Symbol Codes

Lecture 5 of the Course on Information Theory, Pattern Recognition, and Neural Networks. Produced by: David MacKay (University of Cambridge) Author: David MacKay, University of Cambridge A series of sixteen lectures covering the core of the book "Information Theory, Inference, and Learning Algorithms" (Cambridge University Press, 2003, http://www.inference.eng.cam.ac.uk/ma...) which can be bought at Amazon (http://www.amazon.co.uk/exec/obidos/A..., and is available free online (http://www.inference.eng.cam.ac.uk/ma.... A subset of these lectures used to constitute a Part III Physics course at the University of Cambridge. The high-resolution videos and all other course material can be downloaded from the Cambridge course website (http://www.inference.eng.cam.ac.uk/ma.... Snapshots of the lecture can be found here: http://www.inference.eng.cam.ac.uk/it... These lectures are also available at http://videolectures.net/course_infor... (synchronized with snapshots and slides)