Electrical and Computer Engineering 7680

Information Theory

Homepage - Fall 2000

• Instructor: Todd K. Moon,  Associate Professor,  EL 115
• Syllabus:  Class Syllabus
• Class Time:  Tues, Thurs at 10:30 - 11:45  in EL126
• Primay Text Book:  Elements of Information Theory by Thomas M. Cover and Joy A. Thomas
• Supplementary Texts: 
• Information Theory, Inference, and Learning Algorithms by David J.C. Mackay
• Silicon Dreams by Robert W. Lucky
• Portions of Mathematical Methods and Algorithms for Signal Processing by Todd K. Moon and Wynn C. Stirling
• Lecture notes
• Lecture 1: Introduction to Information theory[download in PDF format]
• Lecture 2: Definitions and basic facts [download in PDF format]
• Lecture 3: Some more bounds [download in PDF format]
• Lecture BSS: Application of information theory to blind source separation [download in PDF format]
• Lecture 4: The asymptotic equipartition property [download in PDF format]
• Lecture 5: Entropy rates [download in PDF format]
• Lecture 6: Data compression [download in PDF format]
• Lecture AC: Arithmetic coding [download in PDF format]
• Lecture 7: Channel Capacity [download in PDF format]
• Lecture 8: More on Channel Capacity [download in PDF format]
• Lecture 9: Differential Entropy [download in PDF format]
• Lecture 10: The Gaussian channel [download in PDF format]
• Lecture queue: Bits and Queues [download in PDF format]
• Lecture 13: Maximimum entropy estimation[download in PDF format]
• Homework sets
• Homework [download in PDF format]
• Programming assignments
• Programming assignment 1: Blind source separation [download in PDF format]
• Programming assignment 2: Lossless data compression [download in PDF format]
• Test data for Data Compression
• Question:  E-mail a  question to the instructor
• News:  Read answers to  E-mailed questions