Course Syllabus

Course Information：

Year-Semester Course Number Course Name Phase Credits Hours Required/Elective Instructor Class Student NO. Withdraw Note

Spring Semester
Year 2021 284338 Information Theory 1 3.0 3 ★ Chao-Cheng Wu GEE
IEECSG 12 1 ◎

Year-Semester	Course Number	Course Name	Phase	Credits	Hours	Required/Elective	Instructor	Class	Student NO.	Withdraw	Note
Spring Semester Year 2021	284338	Information Theory	1	3.0	3	★	Chao-Cheng Wu	GEE IEECSG	12	1	◎

Course Plan：

Instructor Chao-Cheng Wu

Email ccwu@ntut.edu.tw

Last Updated 2020-12-14 12:24:22

Course Objective This course aims to study basic and advanced topics in information theory. The course focuses on the concept of information theory, including entropy, source coding, channel capacity, rate distortion theory, and differential entropy, ...etc. The calculation of channel capacity and rate distortion will be used as computer projects for real implementation. Advanced topics will further cover Gaussian channel and some coding theory.

Course Schedule Week 1. Introduction Week 2. Review of Random Variables Week 3. Entropy/Relative Entropy/Mutual Information Week 4. Jensen’s and Fano's Inequality Week 5. Data compression (I) Week 6. Leave for Public Affairs Week 7. Data compression (II) Week 8. Data compression (III) Week 9. Midterm Week 10. Channel capacity (I) Week 11. Channel capacity (II) Week 12. Channel capacity (III) Week 13. Rate Distortion Theory (I) Week 14. Rate Distortion Theory (II) Week 15. Rate Distortion Theory (III) Week 16. Asymptotic equipartition property Week 17. Differential entropy Week 18. Final term

Evaluation and grading policy Homework 20% Midterm 20% Computer Project 60%

Materials All right reserved. Do not reproduce without permission.

Foreign-language textbooks:Yes
Required text: 1. Elements of Information Theory, 2nd Edition, Thomas M. Cover and Joy A. Thomas, Wiley, 2006. Optional text: 1. The Mathematical Theory of Information, Jan Kahre, The Springer, 2002. 2. Information Theory, Inteference, and Learning Algorithms, David J. C. Mackay, Cambridge, 2003. 3. Information Theory: Coding Theorems for Discrete Memoryless Systems, 2md Edition, Imre Csiszar and Janos Korner, Cambridge, 2011

The access to curricular consultation

Note
　

Instructor	Chao-Cheng Wu
Email	ccwu@ntut.edu.tw
Last Updated	2020-12-14 12:24:22
Course Objective	This course aims to study basic and advanced topics in information theory. The course focuses on the concept of information theory, including entropy, source coding, channel capacity, rate distortion theory, and differential entropy, ...etc. The calculation of channel capacity and rate distortion will be used as computer projects for real implementation. Advanced topics will further cover Gaussian channel and some coding theory.
Course Schedule	Week 1. Introduction Week 2. Review of Random Variables Week 3. Entropy/Relative Entropy/Mutual Information Week 4. Jensen’s and Fano's Inequality Week 5. Data compression (I) Week 6. Leave for Public Affairs Week 7. Data compression (II) Week 8. Data compression (III) Week 9. Midterm Week 10. Channel capacity (I) Week 11. Channel capacity (II) Week 12. Channel capacity (III) Week 13. Rate Distortion Theory (I) Week 14. Rate Distortion Theory (II) Week 15. Rate Distortion Theory (III) Week 16. Asymptotic equipartition property Week 17. Differential entropy Week 18. Final term
Evaluation and grading policy	Homework 20% Midterm 20% Computer Project 60%
Materials	All right reserved. Do not reproduce without permission. Foreign-language textbooks:Yes Required text: 1. Elements of Information Theory, 2nd Edition, Thomas M. Cover and Joy A. Thomas, Wiley, 2006. Optional text: 1. The Mathematical Theory of Information, Jan Kahre, The Springer, 2002. 2. Information Theory, Inteference, and Learning Algorithms, David J. C. Mackay, Cambridge, 2003. 3. Information Theory: Coding Theorems for Discrete Memoryless Systems, 2md Edition, Imre Csiszar and Janos Korner, Cambridge, 2011
The access to curricular consultation
Note