Course Syllabus

Course Information：

Year-Semester Course Number Course Name Phase Credits Hours Required/Elective Instructor Class Student NO. Withdraw Note

Spring Semester
Year 2024 331861 Information Theory 1 3.0 3 ★ Chao-Cheng Wu GEE
IEECSG 10 0 電機所和電資外生所合開

Year-Semester	Course Number	Course Name	Phase	Credits	Hours	Required/Elective	Instructor	Class	Student NO.	Withdraw	Note
Spring Semester Year 2024	331861	Information Theory	1	3.0	3	★	Chao-Cheng Wu	GEE IEECSG	10	0	電機所和電資外生所合開

Course Plan：

Instructor Chao-Cheng Wu　Office Hours

Email ccwu@ntut.edu.tw

Last Updated 2024-05-22 11:05:09

Course Objective This course aims to study basic and advanced topics in information theory. The course focuses on the concept of information theory, including entropy, source coding, channel capacity, rate distortion theory, and differential entropy, ...etc. The calculation of channel capacity and rate distortion will be used as computer projects for real implementation. Advanced topics will further cover Gaussian channel and some coding theory.

Course Schedule Week 1(2/20). Introduction Week 2(2/27). Review of Random Variables Week 3(3/5). Entropy Week 4(3/12). Joint Entropy/Conditional Entropy Week 5(3/19). Relative Entropy/Mutual Information Week 6(3/26). Jensen’s and Fano's Inequality Week 7(4/2). Data compression (I) Week 8(4/9). Data compression (II) Week 9(4/16). Data compression (III) Week 10(4/23). Midterm Week 11(4/30). Channel capacity (I) Week 12(5/7). Channel capacity (II) Week 13(5/14). Channel capacity (III) Week 14(5/21). Rate Distortion Theory (I) Week 15(5/28). Rate Distortion Theory (II) Week 16(6/4). Rate Distortion Theory (III) Week 17(6/11). Differential entropy Week 18(6/18). Final term

Evaluation and grading policy Homework 20% Midterm 20% Computer Project 60%

Materials All right reserved. Do not reproduce without permission.

Foreign-language textbooks:Yes
Required text: 1. Elements of Information Theory, 2nd Edition, Thomas M. Cover and Joy A. Thomas, Wiley, 2006. Optional text: 1. The Mathematical Theory of Information, Jan Kahre, The Springer, 2002. 2. Information Theory, Inteference, and Learning Algorithms, David J. C. Mackay, Cambridge, 2003. 3. Information Theory: Coding Theorems for Discrete Memoryless Systems, 2md Edition, Imre Csiszar and Janos Korner, Cambridge, 2011

The access to curricular consultation Please make an appointment via Google Classroom or Email (ccwu@ntut.edu.tw)
The online courses for the 17th and 18th weeks will be conducted via Google Meet, and the meeting link will be posted on Google Classroom.

The course corresponds to the SDGs

Does the course incorporate AI

Note
　

Instructor	Chao-Cheng Wu　Office Hours
Email	ccwu@ntut.edu.tw
Last Updated	2024-05-22 11:05:09
Course Objective	This course aims to study basic and advanced topics in information theory. The course focuses on the concept of information theory, including entropy, source coding, channel capacity, rate distortion theory, and differential entropy, ...etc. The calculation of channel capacity and rate distortion will be used as computer projects for real implementation. Advanced topics will further cover Gaussian channel and some coding theory.
Course Schedule	Week 1(2/20). Introduction Week 2(2/27). Review of Random Variables Week 3(3/5). Entropy Week 4(3/12). Joint Entropy/Conditional Entropy Week 5(3/19). Relative Entropy/Mutual Information Week 6(3/26). Jensen’s and Fano's Inequality Week 7(4/2). Data compression (I) Week 8(4/9). Data compression (II) Week 9(4/16). Data compression (III) Week 10(4/23). Midterm Week 11(4/30). Channel capacity (I) Week 12(5/7). Channel capacity (II) Week 13(5/14). Channel capacity (III) Week 14(5/21). Rate Distortion Theory (I) Week 15(5/28). Rate Distortion Theory (II) Week 16(6/4). Rate Distortion Theory (III) Week 17(6/11). Differential entropy Week 18(6/18). Final term
Evaluation and grading policy	Homework 20% Midterm 20% Computer Project 60%
Materials	All right reserved. Do not reproduce without permission. Foreign-language textbooks:Yes Required text: 1. Elements of Information Theory, 2nd Edition, Thomas M. Cover and Joy A. Thomas, Wiley, 2006. Optional text: 1. The Mathematical Theory of Information, Jan Kahre, The Springer, 2002. 2. Information Theory, Inteference, and Learning Algorithms, David J. C. Mackay, Cambridge, 2003. 3. Information Theory: Coding Theorems for Discrete Memoryless Systems, 2md Edition, Imre Csiszar and Janos Korner, Cambridge, 2011
The access to curricular consultation	Please make an appointment via Google Classroom or Email (ccwu@ntut.edu.tw) The online courses for the 17th and 18th weeks will be conducted via Google Meet, and the meeting link will be posted on Google Classroom.
The course corresponds to the SDGs
Does the course incorporate AI
Note