Introduction to Information Theory

Lecture: Tuesday and Thursday 12-1:45 p.m. Porter Acad, 249.

Instructor: Hamid R. Sadjadpour, BE2-245B, Email:, 
Phone: (831)459-1483.

Instructor Office Hours: 
Thursday 9-11 a.m. Also by appointment. 

Thomas Cover, Joy Thomas, Elements of Information Theory, 2nd edition, Wiley.
Reymond Yeung, A first course in Information Theory, Springer publisher.

Homeworks (10%):
Problems will be given at the end of each chapter. Late homework will not be accepted. 

Exams (45% each):

    Midterm Exam: Thursday, 2/9/2012.
    Final Exam: As scheduled by registration office. 

No make-up exam under any circumstances. Exams most likely are closed book. You can bring 1 page note. 

Tentative Schedule:
Week 2: Chapter 2, Entropy, Relative Entropy and Mutual Information.
Week 3: Chapters 2.
Week 4: Chapters 3, Asymptotic Equipartition Property. 
Week 5: Chapter 4Entropy Rates of A Stochastic Process.
Week 6: Chapter 4. (Midterm 1 contains the first three chapters)
Week 7: Chapter 5, Data Compression. 
Week 8: Chapter 5.
Week 9: Chapter 7, Channel Capacity.
Week 10: Chapter 7 and Chapter 8 if time permits.

Final exam will cover the last part of the course. 
We may not be able to teach all sections of these chapters. 
The exams will be based on class materials and homeworks.


The first week of the class, we will cover a new topic related to information theory of large scale networks.

  • Chapter 2 slides.         Slides
  • HW # 1: Chapter 2, problems 2, 4, 10 (a), 12, 15, 26, and 32. Due date: 1/26/12.


  • Chapter 3 slides.                Slides
  • HW # 2:  Chapter 3, problems 1, 2, 6, and 11(a & b). Due date: 1/31/12.


  • Chapter 4 slides.                Slides
  • HW # 3: Chapter 4, problems 1, 6, 7, 9, 11(a,b,c). Due date: 2/7/12.


  • Midterm Exam, Chapters 2, 3, and 4 during regular class. Date: Thusday, 2/9/2012.
  • Chapter 7 slides.         Slides
  • HW # 4: Chapter 7, problems 3, 4, 7, 11, 13,  20. Due Date: TBD.


  • Chapter 5 slides.                    Slides
  • HW # 5: Chapter 5, problems 1, 2, 4, 6, 12. Due date: 3/1/12.                    Solution

Student Accommodation Requests

If you qualify for classroom accomodations because of a disability, please submit your Accommodation Authorization Letter from the Disability Resource Center (DRC) to me during my office hours or by appointment, preferably within  the first four weeks of the quarter. Contact DRC by phone at 831-4592089 or by email at for more information.

Academic Dishonesty

Any confirmed academic dishonesty including but not limited to copying homeworks or cheating on exams, will result in a no-pass or failing grade. You are encouraged to read the campus policies regarding academic integrity. Examples of cheating include (but are not limited to):

  • Sharing results or other information during an examination.
  • Working on an exam before or after the official time allowed.
  • Submitting homework that is not your own work.
  • Reading another student's homework solution before it is due.
  • Allowing someone else to read your homework solution before the assignment is due.
  • If there is any question as to whether a given action might be construed as cheating, see me before you engage in any such action.

Instructors and Assistants