Information Theory, Inference, and Learning Algorithms The Laws of Cryptography with Java Code A Mathematical Theory of Communication

CS 4953  Coding and  Information Theory  Overview and Syllabus,     Spring 2004  Lecture:     MWF 11:00-11:50 am  Final Exam:    Monday, 3 May 2004,    1:30-4:15pm

Instructor: Neal R. Wagner

Description:

• (Formal)
  

  CS 4383 Coding and Information Theory. (3-0) 3 hours credit. Prerequisites: CS 4363 Cryptography, or concurrent enrollment in CS 4363, or consent of instructor. An introductory course covering the basic information theory concepts of entropy, channel capacity, and Shannon's theorems. Communication codes for error control (detection and correction), and for compression (lossless and lossy) are considered. Numerous applications are studied.

  (CS 4383 is the suggested new number if the course becomes a part of the undergraduate curriculum. The current course is CS 4953: Special Studies in Computer Science.)

• (Informal)
  

  Coding and Information Theory. The term "information" is often misunderstood, but in the theory due to Claude Shannon, it refers to the amount of "surprise" upon the receipt of a message. (No surprise means no information.) The theory involves messages, their transmission, and the encoding of messages for compression, for error handling, and for secrecy. To quote from Shannon: The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Shannon (1948) Shannon did an amazing job of presenting a fully-developed theory in the late 1940s; since then an incredible number of refinements to the theory and applications have emerged. Everything in the modern computer era depends on this material as a foundation.

Textbooks and Reference Books:

• The Sacred Text of Information Theory is Shannon's original 1948 publication: click on the third image above.

• The two books with images above are recommended texts. Each is available online in PDF format as a local copy. Click on each image for more information about the book. You are not allowed to print a copy of any of these books, especially not at UTSA. (The copyright holders do not allow printing any of the books. The printed copy will not do you much good and will waste a lot of paper. You may make electronic copies for your own use.)

• Here is a list of current technical books in coding and information theory: list of books.

Course Topics: