CS 4953, Spring 2004:
Coding and Information Theory
MWF 11:00-11:50 am

Course Topics (Preliminary List)

  1. Introduction.
    1. Intuitive ideas of information.
    2. Intuitive ideas of coding.
    3. Three types of coding.
      1. Source coding for compression.
        Example: Huffman code.
      2. Channel coding for error detection/correction.
        Example: Hamming code.
      3. Secrecy coding with cryptography.
        Example: One-time pad, Advanced Encryption Standard (AES).
    4. Overview of results and applications.
      1. Quick overview of Shannon's theorems.
      2. Sample applications.
    5. Snake oil salesmen: information theory charletons.
  2. Shannon's information theory
    1. Entropy and Shannon's Noiseless Coding Theorem.
    2. The channel, mutual information, and channel capacity.
    3. Shannon's noisy coding theorem.
  3. Advanced lossless compression techniques.
    1. Arithmetic coding.
      1. Applications: JBIG2.
    2. Predictive coding.
      1. Applications: LOCO-I and JPEG-LS.
    3. Dictionary techniques.
      1. LZ77, LZ78, LZW.
      2. Applications: GIF.
  4. Lossy compression techniques.
    1. Quantization.
    2. Transform coding.
    3. Wavelet transforms.
    4. JPEG compression.
    5. MPEG compression.
  5. Miscellaneous topics.
    1. Algorithmic information theory.
    2. Quantum computing.

Revision date: 2003-10-25. (Please use ISO 8601, the International Standard.)