McMillans - Proof of McMillans Theorem One Direction Ohad...

This preview shows pages 1–2. Sign up to view the full content.

Proof of McMillan’s Theorem: One Direction Ohad Shamir November 16, 2008 The following is a proof of one direction in McMillan’s Theorem - we saw the other direction in class. Before we begin, a reminder about notation: we assume there is a source (’makor’) X , which is simply a random variable which takes value in some set X . We use a small x to denote values in X . C is our coding function: for any given source value x , C ( x ) is the codeword which encodes x . Each codeword is a string of symbols from an alphabet D (for instance, D = { 0 , 1 } if we code in bits). ( x ) is the length of the codeword C ( x ). Example: Suppose X takes values in the letters of the english alphabet (namely X = { ‘a’ , ‘b’ , ‘c’ ,..., ‘z’ } ). Suppose we use ASCII to encode the letters as bits: each letter is represented as a string of 7 bits. For instance, C (‘e’) = 01100101, (‘e’) = 7, and (‘ef’) = 14. Theorem 1.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 3

McMillans - Proof of McMillans Theorem One Direction Ohad...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online