This preview shows page 1. Sign up to view the full content.
Unformatted text preview: ECE 563, Fall 2011
Homework 4 Solution Question 1: Since the run‐lengths are a function of determines the entire sequence , … , , … , , . Hence . Any together with the run lengths ,…, ,
 1 Question 2: First note that we assume the outcome of the games are independent. We list all possible values of X: 2 possible ’s with length 4, each with probability 4
1 2 8 possible ’s with length 5
2 2 20 possible ’s with length 6, each with probability 6
3 2 40 possible ’s with length 7, each with probability 5, each with probability From the above list we can determine probabilities of as: 4 1
2 2 1 8 5 1
2 8 1 4 6 20 5 16 7 1
2
1
2 40 5 16 Thus we can easily compute: 2 1
1
log
2
2 8 1
1
log
2
2 1
1
log
8
8 1
1
log
2
2 20 1
1
log
4
4 40 1
1
log
16
16 2   1
1
log
2
2 5.8 1.92 3.9 Question 3: An example of a distribution with distinct and where the divergence is symmetric, try one distribution for two symbols with 0 and 1 1 and, conversely, the other with 0 1 and 1 , hence  log 1 1 log 1 And  1 log 1 log 1 which is the same Question 4: Markov inequality applied to entropy: log 1
: log 1
: log 1 log 1 ...
View
Full
Document
This note was uploaded on 10/24/2011 for the course ELECTRICAL ECE 571 taught by Professor Kelly during the Spring '11 term at University of Illinois, Urbana Champaign.
 Spring '11
 Kelly

Click to edit the document details