# hw4_soln_fall2011 - ECE 563 Fall 2011 Homework 4 Solution...

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ECE 563, Fall 2011 Homework 4 Solution Question 1: Since the run‐lengths are a function of determines the entire sequence , … , , … , , . Hence . Any together with the run lengths ,…, , | 1 Question 2: First note that we assume the outcome of the games are independent. We list all possible values of X: 2 possible ’s with length 4, each with probability 4 1 2 8 possible ’s with length 5 2 2 20 possible ’s with length 6, each with probability 6 3 2 40 possible ’s with length 7, each with probability 5, each with probability From the above list we can determine probabilities of as: 4 1 2 2 1 8 5 1 2 8 1 4 6 20 5 16 7 1 2 1 2 40 5 16 Thus we can easily compute: 2 1 1 log 2 2 8 1 1 log 2 2 1 1 log 8 8 1 1 log 2 2 20 1 1 log 4 4 40 1 1 log 16 16 2 | | 1 1 log 2 2 5.8 1.92 3.9 Question 3: An example of a distribution with distinct and where the divergence is symmetric, try one distribution for two symbols with 0 and 1 1 and, conversely, the other with 0 1 and 1 , hence || log 1 1 log 1 And || 1 log 1 log 1 which is the same Question 4: Markov inequality applied to entropy: log 1 : log 1 : log 1 log 1 ...
View Full Document

{[ snackBarMessage ]}

Ask a homework question - tutors are online