hw4_soln_fall2011 - ECE 563, Fall 2011 Homework 4 Solution...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ECE 563, Fall 2011 Homework 4 Solution Question 1: Since the run‐lengths are a function of determines the entire sequence , … , , … , , . Hence . Any together with the run lengths ,…, , | 1 Question 2: First note that we assume the outcome of the games are independent. We list all possible values of X: 2 possible ’s with length 4, each with probability 4 1 2 8 possible ’s with length 5 2 2 20 possible ’s with length 6, each with probability 6 3 2 40 possible ’s with length 7, each with probability 5, each with probability From the above list we can determine probabilities of as: 4 1 2 2 1 8 5 1 2 8 1 4 6 20 5 16 7 1 2 1 2 40 5 16 Thus we can easily compute: 2 1 1 log 2 2 8 1 1 log 2 2 1 1 log 8 8 1 1 log 2 2 20 1 1 log 4 4 40 1 1 log 16 16 2 | | 1 1 log 2 2 5.8 1.92 3.9 Question 3: An example of a distribution with distinct and where the divergence is symmetric, try one distribution for two symbols with 0 and 1 1 and, conversely, the other with 0 1 and 1 , hence || log 1 1 log 1 And || 1 log 1 log 1 which is the same Question 4: Markov inequality applied to entropy: log 1 : log 1 : log 1 log 1 ...
View Full Document

This note was uploaded on 10/24/2011 for the course ELECTRICAL ECE 571 taught by Professor Kelly during the Spring '11 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online