# hw4-2011 - inequalities. (1/3; 1/3; 1/4; 1/12). • H (X, Y...

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: inequalities. (1/3; 1/3; 1/4; 1/12). • H (X, Y |Z ) ≥ H ( code. Construct a Huﬀmann X |Z ), Show that there are two diﬀerent optimal codes with codeword lengths (1; 2; 3, 3) and (2; 2; 2; 2). Z ) ≥ I (X ; that there exists an optimal code such that some of its codewords are longer than • I ((X, Y ); Conclude Z ), those of the corresponding Shannon-Fano code. • H (X, Y, Z ) − H (X, Y ) ≤ H (X, Z ) − H (X ). Problem 17 A sequence of six symbols are independently drawn according to the distribution of the random ECE 563 ProblemX . The sequence is encoded symbol-by-symbol using an optimal (Huﬀmann) code. The resulted an arbitrary random variable taking its values from the set variable 23 (Card shuffling) Let X be IUC, Fall 2011 U {binary . string is 10110000101. We permutationthe the numbers 1, 2,has. ,ﬁve i.e., T (1), T (2)we. .only know a 1, 2, . . , 52}. Let T be a random know that of source alphabet . . 52, elements, but , . , T (52) is HW #4 random re-ordering ofis one of {1, 2, . . .distributions {0, 4; 0, 3; 0T2; 0,independent of{0, 3;Show 0, 2; 0, 2; 0, 05}. that the distribution the set the two , 52}. We assume that , is 05; 0, 05} and X . 0, 25; that Determine the distribution of X . Issued: October 4th, 2011 H (T (X )) ≥ 11th, Due: October H (X ). 2011 Problem 18 We are asked to determine an object by asking yes-no questions. The object is drawn randomly − Problem 24 set according 1 , X2 , . . . bedistribution. Playing optimally, we need 38.5 questions on the average6 . from a ﬁnite Let X = Xto a certain a binary memoryless stationary source with P{X1 = 1} = 10 Determine aobject. At least how many elements does theexpected codeword length is smaller than 1/10. Problemthe variable-length code of X whose per letter ﬁnite set have? to ﬁnd 1: Problem 25 (Run length coding) Let Huffmann be binary random variables. Let R = (R1 ,binary.) Problem 19 (Shortest codeword of X1 , . . . , Xn codes) Suppose that we have an optimal R2 , . . denote coderun lengths of the symbols. in X,1 , . . . , Xn . > p >is,. for example, thethat lengths of the sequence preﬁx the for the distribution (p1 , . . , pn ) where p1 That . . , pn > 0. Show run 2 1110010001111 is R = (3, 2, 1, 3, 4). Determine the relation between H (X1 , . . . , Xn ), H (R) and H (R, Xn )? • If p1 > 2/5 then the corresponding codeword has length 1. Problem 26 (Entropy of a Markov chain) Let X = X1 , X2 , . . . be a binary stationary Markov chain Problem p < 1/3 then the corresponding codeword has length at least 2. • If 2: with state1transition probabilities Problem 20 (Basketball play-offs) The play-oﬀs of NBA are played between team A and team B in1a+ p 1−p P{X2 = 0|X1 = 0} = p, P{X2 = 1|X1 = 0} = 1−p, P{X2 = 0|X1 = 1} = , P{X2 = 1|X1 = 1} = . 2 seven-game series that terminates as soon as one of the teams wins four games. Let the random variable X 2 represent P{X1 = 0} What is the games (possible values Determinethe outcome .of the series of entropy of the source?of X are AAAA or BABABAB or AAABBBB ). Let Y denote the number of games played. Assuming that the two teams are equally strong, determine the Problem H (X ), H (Y ), H (Y |X ) and H (X |Y ). values of 27 (The second law of thermodynamics) Let X = X1 , X2 , . . . be a stationary Markov chain. Show that H (Xn |X1 ) is monotone increasing (in spite of the fact that H (Xn ) does not change with n by stationarity). Problem 3: Textbook, page 51, 2.36 - Symmetric Relative Entropy Problem 28 (Properties of the binary entropy function) Deﬁne the function h(x) = −x log x − Problem 4: Texbook, page 48, 2.21 - Markov’s inequality for probabilities (1 − x) log(1 − x), for x ∈ (0, 1), and h(0) = h(1) = 0. Show that h satisﬁes the following properties: Problem 5: Texbook, page 92, 4.10 - Pairwise independence • symmetric around 1/2; • continuous in every point of [0, 1]; 4 • strictly monotone increasing in [0, 1/2]; • strictly concave. 5 ...
View Full Document

## This note was uploaded on 10/24/2011 for the course ELECTRICAL ECE 571 taught by Professor Kelly during the Spring '11 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online