hw3 - X n , i.e., X n = n-1 X i =1 b i X n-i . Let F be the...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
EE 376B/Stat 376B Handout #8 Information Theory Thursday, April 20, 2006 Prof. T. Cover Due Thursday, April 27, 2006 Homework Set #3 1. Maximum entropy discrete processes. (a) Find the maximum entropy rate binary stochastic process { X i } i = -∞ , X i ∈ { 0 , 1 } , satisfying Pr { X i = X i +1 } = 1 3 , for all i . (b) What is the resulting entropy rate? 2. Maximum entropy of sums. Let Y = X 1 + X 2 . Find the maximum entropy (over all distributions on X 1 and X 2 ) of Y under the constraint EX 2 1 = P 1 , EX 2 2 = P 2 , (a) if X 1 and X 2 are independent. (b) if X 1 and X 2 are allowed to be dependent. 3. Estimation. Here is the estimation counterpart to Fano’s inequality. Let X be a random variable with differential entropy h ( X ). Let ˆ X be an estimate of X , and let E ( X - ˆ X ) 2 be the expected prediction error. (a) Show E ( X - ˆ X ) 2 1 2 πe e 2 h ( X ) . (b) Given side information Y and estimator ˆ X ( Y ), show E ( X - ˆ X ( Y )) 2 1 2 πe e 2 h ( X | Y ) . 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
4. Mean squared error. Let { X i } n i =1 satisfy EX i X i + k = R k , k = 0 , 1 ,...,p . Consider linear estimators for
Background image of page 2
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: X n , i.e., X n = n-1 X i =1 b i X n-i . Let F be the set all densities f ( x n ) satisfying R ,R 1 ,...,R p . Assume n > p . Find max f ( x n ) min b E ( X n- X n ) 2 . This identies the process which is hardest to estimate from the past. 5. Hadamard. Let K be a 2 n 2 n nonnegative denite symmetric matrix. Show det( K ) n Y i =1 det( K (2 i-1 , 2 i )) , where K ( i,j ) denotes the 2 2 submatrix K ii K ij K ji K jj . 6. Maximum entropy. (a) What is the parametric form maximum entropy density f ( x ) satisfying the two conditions EX 8 = a EX 16 = b. Dont solve for the s. (b) What is the maximum entropy density satisfying the condition E ( X 8 + X 16 ) = a + b ? 2...
View Full Document

Page1 / 2

hw3 - X n , i.e., X n = n-1 X i =1 b i X n-i . Let F be the...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online