# hw1 - minimum 3 Inequality Show ln x ≥ 1-1 x for x> 4...

This preview shows page 1. Sign up to view the full content.

EE 376A Information Theory Prof. T. Weissman Thursday, January 14, 2010 Homework Set #1 (Due: Thursday, January 21, 2010) 1. Coin flips. A fair coin is flipped until the first head occurs. Let X denote the number of flips required. (a) Find the entropy H ( X ) in bits. The following expressions may be useful: summationdisplay n =0 r n = 1 1 - r , summationdisplay n =0 nr n = r (1 - r ) 2 . (b) A random variable X is drawn according to this distribution. Find an “efficient” sequence of yes-no questions of the form, “Is X contained in the set S ?” Compare H ( X ) to the expected number of questions required to determine X . 2. Minimum entropy. What is the minimum value of H ( p 1 ,...,p n ) = H ( p ) as p ranges over the set of n -dimensional probability vectors? Find all p ’s which achieve this
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: minimum. 3. Inequality. Show ln x ≥ 1-1 x for x > . 4. InFnite entropy. This problem shows that the entropy of a discrete random variable can be in±nite. Let A = ∑ ∞ n =2 ( n log 2 n )-1 . (It is easy to show that A is ±nite by bounding the in±nite sum by the integral of ( x log 2 x )-1 .) Show that the integer-valued random variable X de±ned by Pr( X = n ) = ( An log 2 n )-1 for n = 2 , 3 , ... , has H ( X ) = + ∞ . 5. Markov’s inequality ±or probabilities. Let p ( x ) be a probability mass function. Prove, for all d ≥ 0, Pr { p ( X ) ≤ d } log p 1 d P ≤ H ( X ) . (1) 1...
View Full Document

• Fall '09
• Probability theory, Probability mass function, discrete random variable, Prof. T. Weissman, n-dimensional probability vectors

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern