HW1_ES250

# HW1_ES250 - Harvard SEAS ES250 Information Theory Homework...

• Notes
• 3

This preview shows pages 1–2. Sign up to view the full content.

Harvard SEAS ES250 – Information Theory Homework 1 (Due Date: Oct. 2 2007) 1. Let p ( x, y ) be given by X ´ Y 0 1 0 1/3 1/3 1 0 1/3 Evaluate the following expressions: (a) H ( X ), H ( Y ) (b) H ( X | Y ), H ( Y | X ) (c) H ( X, Y ) (d) H ( Y ) - H ( Y | X ) (e) I ( X ; Y ) (f) Draw a Venn diagram for the quantities in (a) through (e) 2. Entropy of functions of a random variable (a) Let X be a discrete random variable. Show that the entropy of a function of X is less than or equal to the entropy of X by justifying the following steps: H ( X, g ( X )) ( a ) = H ( X ) + H ( g ( X ) | X ) ( b ) = H ( X ) H ( X, g ( X )) ( c ) = H ( g ( X )) + H ( X | g ( X )) ( d ) H ( g ( X )) Thus H ( g ( X )) H ( X ). (b) Let Y = X 7 , where X is a random variable taking in positive and negative integer values. What is the relationship of H ( X ) and H ( Y )? What if Y = cos( πX/ 3) ? 3. One wishes to identify a random object X p ( x ). A question Q r ( q ) is asked at random accord- ing to r ( q ). This results in a deterministic answer A = A ( x, q ) ∈ { a 1 , a 2 , · · · } . Suppose the object X and the question Q are independent. Then I ( X ; Q, A ) is the uncertainty in X removed by the question-answer ( Q, A ).

This preview has intentionally blurred sections. Sign up to view the full version.

This is the end of the preview. Sign up to access the rest of the document.
• Fall '09
• Probability theory, Harvard SEAS

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern