HW1_ES250 - Harvard SEAS ES250 Information Theory Homework...

Info icon This preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
Harvard SEAS ES250 – Information Theory Homework 1 (Due Date: Oct. 2 2007) 1. Let p ( x, y ) be given by X ´ Y 0 1 0 1/3 1/3 1 0 1/3 Evaluate the following expressions: (a) H ( X ), H ( Y ) (b) H ( X | Y ), H ( Y | X ) (c) H ( X, Y ) (d) H ( Y ) - H ( Y | X ) (e) I ( X ; Y ) (f) Draw a Venn diagram for the quantities in (a) through (e) 2. Entropy of functions of a random variable (a) Let X be a discrete random variable. Show that the entropy of a function of X is less than or equal to the entropy of X by justifying the following steps: H ( X, g ( X )) ( a ) = H ( X ) + H ( g ( X ) | X ) ( b ) = H ( X ) H ( X, g ( X )) ( c ) = H ( g ( X )) + H ( X | g ( X )) ( d ) H ( g ( X )) Thus H ( g ( X )) H ( X ). (b) Let Y = X 7 , where X is a random variable taking in positive and negative integer values. What is the relationship of H ( X ) and H ( Y )? What if Y = cos( πX/ 3) ? 3. One wishes to identify a random object X p ( x ). A question Q r ( q ) is asked at random accord- ing to r ( q ). This results in a deterministic answer A = A ( x, q ) ∈ { a 1 , a 2 , · · · } . Suppose the object X and the question Q are independent. Then I ( X ; Q, A ) is the uncertainty in X removed by the question-answer ( Q, A ).
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern