MATH 187 entropy_quiz_solution

MATH 187 entropy_quiz_solution - H [ X j ( Y;Z ) = (0 ;...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
Math 187 Prof. Garsia prepared by Alex Brik 05-04-2009 Typical Quiz on Entropy Solution. 1. H [ X ] = P ( X = 0) log 2 1 P ( X = 0) ± + P ( X = 1) log 2 1 P ( X = 1) ± + P ( X = 1) log 2 1 P ( X = 1) ± + + P ( X = 3) log 2 1 P ( X = 3) ± = = 3 8 log 2 8 3 ± + 1 4 log 2 (4) + 1 4 log 2 (4) + 1 8 log 2 (8) ± 1 : 906 H [ X ] ± 1 : 906 : 2. 1.585 bits per spin, 158500 bits to store 100000 spins. H [ Z ] = P ( Z = 1) log 2 1 P ( Z = 1) ± + P ( Z = 2) log 2 1 P ( Z = 2) ± + P ( Z = 3) log 2 1 P ( Z = 3) ± = = 1 3 log 2 3 + 1 3 log 2 3 + 1 3 log 2 3 ± 1 : 585 : H [ Z ] ± 1 : 585 : 3. H [ Z j X = 0] = P ( Z = 1 j X = 0) log 2 1 P ( Z = 1 j X = 0) ± + P ( Z = 2 j X = 0) log 2 1 P ( Z = 2 j X = 0) ± = + P ( Z = 3 j X = 0) log 2 1 P ( Z = 3 j X = 0) ± : P ( Z = 1 j X = 0) = P ( Z X = 0) P ( X = 0) = 2 = 8 3 = 8 = 2 = 3 P ( Z = 2 j X = 0) = 0 P ( Z = 3 j X = 0) = 1 3 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
H [ Z j X = 0] & 0 : 9183 4. H [ X j Y;Z ] = 0 H [ X j Y;Z ] = P [( Y;Z ) = (0 ; 1)] H [ X j ( Y;Z ) = (0 ; 1)]+ P [( Y;Z ) = (0 ; 2)] H [ X j ( Y;Z ) = (0 ; 2)] + + P [( Y;Z ) = (0 ; 3)] H [ X j ( Y;Z ) = (0 ; 3)] + P [( Y;Z ) = ( ± 1 ; 1)] H [ X j ( Y;Z ) = ( ± 1 ; 1)] + + P [( Y;Z ) = ( ± 1 ; 2)] H [ X j ( Y;Z ) = ( ± 1 ; 2)]+ P [( Y;Z ) = ( ± 1 ; 3)] H [ X j ( Y;Z ) = ( ± 1 ; 3)] + + P [( Y;Z ) = (1 ; 1)] H [ X j ( Y;Z ) = (1 ; 1)] + P [( Y;Z ) = (1 ; 2)] H [ X j ( Y;Z ) = (1 ; 2)] + + P [( Y;Z ) = (1 ; 3)] H [ X j ( Y;Z ) = (1 ; 3)] : Now H [ X j ( Y;Z ) = (0 ; 1))] = P [ X = 1 j ( Y;Z ) = (0 ; 1)] ² log 2 1 P [ X = 1 j ( Y;Z ) = (0 ; 1)] ± = 0 : H [ X j ( Y;Z ) = (0 ; 2))] = 0
Background image of page 2
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: H [ X j ( Y;Z ) = (0 ; 3))] = 0 H [ X j ( Y;Z ) = ( 1 ; 1))] = 0 H [ X j ( Y;Z ) = ( 1 ; 2))] = 0 H [ X j ( Y;Z ) = ( 1 ; 3))] = 0 H [ X j ( Y;Z ) = (1 ; 1))] = 0 H [ X j ( Y;Z ) = (1 ; 2))] = 0 H [ X j Y;Z ] = 0 : H [ X j ( Y;Z ) = (1 ; 3))] = 0 5. H [ Z j Y ] & 1 : 5850 H [ Z j Y ] = P ( Y = 1) H [ Z j Y = 1]+ P ( Y = 0) H [ Z j Y = 0]+ P ( Y = 1) H [ Z j Y = 1] : Now H [ Z j Y = 1] = 3 X k =1 P [ Z = k j Y = 1] log 2 & 1 P [ Z = k j Y = 1] = 3 X k =1 1 3 log 2 3 & 1 : 585 : H [ Z j Y = 1] & 1 : 585 H [ Z j Y = 0] & 1 : 585 So H [ Z j Y ] & 1 : 585 2...
View Full Document

This note was uploaded on 05/22/2010 for the course MATH 187 taught by Professor Staff during the Spring '08 term at UCSD.

Page1 / 2

MATH 187 entropy_quiz_solution - H [ X j ( Y;Z ) = (0 ;...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online