IT3_2005-6_paper_v2 - INFORMATION THEORY 3 MATH 34600 FINAL...

Info icon This preview shows pages 1–3. Sign up to view the full content.

INFORMATION THEORY 3 MATH 34600 FINAL EXAM PAPER 2005/06 This paper contains three questions; each is worth 25 marks. A candidate’s two best answers will be used for assessment. Time given is 2(?) hours. Date of the exam: dd/mm/2005 ... am to ... pm. A calculator of the approved type is allowed. 1
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

1. Consider the source X with alphabet A = { α, β, γ, δ, ε, ϕ } and probability distribution P X : x α β γ δ ε ϕ P X ( x ) .3 .2 .15 .125 .125 .1 (a) State Kraft’s condition for the existence of a prefix-free binary code for a general alphabet X with supposed codeword lengths x (inte- gers), x ∈ X . [4 marks] (b) Construct the Shannon-Fano code for the source X , writing down the required code word lengths first. [6 marks] (c) Calculate the expected length of your code, and the entropy of the source X . [4 marks] (d) Explain the significance of the entropy in the context of Shannon’s source coding theorem. Be careful to define any notation and termi- nology you introduce. [3 marks] (e) Still referring to Shannon’s source coding theorem: Explain in words the difference of performance of your code constructed in (b) and the asymptotically optimal source code. [3 marks] (f) Can you improve the expected length of the code you constructed in part (b)? If so, show how to do this. (Hint: is the Kraft inequality
Image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern