This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: UCSB Fall 2009 ECE 235: Problem Set 4 Addendum (and brief notes on the Chernoff bound) Assigned: Thursday, October 29 Due: Thursday November 5 (by noon, in course homework box) Reading: Hajek, Chapter 2; Lecture notes Topics: LLN and Chernoff bound Reminder: Midterm is in class, Tue November 10. Summary of Chernoff bound: Let X 1 ,X 2 ,... be i.i.d. with mean E [ X i ] ≡ m , and let Λ( θ ) = log E bracketleftbig e θX 1 bracketrightbig denote the log moment generating function. The Chernoff bound for a single random variable is P [ X 1 > a ] ≤ e − ( aθ − Λ( θ )) , θ > (1) We can now optimize this over θ > 0 to get the best bound. This bound is only useful for tail probabilities of being larger than the mean. For a < m , the optimal value of the bound is one, a trivial answer. Similarly, P [ X 1 < a ] ≤ e − ( aθ − Λ( θ )) , θ < (2) We can now optimize this over θ < 0 to get the best bound. This bound is only useful for tail probabilities of being smaller than the mean. For a > m , the optimal value of the bound is one, a trivial answer. When we are in the nontrivial regime (i.e., for tail probabilities of being away from the mean), the optimized Chernoff bound for a single random variable is given in both cases by the same expression: P [ X 1 > a ] ≤ e − ℓ ( a ) , a > m P [ X 1 < a ] ≤ e − ℓ ( a ) , a < m (3) where ℓ ( a ) = max θ aθ − Λ( θ ) As we showed in class, the maximum above is attained for θ > 0 (and is positive) if a > m , and it is attained for θ < 0 (and is positive) if a < m , so that (3) is consistent with (1) and (2)....
View
Full
Document
This note was uploaded on 12/29/2011 for the course ECE 253 taught by Professor Brewer,f during the Fall '08 term at UCSB.
 Fall '08
 Brewer,F

Click to edit the document details