This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: UCSB Fall 2009 ECE 235: Problem Set 4 Addendum (and brief notes on the Chernoff bound) Assigned: Thursday, October 29 Due: Thursday November 5 (by noon, in course homework box) Reading: Hajek, Chapter 2; Lecture notes Topics: LLN and Chernoff bound Reminder: Midterm is in class, Tue November 10. Summary of Chernoff bound: Let X 1 ,X 2 ,... be i.i.d. with mean E [ X i ] ≡ m , and let Λ( θ ) = log E bracketleftbig e θX 1 bracketrightbig denote the log moment generating function. The Chernoff bound for a single random variable is P [ X 1 > a ] ≤ e − ( aθ − Λ( θ )) , θ > (1) We can now optimize this over θ > 0 to get the best bound. This bound is only useful for tail probabilities of being larger than the mean. For a < m , the optimal value of the bound is one, a trivial answer. Similarly, P [ X 1 < a ] ≤ e − ( aθ − Λ( θ )) , θ < (2) We can now optimize this over θ < 0 to get the best bound. This bound is only useful for tail probabilities of being smaller than the mean. For a > m , the optimal value of the bound is one, a trivial answer. When we are in the non-trivial regime (i.e., for tail probabilities of being away from the mean), the optimized Chernoff bound for a single random variable is given in both cases by the same expression: P [ X 1 > a ] ≤ e − ℓ ( a ) , a > m P [ X 1 < a ] ≤ e − ℓ ( a ) , a < m (3) where ℓ ( a ) = max θ aθ − Λ( θ ) As we showed in class, the maximum above is attained for θ > 0 (and is positive) if a > m , and it is attained for θ < 0 (and is positive) if a < m , so that (3) is consistent with (1) and (2)....
View Full Document
This note was uploaded on 12/29/2011 for the course ECE 253 taught by Professor Brewer,f during the Fall '08 term at UCSB.
- Fall '08