This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: 6.262: Discrete Stochastic Processes Lecture 24  5/5/10 • Sequential decision theory (when to de cide?) – Cherno ff bound and tilted rv’s – Wald’s identity – Bound on threshold crossing probability – Sequential decisions • Martingales – Isolation of past – Submartingales and supermartingales – Stopped martingales and Wald identity 1 Cherno ff bound review Assume { X n ; n ≥ 1 } is IID sequence with X < and E e Xr < ∞ for r ≥ . Let γ ( r ) = ln( E e Xr ) . Then for a > X Pr { S n ≥ na } ≤ exp { n [ γ ( r ) ra ] } for r > where S n = X 1 + · · · + X n . = = = = = = slope = X r o γ ( r o ) r * slope = γ ( r o ) = a γ ( r o ) r o a r Optimizing r = r o 2 Tilted probabilities Assume X is discrete with PMF p X ( x ) . For any given r , define the tilted PMF q X,r ( x ) = p X ( x ) exp[ rx γ ( r )] , where the tilted X is the same mapping from sample points to real values as X , but the prob ability is di ff erent, tilted to favor large values for r > . = = = = =2 1 1 2 p X ( x )2 1 1 2 = = = = = q X,r ( x ) x q X,r ( x ) = ∑ x p X ( x ) e xr E e Xr = 1 3 Assume X n ; n ≥ 1 are IID and also define them to be IID in this new tilted measure. Then q X n , r ( x 1 , . . . , x n ) = p X n ( x 1 , . . . , x n ) exp[ n i =1 rx i n γ ( r )] Sum over all x 1 , . . . , x n that have a common sum s n . Then q S n ,r ( s n ) = p S n ( s n ) exp[ rs n n γ ( r ) Thus a sum of tilted IID rv’s are related to the untilted sum in the same way as a single tilted rv is related to the untilted rv. This is a key to large deviation analysis. 4 The mean of X , tilted by r , is E r [ X ] = x x p X ( x ) exp[ rx γ ( r ) = ∑ x p X ( x ) exp( rx ) E [exp( rX )] = γ ( r ) Now consider Pr { S n ≥ na } again, using tilted probabilities with r chosen so that γ ( r ) = a . Pr { S n ≥ na } = s n ≥ na p S n ( s n ) = s n ≥ na q S n ,r ( s n ) exp[ rs n + n γ ( r )] ∼ exp[ rna + n γ ( r )] since S n with the tilted measure obeys the CLT and is centered on na . Thus the Cherno ff bound is exponentially tight. 5 Thm: (Wald) Let γ ( r ) = ln( E [exp( rX )]) exist over ( r < < r + ) . Let J be trial at which S n first crosses α > or β < . Then E [exp( rS J J γ ( r )] = 1 for r ∈ ( r , r + ) Pf: q X n ,r ( x 1 , . . . , x n ) = p X n ( x 1 , . . . , x n ) exp[ rs n n γ ( r )] q J,r ( n ) = x n : { J = n } p X n ( x n ) exp[ rs n n γ ( r )] = E [exp( rS n n γ ( r ))  J = n ] p J ( n ) Summing both sides over n completes the proof....
View
Full Document
 Spring '08
 Moon,J
 Derivative, Zn, Wald, Martingale, max Zi

Click to edit the document details