6.262.Lec24

# 6.262.Lec24 - 6.262 Discrete Stochastic Processes Lecture...

This preview shows pages 1–4. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 6.262: Discrete Stochastic Processes Lecture 24 - 5/5/10 • Sequential decision theory (when to de- cide?) – Cherno ff bound and tilted rv’s – Wald’s identity – Bound on threshold crossing probability – Sequential decisions • Martingales – Isolation of past – Submartingales and supermartingales – Stopped martingales and Wald identity 1 Cherno ff bound review Assume { X n ; n ≥ 1 } is IID sequence with X < and E e Xr < ∞ for r ≥ . Let γ ( r ) = ln( E e Xr ) . Then for a > X Pr { S n ≥ na } ≤ exp { n [ γ ( r )- ra ] } for r > where S n = X 1 + · · · + X n . = = = = = = slope = X r o γ ( r o ) r * slope = γ ( r o ) = a γ ( r o )- r o a r Optimizing r = r o 2 Tilted probabilities Assume X is discrete with PMF p X ( x ) . For any given r , define the tilted PMF q X,r ( x ) = p X ( x ) exp[ rx- γ ( r )] , where the tilted X is the same mapping from sample points to real values as X , but the prob- ability is di ff erent, tilted to favor large values for r > . = = = = =-2 -1 1 2 p X ( x )-2 -1 1 2 = = = = = q X,r ( x ) x q X,r ( x ) = ∑ x p X ( x ) e xr E e Xr = 1 3 Assume X n ; n ≥ 1 are IID and also define them to be IID in this new tilted measure. Then q X n , r ( x 1 , . . . , x n ) = p X n ( x 1 , . . . , x n ) exp[ n i =1 rx i- n γ ( r )] Sum over all x 1 , . . . , x n that have a common sum s n . Then q S n ,r ( s n ) = p S n ( s n ) exp[ rs n- n γ ( r ) Thus a sum of tilted IID rv’s are related to the untilted sum in the same way as a single tilted rv is related to the untilted rv. This is a key to large deviation analysis. 4 The mean of X , tilted by r , is E r [ X ] = x x p X ( x ) exp[ rx- γ ( r ) = ∑ x p X ( x ) exp( rx ) E [exp( rX )] = γ ( r ) Now consider Pr { S n ≥ na } again, using tilted probabilities with r chosen so that γ ( r ) = a . Pr { S n ≥ na } = s n ≥ na p S n ( s n ) = s n ≥ na q S n ,r ( s n ) exp[- rs n + n γ ( r )] ∼ exp[- rna + n γ ( r )] since S n with the tilted measure obeys the CLT and is centered on na . Thus the Cherno ff bound is exponentially tight. 5 Thm: (Wald) Let γ ( r ) = ln( E [exp( rX )]) exist over ( r- < < r + ) . Let J be trial at which S n first crosses α > or β < . Then E [exp( rS J- J γ ( r )] = 1 for r ∈ ( r- , r + ) Pf: q X n ,r ( x 1 , . . . , x n ) = p X n ( x 1 , . . . , x n ) exp[ rs n- n γ ( r )] q J,r ( n ) = x n : { J = n } p X n ( x n ) exp[ rs n- n γ ( r )] = E [exp( rS n- n γ ( r )) | J = n ] p J ( n ) Summing both sides over n completes the proof....
View Full Document

{[ snackBarMessage ]}

### Page1 / 13

6.262.Lec24 - 6.262 Discrete Stochastic Processes Lecture...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online