# Module2_4 - Module 2 Lecture 4 Fundamental Concepts...

This preview shows pages 1–7. Sign up to view the full content.

Module 2, Lecture 4 Fundamental Concepts: Information Inequalities II G.L. Heileman Module 2, Lecture 4

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Stochastic Processes A number of the inequalities we consider next are associated with Markov chains, a special type of stochastic process. Definition (Discrete stochastic process) A discrete stochastic (or random) process is an indexed collection of discrete RVs X 1 , . . . , X n = { X i } , in which there can be an arbitrary dependence among the X i ’s. Such a process is completely described by the joint pdf Pr { X 1 = x 1 , . . . , X n = x n } = p ( x 1 , . . . , x n ) = p ( x 1 ) p ( x 2 | x 1 ) p ( x 3 | x 2 , x 1 ) · · · p ( x n | x n - 1 , . . . , x 1 ) . We have already considered the special case when the X i ’s are iid. The independence part of iid implies p ( x 1 , . . . , x n ) = p ( x 1 ) p ( x 2 ) · · · p ( x n ) . This is also referred to as a zero memory source . G.L. Heileman Module 2, Lecture 4
Stochastic Processes Definition (Stationary stochastic process) A stochastic process is stationary if the joint distribution of any subset of the sequence of RVs is invariant with respect to shifts in the index, i.e., Pr { X 1 = x 1 , . . . , X n = x n } = Pr { X 1+ l = x 1 , . . . , X n + l = x n } for any shift l , and for any x 1 , . . . , x n . I.e., the pdf does not change over time. An iid stochastic process is stationary. Proof: Pr { X 1 = x 1 , . . . , X n = x n } = Pr { X 1 = x 1 } · · · Pr { X n = x n } (indepen.) = Pr { X 1 } · · · Pr { X n } (ident. distrib.) = Pr { X 1+ l } · · · Pr { X n + l } (ident. distrib.) = Pr { X 1+ l = x 1 , . . . , X n + l = x n } for any shift l . G.L. Heileman Module 2, Lecture 4

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Markov Chains Definition ( m -th order Markov chain) A discrete stochastic process X 1 , X 2 , . . . is said to be an m -th order Markov chain (or process) if for n = 1 , 2 , . . . p ( x n +1 | x n , . . . , x 1 ) = p ( x n +1 | x n , . . . , x n - m +1 ) . I.e., the probability of the next “output” depends only on the previous m outputs. Thus, the joint pdf of an 1-st order Markov chain can be written as p ( x , . . . , x n ) = p ( x 1 ) p ( x 2 | x 1 ) p ( x 3 | x 2 ) · · · p ( x n | x n - 1 ) . G.L. Heileman Module 2, Lecture 4
Markov Chains Definition (Time invariant Markov chain) An m -th order Markov chain is said to be time invariant if the conditional probability p ( x n +1 | x n , . . . , x n - m +1 ) is independent of n . I.e., the next output still depends on the previous m outputs, but not on the time that they occured. Thus, for n = m , m + 1 , . . . Pr { X n +1 = α m +1 | X n = α m , . . . , X 1 = α 1 } = Pr { X m +1 = α m +1 | X m = α m , . . . , X 1 = α 1 } for any α 1 , . . . , α m +1 . In the text (and in these notes), the term Markov chain will generally mean a 1-st time invariant Markov chain. G.L. Heileman Module 2, Lecture 4

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Markov Chains If { X i } is a Markov chain, then X n is called the state of the Markov chain at time n . A time-invariant 1-st order Markov chain is completely characterized by its initial state and a probability transition matrix P = [ P ij ], where P ij = Pr { X n +1 = j | X n = i } for all possible states i and j . I.e., P ij expresses the probability of progressing from state i to state j in one trial.
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern