processes
M. Stochastic Veeraraghavan; Feb. 10, 2004 A stochastic process (SP) is a family of random variables { X ( t ) t T } defined on a given probability space, indexed by the time variable t , where t varies over an index set T . [1]
Just as a random variable assigns a number to each outcome s in a sample space S , a stochastic process assigns a sample function x ( t, s ) to each outcome s . 1
A sample function x ( t, s ) is the time function associated with outcome s of an experiment. The ensemble of a stochastic process (sp) is the set of all possible time functions that can result from an experiment.
x ( t, s 1 ) s1 s2 s3 x ( t, s 2 ) x ( t, s 3 )
Sample function
Figure 1:Representation of a stochastic process - relation to random variables
t = t1 X ( t 1 )is a random variable
Example of an sp: Number of active calls M ( t ) at a switch at time t . One trial of the experiment yields the sample function m ( t, s ) where the number of active calls is measured for every second over one 15 minute interval. Say this measurement is taken every day starting at 10AM. An ensemble average can be obtained from all measurements for t = 2min after 10AM. Or a time average can be obtained over a 15-minute interval based on one-days measurements.
1. Most of the statements in this writeup have been taken verbatim from [2]; exceptions are primary from [1] as noted.
1
Types of stochastic processes: Discrete value and continuous value; Discrete time and continuous time. If each random variable X ( t ) for different values of t are discrete rv, then the sp is a discrete value. If the process is defined only for discrete time instants, then it is a discrete time sp.
Random sequence: for a discrete time process, a random sequence X n is an ordered sequence of random variables X 0 , X 1 , ....- Essentially a random sequence is a discrete-time stochastic process. Relation between sp and rv: A discrete value sp is defined by the joint PMF P X ( t1 ), X ( t k ) ( x 1, , x k ) , while a continuous value sp is defined by the joint PDF.
Independent, identically distributed random sequence: Let X n be an iid random sequence. All X i has the same distribution. Therefore P Xi ( x ) = P X ( x ) . For a discrete value process, the sample vector X n1, , X nk has the joint PMF (by property of independent random variables)
n
P X ( t1 ), X ( tk ) ( x 1, , x k ) = P X ( x 1 )P X ( x 2 )P X ( x n ) = For continuous value sp, same operation but with pdf.
PX ( xi )
i=1
(1)
A counting process: A stochastic process N ( t ) is a counting process if for every sample function n ( t, s ) = 0 for t < 0 and n ( t, s ) is integer valued and nondecreasing with time. N(t)
S1 X1 X2
Figure 2:Sample path of a counting process
S2 S3 X3
X4
S4
2
A counting process is always a discrete-value SP (because n ( t, s ) is integer valued) but can be either continuous-time or discrete-time. A renewal process is a counting process in which the interarrival times X 1, X 2, are an iid random sequence. Poisson process: is a counting process: in which number of arrivals in ( t 0, t 1 ] , which is N ( t 1 ) N ( t 0 ) , is a Poisson random variable with expected value ( t 1 t 0 ) For any pair of non-overlapping intervals ( t 0, t 1 ] and ( t 0', t 1' ] , number of arrivals in each interval N ( t 1 ) N ( t 0 ) and N ( t 1' ) N ( t 0' ) are independent random variables. M , the number of arrivals in the interval ( t 0, t 1 ] , which is N ( t 1 ) N ( t 0 ) , has a PMF [ ( t t ) ] m e ( t1 t0 ) 1 0 ---------------------------------------------------PM ( m ) = m! 0 m = 0, 1, (see RV.pdf for Poisson rv pmf) otherwise
(2)
A Poisson process is a renewal process, i.e., the Xs form an iid random sequence. Consider a set of sample functions of a Poisson process as in Fig. 1. If we take the random variable N ( t 1 ) at time instant t 1 (just like X ( t 1 ) in Fig. 1) then the distribution of this random variable is Poisson with parameter t 1 because it represents the cumulative number of arrivals until time t 1 from time 0. It is not sufficient to describe the distribution of the random variable at any one instant in time to define an SP. To completely define a stochastic process, we need to give the joint PMF, which is given by the following Theorem. Theorem 6.2 of [2]: For a Poisson process of rate , the joint PMF of N ( t 1 ), , N ( t k ) , t 1 < < t k is
3
n1 e 1 n2 n1 e 2 nk nk 1 e k 1 k ----------------- ------------------------- ----------------------------- 2 0 n1 nk P N ( t 1 ), , N ( t k ) ( n 1, , n k ) = n 1! ( n 2 n 1 )! ( n k n k 1 )! 0 otherwise where i = ( t i t i 1 ) . Why is the following incorrect? P N ( t 1 ), , N ( t k ) ( n 1, , n k ) = P N ( t1 ) ( n 1 )P N ( t2 ) ( n 2 )P N ( tk ) ( n k )
(3)
(4)
This is because N ( t 1 ), N ( t 2 ), , N ( t k ) are not independent random variables. ( 0, t 2 ) overlaps with ( 0, t 1 ) , hence N ( t 2 ) is not independent of N ( t 1 ) . We use the independence rule P ( A B ) = P ( A )P ( B ) , where A = N ( t1 )
k
and
1 e 1 --------------- . B = N ( t 2 ) N ( t 1 ) , both of which are Poisson r.v. and hence we use the PMF k! To fully specify a stochastic process, is it sufficient to define the distributions of the random variables at different instants in time? Answer is no: we need the joint PMF of these random variables. If the SP is a renewal process, the intervals are independent and so using the independence rule, the joint PMF is easy to define. Properties of Poisson processes: I. Relation between a Poisson process and exponential distribution: For a Poisson process of rate , the interarrival times X 1, X 2, are an iid with the exponential PDF with parameter . [2, page 214]: Proof [4, page 35]: The first time interval, i.e., time to the first arrival X 1 , can be characterized as follows: P ( X1 > t ) = P ( N ( t ) = 0 ) = e
t
(5)
This means X 1 is exponentially distributed with parameter . Now to find the distribution of X 2 conditional on X 1 :
4
P ( X 2 > t X 1 = s ) = P { 0 arrivals in ( s , s + t ] X 1 = s } = P { 0 arrivals in ( s , s + t ] } = e
t
(6)
Since the number of arrivals in the interval ( s , s + t ] is independent of X 1 , we drop the condition in the second step of the above equation. 1 The above means that X 2 is also an exponential random variable with mean -- . Furthermore we can conclude from the above that X 2 is independent of X 1 . Note that N ( t 2 t 1 ) and N ( t 3 t 2 ) are not iid unles ( t 2 t 1 ) is equal to ( t 3 t 2 ) if they are non-overlapping. They are independent, but do not have the same parameter in order to be identically distributed. Extending the argument, let t n 1 = x 1 + x 2 + + x n 1 , where X 1 = x 1, , X n 1 = x n 1 . P ( X n > x X 1 = x 1, X n 1 = x n 1 ) = P ( N ( t n 1 + x ) N ( t n 1 ) = 0 X 1 = x 1, X n 1 = x n (7) 1) because if N ( t n 1 + x ) N ( t n 1 ) is not 0 then it means X n ended before x time passed. Since the event N ( t n 1 + x ) N ( t n 1 ) = 0 is independent of the lengths of X 1, , X n 1 , we can equate (7) to be P ( N ( t n 1 + x ) N ( t n 1 ) = 0 ) . Setting N ( t n 1 + x ) N ( t n 1 ) = M , P M ( 0 ) ( x ) e x from (2) is equal to P M ( 0 ) = ----------------------- = e . Since the number of arrivals in the interval 0! [ t n 1, t n 1 + x ] is independent of the past history described in X 1, X n 1 . Conclusion: X n are an exponentially distributed random variable, and is independent of the interval n . In other words, X 1, X 2, are iid random variables.
0 x
II. Approximating exponential For functions: every t 0 and 0 , P { N ( t + ) N ( t ) = 0 } = 1 + o ( ) P { N ( t + ) N ( t ) = 1 } = + o ( ) P{N(t + ) N(t) = 2} = o() where lim o ( ) = 0 ---------0 These can be verified by using Taylors series expansion of e
(8) (9) (10) (11)
( ) = 1 + ------------- . 2
2
5
III. Merging of Poisson processes: If two more independent Poisson processes A 1, A k are merged into a single process A = A 1 + + A k , the latter is a Poisson process with its parameter equal to the sum of the rates of its components. IV. Splitting of a Poisson process: If a Poisson process is split into two processes by independently assigning each arrival to the first and second of these processes with probability p and ( 1 p ), respectively, the two arrival processes are Poisson (note: it is essential that the assignment of each arrival is independent of the previous assignment; if for example, all even arrivals are sent to the first queue and all odd to the second, the two processes will not be Poisson). Examples of stochastic processes: 1. Counting process 2. Renewal process 3. Poisson process 4. Markov process 5. Brownian motion Note that autocovariance and autocorrelation are time-varying functions (unlike covariance and correlation of two random variables, which are numbers). From [2, pg. 217, 226]: Autocovariance and autocorrelation functions indicate the rate of change of the sample functions of a stochastic process. Autocovariance function of a stochastic process X ( t ) is C X ( t, ) = Cov [ X ( t ), X ( t + ) ] Autocorrelation function of a stochastic process X ( t ) is R X ( t, ) = E [ X ( t )X ( t + ) ] Relation between these functions: C X ( t, ) = R X ( t, ) X ( t ) X ( t + ) time. [Strict stationary] (14) (13) (12)
Stationary process [2, pg. 220, 226]: An SP is stationary if the randomness does not vary with
6
A stochastic process X ( t ) is stationary if and only if for all sets of time instants t 1, t m , and any time difference (i.e., joint pdf does not change with time) f X ( t 1 ), X ( tm ) ( x 1, , x m ) = f X ( t 1 + ), X ( tm + ) ( x 1, , x m ) (15)
Same thing for a stationary random sequence - where the time difference is a discrete value k . Properties of a stationary process (mean stays constant and autocovariance and autocorrelation only depend on time interval): X ( t ) = X R X ( t, ) = R X ( 0, ) = R X ( ) C X ( t, ) = R X ( ) X = C X ( ) Wide-sense stationary process [2, pg. 223, 226]: An SP is w.s.stationary if the expected value is constant with time and the autocorrelation depends only on the time difference between two random variables. X ( t ) is a w.s. stationary process if and only if for all t : E [ X ( t ) ] = X R X ( t, ) = R X ( 0, ) = R X ( ) Similar definition for a wide-sense stationary random sequence. Markov process [1, pg. 337] is an sp whose dynamic behavior is such that probability distribution for its future development depends only on its present state and not how the process arrived in that state. If state space is discrete, then it is a Markov chain. (17)
2
(16)
If X 's are discrete random variables we get this Definition of a DTMC: A DTMC { X n n = 0, 1, 2, } is a discrete time discrete value random sequence such that given X 0, X 1, , X n 1 , the next random variable X n depends only on X n 1 through the transition probability P [ X n = j X n 1 = i, X n 2 = i n 2, , X 0 = i o ] = P [ X n = j X n 1 = i ] = P ij (18)
7
Definition of a CTMC [3]: A CTMC is { X ( t ) t 0 } is a continuous time, discrete value random process if for t o < t 1 < < t n < t , with t and t r 0 for r = 1, 2, , n , P [ X ( t ) = j X ( t n ) = i, X ( t n 1 ) = i n 1, , X ( t 0 ) = i o ] = P [ X ( t ) = j X ( t n ) = i ] (19)
Relationship between Markov chain and Poisson process: Is the Poisson process a special case of a Markov chain? It is indeed a CTMC. Heres why. By definition of Poisson process, the number of arrivals in two non-overlapping intervals are independent; therefore the number of arrivals in ( t n, t ) is independent of the arrivals in ( t 0, t n ) if t o < t 1 < < t n < t . The random variable N ( t ) does depend upon N ( t n ) . Its dependence on N ( t n ) is as follows: P [ N ( t ) = j N ( tn ) = i ] = P [ N ( t ) N ( tn ) = j i ] But it does not depend on N ( t 0 ) , N ( t 1 ) , etc. Therefore, P [ N ( t ) = j N ( t n ) = i, N ( t n 1 ) = i n 1, , N ( t 0 ) = i o ] = P [ N ( t ) = j N ( t n ) = i ], (21) (20)
which by the definition of a CTMC as per (19) implies the ...