Discrete-time stochastic processes

# 7 also pr wn is equal to the probability that the

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: n increment of size δ , and let [Wδ ] be the transition matrix for [Λ]. b) Express [Wδ ]n in terms of [V ] and [Λ]. c) Expand [Wδ ]n in the same form as (6.36). d) Let t be an integer multiple of δ , and compare [Wδ ]t/δ to [P (t)]. Note: What you see from this is that ∏i in (6.36) is replaced by (1/δ )ln(1 + δ ∏i ). For the steady state term, ∏1 = 0, this causes no change, but for the other eigenvalues, there is a change that vanishes as δ → 0. Exercise 6.10. Consider the three state Markov process below; the number given on edge (i, j ) is qij , the transition rate from i to j . Assume that the process is in steady state. ✎☞ 1✐ ✍✌ ■ 2 1 1 2 4 ✎☞ ❘ 3✙ ✍✌ q✎☞ 2 ✍✌ ✯ 4 6.9. EXERCISES 271 a) Is this process reversible? b) Find pi , the time-average fraction of time spent in state i for each i. c) Given that the process is in state i at time t, ﬁnd the mean delay from t until the process leaves state i. d) Find πi , the time-average fraction of all transitions that go into state i for each i. e) Suppose the process is in steady state at time t. Find the steady state probability that the next state to be entered is state 1. f ) Given that the process is in state 1 at time t, ﬁnd the mean delay until the process ﬁrst returns to state 1. g) Consider an arbitrary irreducible ﬁnite-state Markov process in which qij = qj i for all i, j . Either show that such a process is reversible or ﬁnd a counter example. Exercise 6.11. a) Consider an M/M/1 queueing system with arrival rate ∏, service rate µ, µ > ∏. Assume that the queue is in steady state. Given that an arrival occurs at time t, ﬁnd the probability that the system is in state i immediately after time t. b) Assuming FCFS service, and conditional on i customers in the system immediately after the above arrival, characterize the time until the above customer departs as a sum of random variables. c) Find the unconditional probability density of the time until the above customer departs. Hint: You know (from splitting a Poisson process) that the sum of a geometrically distributed number of IID exponentially distributed random variables is exponentially distributed. Use the same idea here. Exercise 6.12. a) Consider an M/M/1 queue in steady state. Assume ρ = ∏/µ < 1. Find the probability Q(i, j ) for i ≥ j > 0 that the system is in state i at time t and that i − j departures occur before the next arrival. b) Find the PMF of the state immediately before the ﬁrst arrival after time t. c) There is a well known queueing principle called PASTA, standing for “Poisson arrivals see time averages”. Given your results above, give a more precise statement of what that principle means in the case of the M/M/1 queue. Exercise 6.13. A small bookie shop has room for at most two customers. Potential customers arrive at a Poisson rate of 10 customers per hour; they enter if there is room and are turned away, never to return, otherwise. The bookie serves the admitted customers in order, requiring an exponentially distributed time of mean 4 minutes per customer. a) Find the steady state distribution of number of customers in the shop. b) Find the rate at which potential customers are turned away. c) Suppose the bookie hires an assistant; the bookie and assistant, working together, now serve each customer in an exponentially distributed time of mean 2 minutes, but there is 272 CHAPTER 6. MARKOV PROCESSES WITH COUNTABLE STATE SPACES only room for one customer (i.e., the customer being served) in the shop. Find the new rate at which customers are turned away. Exercise 6.14. Consider the job sharing computer system illustrated below. Incoming jobs arrive from the left in a Poisson stream. Each job, independently of other jobs, requires pre-processing in system 1 with probability Q. Jobs in system 1 are served FCFS and the service times for successive jobs entering system 1 are IID with an exponential distribution of mean 1/µ1 . The jobs entering system 2 are also served FCFS and successive service times are IID with an exponential distribution of mean 1/µ2 . The service times in the two systems are independent of each other and of the arrival times. Assume that µ1 > ∏Q and that µ2 > ∏. Assume that the combined system is in steady state. ∏ Q✲ ✲ System 1 µ1 ✲ System 2 µ2 ✲ 1−Q a Is the input to system 1 Poisson? Explain. b) Are each of the two input processes coming into system 2 Poisson? Explain. d) Give the joint steady state PMF of the number of jobs in the two systems. Explain brieﬂy. e) What is the probability that the ﬁrst job to leave system 1 after time t is the same as the ﬁrst job that entered the entire system after time t? f ) What is the probability that the ﬁrst job to leave system 2 after time t both passed through system 1 and arrived at system 1 after time t? Exercise 6.15. Consider the following combined queueing system. The ﬁrst queue system is M/M/1 with service rate µ1 . The second queue system has IID exponent...
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online