Discrete-time stochastic processes

The steady state expected reward per unit time

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: benius theorems, which are useful in their own right and also provide the necessary results about [P ]n in general. 4.4 Perron-Frobenius theory A real vector x (i.e., a vector with real components) is deﬁned to be positive, denoted x &gt; 0, if xi &gt; 0 for each component i. A real matrix [A] is positive, denoted [A] &gt; 0, if Aij &gt; 0 for each i, j . Similarly, x is non-negative, denoted x ≥ 0, if xi ≥ 0 for all i. [A] is non-negative, denoted [A] ≥ 0, if Aij ≥ 0 for all i, j . Note that it is possible to have x ≥ 0 and x 6= 0 without having x &gt; 0, since x &gt; 0 means that all components of x are positive and x ≥ 0, x 6= 0 means that at least one component of x is positive and all are non-negative. Next, x &gt; y and y &lt; x both mean x − y &gt; 0. Similarly, x ≥ y and y ≤ x mean x − y ≥ 0. The corresponding matrix inequalities have corresponding meanings. We start by looking at the eigenvalues and eigenvectors of positive square matrices. In what follows, when we assert that a matrix, vector, or number is positive or non-negative, we implicitly mean that it is real also. We will prove Perron’s theorem, which is the critical result for dealing with positive matrices. We then generalize Perron’s theorem to the Frobenius theorem, which treats a class of non-negative matrices called irreducible matrices. We ﬁnally specialize the results to stochastic matrices. 4.4. PERRON-FROBENIUS THEORY 151 Perron’s theorem shows that a square positive matrix [A] always has a positive eigenvalue ∏ that exceeds the magnitude of all other eigenvalues. It also shows that this ∏ has a right eigenvector ∫ that is positive and unique within a scale factor. It establishes these results by relating ∏ to the following frequently useful optimization problem. For a given square matrix [A] &gt; 0, and for any non-zero vector6 x ≥ 0, let g (x ) be the largest real number a for which ax ≤ [A]x . Let ∏ be deﬁned by ∏= sup g (x ). (4.20) x 6=0 ,x ≥0 P We can express g (x ) explicitly by rewriting ax ≤ Ax as axi ≤ j Aij xj for all i. Thus, the largest a for which this is satisﬁed is P j Aij xj g (x ) = min gi (x ) where gi (x ) = . (4.21) i xi P Since [A] &gt; 0, x ≥ 0 and x 6= 0 , it follows that the numerator i Aij xj is positive for all i. Thus gi (x ) is positive for xi &gt; 0 and inﬁnite for xi = 0, so g (x ) &gt; 0. It is shown in Exercise 4.10 that g (x ) is a continuous function of x over x 6= 0 , x ≥ 0 and that the supremum in (4.20) is actually achieved as a maximum. Theorem 4.5 (Perron). Let [A] &gt; 0 be a M by M matrix, let ∏ &gt; 0 be given by (4.20) and (4.21), and let ∫ be a vector x that maximizes (4.20). Then 1. ∏∫ = [A]∫ and ∫ &gt; 0. 2. For any other eigenvalue µ of [A], |µ| &lt; ∏. 3. If x satisﬁes ∏x = [A]x, then x = β∫ for some (possibly complex) number β . Discussion: Property (1) asserts not only that the solution ∏ of the optimization problem is an eigenvalue of [A], but also that the optimizing vector ∫ is an eigenvector and is strictly positive. Property (2) says that ∏ is strictly greater than the magnitude of any other eigenvalue, and thus we refer to it in what follows as the largest eigenvalue of [A]. Property (3) asserts that the eigenvector ∫ is unique (within a scale factor), not only among positive vectors but among all (possibly complex) vectors. Proof* Property 1: We are given that ∏ = g (∫ ) ≥ g (x ) for each x ≥ 0 , x 6= 0 . (4.22) P We must show that ∏∫ = [A]∫ , i.e., that ∏∫i = j Aij ∫j for each i, or equivalently that P j Aij ∫j ∏ = g (∫ ) = gi (∫ ) = for each i. (4.23) ∫i Thus we want to show that the minimum in (4.21) is achieved by each i, 1≤i≤M. To show this, we assume the contrary and demonstrate a contradiction. Thus, suppose that 6 Note that the set of nonzero vectors x for which x ≥ 0 is diﬀerent from the set {x &gt; 0} in that the former allows some xi to be zero, whereas the latter requires all xi to be zero. 152 CHAPTER 4. FINITE-STATE MARKOV CHAINS g (∫ ) &lt; gk (∫ ) for some k. Let e k be the kth unit vector and let ≤ be a small positive number. The contradiction will be to show that g (∫ + ≤e k ) &gt; g (∫ ) for small enough ≤, thus violating (4.22). For i 6= k, P P j Aij ∫j + ≤Aik j Aij ∫j gi (∫ + ≤e k ) = &gt; . (4.24) ∫i ∫i gk (∫ + ≤e k ), on the other hand, is continuous in ≤ &gt; 0 as ≤ increases from 0 and thus remains greater than g (∫ ) for small enough ≤. This shows that g (∫ + ≤e k ) &gt; g (∫ ), completing the contradiction. This also shows that ∫k must be greater than 0 for each k. Property 2: Let µ be any eigenvalue of [A]. Let x 6= 0 be a right eigenvector (perhaps complex) for µ. Taking the magnitude of each side of µx = [A]x , we get the following for each component i X X |µ||xi | = | Aij xj | ≤ Aij |xj |. (4.25) j j Let u = (|x1 |, |x2 |, . . . , |xM |), so (4.25) becomes |µ|u ≤ [A]u . Since u ≥ 0,...
View Full Document

This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online