This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: CHAPTERl6 Quadratic Forms
and Deﬁnite
Matrices The natural starting point for the study of optimization problems is the simplest
such problem: the optimization of a quadratic form. There are a number of good
reasons for studying quadratic optimization problems ﬁrst. Quadratic forms are
the simplest functions after linear ones. Like linear functions, they have matrix
representations, so that studying the properties of a quadratic form reduces to
studying properties of a symmetric matrix. Quadratic forms provide an excellent
introduction to the vocabulary and techniques of optimization problems. Fur
thermore, the second order conditions that distinguish maxima from minima in
economic optimization problems are stated in terms ofquadratic forms. Finally, a
number of economic optimization problems have a quadratic objective function,
for example, risk minimization problems in ﬁnance. where riskiness is measured
by the (quadratic) variance of the returns from investments. Example 16.] Among the functions of one variable, the simplest functions with
a unique global extremum are the pure quadratics: y = x3 and y I ~x3. The
former has a global minimum at x = (l; the latter has a global maximum at
x = 0, as illustrated in Figure 16.1. 16.1 QUADRATIC FORMS
Recall the definition of a quadratic form on R“ from Section l3.3. Definition A quadratic form on R“ is a realvalued function of the form Q(X1, . . . , X") 2 2 (IQ/WAC}, (i) [$1] in which each term is a monomial of degree two.
The presentation in Section 13.3 showed that each quadratic form Q can be
represented by a symmetric matrix A so that
Q(x) : XT ~A ' x. (Z) 375 Figure
16.1 376 QUADRATIC FORMS AND DEFINITE MATRlCES [161 The ﬁtnctions afﬁx) = x: and [(x) I: ~x3.
For example, the general two—dimensional quadratic form
ﬁ 7,
(1111‘? 'l“ “13X1X3 “l‘ (£32065 can be written as And the general three dimensional quadratic form 7 a r 7
(1111‘? “l,“ (133353 + £133.76; “l‘ a12x1x2 + C(13XIX3 “l“ £133X3X3 can be written as: I l
(11] :(lp 36113 ,
w _ X1
(’6: X: {2) i012 61:: i023 1‘2
l l X3
3611: 3612; an 16.2 DEFINITENESS OF QUADRATIC FORMS (3) (4) A quadratic form always takes an the value zero at the point x z 0. Its distiax
guishing characteristic is the set of values it takes when x i 0. In this chapter. we 11 6.21 DEFthTENESS or QUADRATIC FORMS 377 focus on the question of whether X z 0 is a max, a min or neither of the quadratic
forms under consideration The general quadratic form of one variable is v 2 (ix; 1fa > t), then ax: is
always 2“: 11 and equals (1 only when: r: (1. Such a form is called positive deﬁnite;
x '2 t) is its global minimizer. lt‘a < (1. then at2 is always S (l and equals 1) only
when r r: 1). Such a quadratic form is called negative deﬁnite; x i t) is its global
maximize]: Figure 16.1 illustrates these two situations. in two dimensions, the quadratic torm Q;(.r;, .t‘g) : if + if is always greater
than zero at (15,, .t‘;) i (0.11). So, we call Q, positive deﬁnite. Quadratic forms
like Q3(.\'1, x3) : will '* t§ which are strictly negative except at the origin, are
called negative deﬁnite. Quadratic forms like Qgtiri, 1‘3): t; _ if which take on
both positive and negative values (Q41, (1) 1: +1 and QM). 1) a: l ) are called
indeﬁnite. There are two intermediate cases: a quadratic form which is always 1:: 11 but
may equal zero at some nonzero its is called positive semideﬁnite. This property
is illustrated by the quadratic form r , v i , t
Q4(‘r1, 1‘3} 2 (x1 + X1)“ ‘2 .t'; r Exit; rt“ .t‘g, which is never negative but which equals zero at nonzero points such as (xi, x3):
(1, ~ 1) or (~2. 2 . A quadratic form like Q5(X1,X1): mm i” X3)“: which is never
positive but can be zero at points other than the origin. is called negative semidef
inite. Figures 162 through 16.6 present the graphs ot‘the above ﬁve quadratic forms
Q1, . . . , Q5. Every quadratic form on R‘2 has a graph similar to one of these ﬁve.
For example, every positive deﬁnite quadratic: on R‘2 has a howl shaped graph as
in Figure 16.2, and every indeﬁnite quadratic: on R2 has a saddleshaped graph as
in Figure 16.4. 4 xi 7 Graph oft/w positive deﬁnite form Q, {X}, 1‘3) :2 x! 4% r23. Figure
16.2 /
\5 Figure 16.3 (imp/z (Jr/'I/Ic’uc’gtlll'k‘c’(ftf/illi1(’f})l'lit QgUw X3) ~ y]: \f:. \ , Figure 16.4 The graph oft/w imiq/iniuﬂ/U/‘In Q4“. 3;) A”; Ug.\ \ i
M ,/ Figure 16.5 T/u’grupi: (g/‘l/u'pmm'w \(‘HIR/(f/rlliit’l/IUIWI Q,;(.\';, Q) ,, (n 4 ,x'3)3. ‘ t 6.21 ott’thtiMess < )ti (MENDRATK‘ rows 379 Figure
IM’grrip/z oft/ivnegative.x't'mirlcﬁ/initt’form Q)~;l:.\"g..\'j) # '(tr; Xi)”. [6.6 Definite Symmetric Matrices A symmetric matrix is called positive deﬁnite pmitive semidelinite, negative
definite ete. according to the tlL‘lli‘lllCllCSS ot’ the. eoriesponding quatlratie l‘orm
th) w xii/ix. Since we will usually be applying this terminology to xymmetrie
matrices tlit‘eetly. we focus on ouch matrices for our formal deﬁnitions ol‘tletinite— ”€53.
Definition Let A he (in u xi 1! symmetric l'illllflx. then A is: (a) positive deﬁnite it' x’.tx '~ t) for all x z 0 in R’\ (1’)) positive Semideﬁnite il‘ x’i/lx 4.: t) for all x: {1 0 in R“, (M negative deﬁnite il‘x’nx t t) for all x €51 0 in R". (4!) negative semideﬁnite if X’Ax t) for all x #4 0 in R". and (t') indeﬁnite ill X’Ax , U for some x in R" and ‘ t) tor some other it in R“. Remark A matrix that is positive (negative) definite is automatically positive
(negative) xemitlelinite. Otherwise. every xymmetrie matrix falls into one of the
athove live eategories. Application: Second Order Conditions and Convexity The delinitenexs of a symmetrie matrix plays (in important role in economic
theory and in applied mathematies in general. For example, for it function it [(1') of one variable. the ,sign of the xeeond derivative flint) at :t critical point
A}, of f gives a necessary eontlition and ti sullieient condition for determining
whether I” i5 it maximum of f, at minimum of ft or neither, The generalization
of this aeeontl derivative text to higher tlilliCliSiCHlS involves eheeking whether the 380 QUADRATIC FORMS AND DEFINITE MATRICES H 6] second derivative matrix (or Hessian) of f is positive definite” neg
or indeﬁnite at a critical point of f. In a similar vein, a function y 2: f(x) of one variable is concav:
derivative f”(vx) is E: (l on some interval. The generalization of this n
dimensions states that a function is concave on some region if its sec:
matrix is negative semideﬁnize for all x in the region. Application: Conic Sections
In plane geometry, the conic section described by the level curve
lenxz) E (111x12 + aisxixz 4” (13215 ’5“ is completely determined by the deﬁniteness of Q or of its associat: 1
an “56112 A a ‘ “ .
3012 6122 Figure 16.7 illustrates the connection. The horizontal plane {X3 :3 l}
in Figure 16.2 in an ellipse or Circle. Therefore, ifA is positive deﬁt
is an ellipse or circle. Since {353 x 1} cuts the graph in Figure 16.4 in
as also illustrated in Figure 16.7, equation (5) describes an hyp Figure
16.7 Levels sets of graphs of quadratic forms. : definite, ts second
to higher
derivative (3) ttttrix .the graph
the set (5)
typerbola,
ila ifA is L/V/ H 6.2] DEFlNlTENESS OF QUADRATlC FORMS 381 indeﬁnite. Since {x3 2 l} cuts the graph of Figure 16.5 in a pair of parallel lines,
equation (5) deﬁnes two lines ifA ispositive semideﬁnite but not positive deﬁnite.
Finally, since the plane {X3 : 1} lies strictly above the graphs in Figures 16.3 and
16.6, the set (5) is empty if A is negative deﬁnite or even negative semideﬁnite. Principal Minors of a Matrix In this section we will describe a simple test for the deﬁniteness of a quadratic
form or of a symmetric matrix. To describe this algorithm, we need some more
vocabulary. Definition Let A be an it X 22 matrix. A k X k submatrix of A formed by
deleting n M k columns, say columns 2'], i3, . . . , tack and the same it — k rows,
rows i1, i3, . . ., inck, from A is called a kth order principal submatrix of A. The
determinant of a k X k principal submatrix is called a kth order principal minor ofA. Example 16.2 For a general 3 X 3 matrix an £112 6113
A : €321 €122 €123 ,
€131 €132 6333 there is one third order principal minor: det(A). There are three second order
principal minors: l
l
l
l
l
l
l
i
g
E
l
l
l
g
5
i
l
l
l
l (I) :H :12 , formed by deleting column 3 and row 3 fromA;
21 22 (2) :11 :13 , formed by deleting column 2 and row 2 from A;
33 33 (3) 422 {£23 , formed by deleting column 1 and row i fromA.
(132 “33 There are three ﬁrst order principal minors: (i) [an [, formed by deleting the, last 2 rows and columns,
(2) [egg 1, formed by deleting the ﬁrst and third rows and the ﬁrst and third
columns, and l
l
l
l
l
l
l
l
i (3) l 6133 l, formed by deleting the ﬁrst 2 rows and columns. l
E It is important to understand why no other submatrix of A is a principal submatrix. For practice, list all the principal minors of a general 4 X 4 matrix. Among the kth order principal minors of a given matrix, there is one special
one that we want to highlight. 382 QUADRATIC FORMS AND DEFlNlTE MATRICES {l 6} Definition Let A be an n X H matrix. The kth order principal submatrix ofA
obtained by deleting:rthe last it ~ It rows and the last I:  k columns from A is
called the kth order leading principal submatrix of A. lts determinant is called
the kth order leading principal minor of/t. We will denote the kth order leading
principal submatrix by AR and the corresponding leading principal minor by lAkl. An n X :1 matrix has n leading principal submatricesm the top—leftmost l X l
submntrix, the top»let‘tmost 2 X 2 submutrixg etc. For the general 3 X 3 matrix of
Example 16.2, the three leading principal minors are an all an (1)1 (1131
Y an (133 (£33
6121 112: lat! l,
(131 €132 €133 The following theorem provides a straightforward algorithm which uses the
leading principal minors to determine the deﬁniteness of a given matrix. We
present its proof in the Appendix of this chapter. We will present other criteria for
the deﬁniteness of a symmetric matrix in Section 23.7. . . , l ‘ Theorem 16.1 Lorri be an n X n symmetric matrix. Then. _ (a) A is positive deﬁnite if and only if all its n leading principal minors are
(strictly) positive (:5) A is negative deﬁnite if and only if its n leading principal minors
alternate in Sign as follows: The kth order leading, principal minor should have the same sign as t
(w if. (c) lf some kth order leading principal minor ofn (or some pair of them) ,
is nonzero but does not fit either of the above two Sign patterns, then A
is indefinite. This case occurs when A has; a negative ﬁrth order leading
principal minor for an even integer k or whenA has a negative kth order
leading principal minor and a positive Eth order leading principal minor
for two distinct odd integers k and 6 . t i i t l lA;[<:t), iAgl>0, [A31<:0, etc. E
l t One way that the leading principal minor test of Theorem loll can fail for a
given symmetric matrix A is that some lending principal minor ot‘A is; zero while
the nonzero ones lit the sign pattern in either a) or b) of Theorem 16.1. When this
occurs, the matrixn is not deﬁnite and it may or it may not he semidctinite. in this
case, to check for semideﬁniteneos, one no longer has the luxury of checking only
the 11 leading principal minors of A¥ but must check the Sign of every principal
minor ofrl, using the test described by the following theorem. {16.2} DEFINITENESS OF QUADRATlC FORMS 383 w» ' Theorem 16.2 Let A be an n X n symmetric matrix. Then, A is positive 1 l i semideﬁmte if and only if every prmmpal minor of A [S 2 (l; A IS negative
1
l
l . semideﬁnite if and iny if every principal minor of odd order is :«S i) and every
1 principal minor of even order is; a 0. Example [6.3 Suppose A is a 4 X 4 symmetric matrix and, as usual, write lAil
for its ith order [ending principal minor. (a)l flA l >1,) lAzl> () lA l.> Ll44l 1> () then Ais pasitive deﬁnite
(and conversely).
(b) If lAll< (l, l13l> 1) Mile: (),lA4l > 1), then A is negative definite 1 (and conversely). . g (e) If lAil > 1}, L421 Z} 0, lAgl m C), lA4l < 0, then A is indeﬁnite because
1 0fA4. : (d) If lAll < 0, L431 <1, {1, LA3l < 0. lA4l < 0, thenA is indeﬁnite because
0ng (and A4). (e) if [All a 0, lAgl< 0 lAglL“) f) l/14l w 0 then A is indeﬁnite because
i Offiz. (f) If lAll :2 O, lAgl 2 1), lfigl :2» 1}, lA4l “>2 O, thenA is not definite. It is; not
1 negative semideﬁnite, but it may be possitive semidefinire. To check fer
poeitive semideﬁniteness one must check all lﬁ principal miners ()f A
net just the fcurl e ariing principal ones If mine {if the principal miners;
, are negative, then A is positive semideiinite. if at leaet one of them is
1 negativezi is indefinite. 1 (3) if lAgl ﬂ 1), lAzl :v 1‘), lAﬂ m U, lA4l :21 0, then A is not deﬁnite? but
it may be positive semidefiniie or negazive Semideiinite. To Liecide, one
5 must again check all 15 of its principai minors. To metivate these two {hecrems and to understand their elgerithms belier, we
Aiil examine them in same detail fer the simplest symmeiric matrices M» diagonal
matrices and men 2 A2 matrices; The Definiteness of Diagonal Matrices The simplest n X ii Symmetric matrices are the diagonal matricee. They else}
currespend to the simplex: quadratic forms since. (11 i} ‘ ‘ ’ U Ii 1 0 (13 ' ' ‘ 0 X} ,,
(xii 1‘3 . ‘ ‘ “ff? ) : : . . : : (6) 0 U . . . an x” ‘3 TI
1* £31.26; + :3ng + '  ' 1* (1,4,: :1, 384 Qtitxtmttttt:roentgtxxt)ot—gtmtrrmanners Wt] at sum otxquures. Clearly, this quadratic form will he positive deﬁnite ifund only if
all the (ifs are positive and negative definite if and only if all the afx til'tl negative.
It will be positive semideﬁnite if and only if all the (ifs are 1: t) and negative
semi lel’inite if and only it' all the (ifs are is t). If there are two (ifs of opposite
signs. this form will he indelinite. Since till the principal suhmzitriees are diagonal matrices, their determinants; W
the principal minornu AFC just produeta of the (ifs. If all the (ifs are positive. then
tlll their produetx are positive and so till the lending principal minors are positive.
lf till the (1,39 are negative (so the form is negative definite). then products of odd
numbera of the {ifs will he negative and products of even numbers of the en‘s will
he positive. This eorrespondx to the alternating signs eondition in b) of Theorem
lti.l and indicates why we should expect xueh an alternating, Sign eondition instead
of am all negative condition in the text for negative detiniteness. it at is zero in (o). the form ennnot he detinite Since it will he zero when
evaluated at ( l, t), . . . , 0). Notice that in this diagonal ease. all the leading principal
minors of ((3) will also he. zero. independent of the Signs of the other afs. In order
to cheek that all the (ifs have the proper Sign when mime ol’ them are zero. one
must cheek much more than just the leading principal minors. The Definiteness of 2 x 2 Matrices One can verify Theorems 16.1 Lind 16.3 direetly for 3 X1 3 xvinmetrie mutriees by completing the square in the corresponding quadratic form. (‘onsider the general
. t 3 quadratic lorm on R“: (7) Ext; 4 at; * Zing: t” (it . For ease of notation. we are using, a. it. and c“ in this example in plztee of a”,
an. and (133. respectively. It a 4: t), then Q cannot he positive or negative definite
heeuuse Q( 1,0) '4 0. So, assume now that a :r': (l and complete the Square in (7)
hy adding and subtracting [Rig/a in expres‘sion (”3): t I)“ t, I)" t ﬂights) V at; 3' ﬂung t” (it: t» »»»»» ~x: t:
“‘ a ‘“ a “
j , t
t J) Ir 3 I)“ t t
M u t: L "'"Xtt; r ”J7 w: * (Lt:
' (I it” “ (I W " it 0.2} titérixi'rEM—ss ()F QtantWMtr mews 385 ll' hath coefficients, a and (ark 1):)/u in (8‘) are positive. then Q will never he
negative. It will equal xem only when that ist when .t'; Z (l and X3 *4 (l. in other words. it” i“ l ll Zliltl tlettt :: (I f)
i),
h f’ i then Q is; positive definite. (‘tinversely in order for Q to he punitive delinitei we
need both a and iiet .sl 2 my I33 to he positive. Similarly. Q will be negative deﬁnite if and only it lmth coefficients in expres
sion (8). a and (av _, fill/at are negative. This situation occurs il‘antl only it}: (l
and m' [)3 ii» (1. that is. when the. leading principal minors alternate in sign. If (ur ~ [23). the SCCOIlLl order principal minor. is negative, then the twti well
lieients in (8) have oppnxite signs. in particular. will have opposite signs; so Q is indefinite. limmpi’y [(14 Consider M
”w id
x} J}
[J
4..
V“ ‘ J and I} '~ (4 :7 _ Since Mg! ,; 3 and litlgi "~‘—’ 5. A is punitive definite. Since flit! ‘ 3 and
ilfgi '' 3.1% is indefinite. [Stump/e 1'65 Consider n n
if iiiiii (ll it) : Note that f6}! ' (l and lC‘ﬂ :: (l. "l‘he delinitenegs til (‘ depends cnmpletely
an t‘: (‘ is; positive semidetinite if r z: t) and negative Seinidetinite it c‘ 1; ti.
: This is especially (inviting if tine hmks zit the correspmitling quadratic form f Qr‘UwVZ) i: “'5' :i
9’;
i
i
ii
§ 386 QUADRATIC FORMS AND DEFINITE MATRICES [161 EXERCISES 16.1 Determine the deﬁniteness of the following symmetric matrices: a) (:3 1) (1+: .2) a (“i :g) c» (i 1:) 120 ~i l 0 $332 6) 245 n iwi n g) "“ ‘
n 5 6 0 o «a 3 0 4 U “ 0506 16.2 Let Q(x) z“— );fo be a quadratic form on R”. By evaluating Q on each of the
coordinate axes in R“, prove that a necessary condition for a symmetric matrix to
be positive deﬁnite (positive semideﬁnite) is that all the diagonal entries be positive
(nonnegative). State and prove the corresponding result for negative and negative
semideﬁnite matrices. Give an example to show that this necessary condition is not
sufﬁcient. 16.3 Using the method of the previous exercise, sketch a proof that if A is positive (or
negative) deﬁnite, then every principal submatrix of A is also positive (or negative)
deﬁnite. 16.4 How many kth order principal minors will an n X :1 matrix have for each k 3 n? 16.5 Mimic the computation in (8) to prove Theorem 16.1 for a general symmetric 3 X 3
matrix. {Hint After “completing the square” twice, you should find that at: 61:2 323 X1
(X; x»; 963) (112 (133 {123 X2 3
{In (133 (133 Vng ) (it: 6313 “ IflzI ( ' $121623 *" Cheats )2 Hal , a
A , ‘I’ w, “t“ W, ‘9' , “if W] i”  ,, ",
1 I} (I) at: 1:2 an E3) IAiI r3 If‘lzI ‘3 A {it} 1 16.3 LINEAR CONSTRAINTS AND BORDERED MATRICES Definiteness and Optimality Keep in mind the fact that determining the deﬁniteness of a quadratic form Q
is equivalent to determining whether x a 0 is a max, 3 min, or neither for the
realvalued function Q. For example, x m 0 is the unique global minimum of
quadratic form Q if and only if Q is positive deﬁnite, by the very deﬁnition of
positive deﬁniteness. Similarly, x z 0 is the unique global maximum of Q if and
only if Q is negative deﬁnite. The characterization of deﬁniteness in Theorem 16.1 works only if there are
no constraints in the problem under consideration, that is, if x can take on any
value in R“. If there are constraints, the analysis becomes more delicate. H63} LINEAR CONSTRAINTS AND BORDERED MATRlCES 387 Example 16.6 The quadratic form Q(x1, x2) = x33  3633 on R2 is indeﬁnite; the l origin is neither a max nor a min. But, if we restrict our attention to the X] axis,
i that is, if we impose the constraint x2 = t), then Q(x1, 0) = t}: has a strict
global minimum 21th :1“ Or and therefore Q is positive deﬁnite on the constraint
‘ set {x2 2 0}. Alternatively, if we impose the constraint x1 x (f) and consider
Q only on the .rg—axis, then 3:3 3 t) is a global max of (2(0, .153) == ~x§ and
Q is negative deﬁnite on the subspace {x1 x 0}. On the line x1 w 2x2 :2 0,
one, x3) = (2ng ~ 1% a 3x§ is positive deﬁnite. As will be shown in Chapter 19, the second order condition which distinguishes
maxima from minima in a constrained optimization problem is a condition on the
deﬁniteness of a quadratic form which is restricted to a linear subspace. Since
most optimization problems in economics involve constraints on the variables
under study, the rest of this chapter will discuss the deﬁniteness of quadratic forms
which are restricted to linear subspaces of R“. Let us look in detail at the simplest such problem: the problem of determining
the definiteness of, or of optimizing? a general quadratic form of two variabies: Qﬂrb x2) e“: we}: + 2bx1x2 + at“; x (x1 x3 ) (Z S) (:1), (9)
~ , 2 on the general linear subspace
31,351 ”i“ 8X2 :5" 0 (10) in Example 16.6, we worked within : 0, then with B r: 0, and ﬁnaily withxt 3 i
nndB 3‘“ ”“2. Since our focus is on the matrix and not on the quadratic form itself, we have
muitipiied the coefﬁcient of xix; in (9) by 2 so that we do not have to deal with
1/23 in the corresponding matrix. The simpiest approach to this probiem is to
solve (’10) for x; in terms of ac}; obtain x1 m wag/A, and then substitute this
expression for x; in the objective function (9): B, 8
details) (:32 r 253 i t (11) i183 ' 2lan 4* (:42
{L12 3
x: . We concinde from (it) that Q is positive deﬁnite on the constraint set (‘10) if
and only if {182 ~ 2MB + CA2 > 0 and negative deﬁnite on (10) if and only if
6282 w ZbAB + at: < (i. There is a convenient way of writing this expression: 388 QUADRATlC FORMS AND DEFINITE MATRICES ltnl , (in B
(ml—3m3+wf:~nm A a b. on
B I) c: The matrix in (12) is obtained by “bordering“ the 2 X 2 matrix (9) of the quadratic
Q on the top and left by the coefficients of the linear constraint (' 10). The following
theorem summarizes these calculations. vaww MMMMMMMMMMMMMMMMMMMMM e
E Theorem 16. 3 The quadratietoertt‘E, t3) : urf 4:» 2hr; r; H t: ispositiVe t
E (respectively negatiVeNehniteonthe eonstruintset it rlit;  {litundoan it
t
E ()A n E
E dct A a h i
E B b c'
E is negative (respectively, positive). E
l This same result holds for the general problem of determining the tletiriite~ ness of “it (It: ulrr Xi»
,r ”l: “3: ‘ ‘ ‘ ”3:: x:
l W  k E E
MUHXAX”UE'H u) . . . . . on
“in “3!: i ' ' ”mt «XI!
on the linear constraint set
t (l
Bl] Bi: Hln ’
X: (l
. . . i V r U“
”ml Bml‘ ' h ' Bum E)
4,; , Border the matrix (l3) of the quadratic form on the top and on the left hy the
matrix ( 14) of the linear constraints: 0 n t m; BM
E :
U 0 i Hm! Bum
f] ; MWW ................................................. , (Eg)
1;” [grill i £11] ”In WW,WWEWM;»%WMMé E; r; w mmmm ,l t vr E E .37; ,E .455“: E .r , ,, , t_, , V, ,, , , t ,, ‘ H643} UNEARCONSTRNNTSANDBORDERED MATRKZES 389 We first need to ﬁgure out which submatrices of H to connider. In studying the
quadratic (0) on the constraint set (Ill). we had only one condition to check in
'l‘heorem 10.3, because our Single constraint in R2 resulted in a onedimensional
problem. The above problem (l3. 14) has m linear equations of II variables. We
therefore expect that this problem is really i? e m dimensional and therefore that
we will have n M in conditions to check for the matrix H in (15). Furthermore, from
our experience in Section 10.2. we expect that we will look for leading principal
minors of the more sign to check for positive deﬁniteness and for lending principal
minors of alternating signs to check for negative deliniteness. The following
theorem indicates that these expectutions are correct. and it makex precise the
exact Sign patterns that we need to verify. ., M, MM.MHM.WM..M.H_,“”’ ~m_wmmlc Aercw WM ._”Heme,_,._#w._kw.~myewwm~ w._we._»w_awmw,
l Theorem 16.4 To determine the detinitencsn of ti quadratic form ([3) of n :
\rzii‘itiblest Q(x) .2 xiiilx. when restricted to it constraint set (14) given by in Q
linear equations; Bx 2: 0‘ construct the (It 4; m) X (it +~ m) symmetric matrix i
If by bordering, the matrix i zihove and to the left by the coefficients B of the g
E linear constraints: l
i M 0 I;
H'in" t) (“heck the signs ol~ the last n 'M m leading principal minors of”. starting with
the determinant ol‘H itself. (a) If tlet I! has the same sign iifi (W l)” and it these lain n M in leading
principal minors allcmarc in Sign, then Q is; litigator definite on the
constraint set BX t: l). and x :3 0 in a atrict glottal max of Q on this
constraint set. ([9) lt'det H and thene last n M in lending principal minors all have the same
sign {ES ( r» l )"i then Q is positive definite on the constraint set B): 3 0.
and x I: 0 is a strict global min ol‘Q on this constraint set. (if) lf both of thexe conditions a) and hi) tire violated by nonzero leading
principal minors. then Q is indefinite on the constraint set 133: r: 0, and
x 11: 0 is neither a max not a min of Q on thin constraint set. i ,,,,,,,,,,,,,,,,,,,,,,,, M, .._., wwwmﬂc i, W. _ “We “Mariam“ __W_..,.,H...,.....,.c.,e..w..._,._ a, ..~.cw.../,,.,M_.~c.cwwwm muwwwtﬁrm, WNW} We will not present the rather intricate prool’ol‘ this theorem here. Notice that
its conclusions are consiatcnt with the conclusions of Theorem lid for it 1: Z and
m i: l. where for the curse of two variables and one constraint. we only needed to
compute n e m z: 2 l :1 l determinant. Exercise ltii‘) lookS :it some important
special cases of ”l‘heorcm 16.4. [hump/t2 [(3.7 To check the tlelinitencss ol 7 ‘i Qtrg. r; X}, .t‘.;) 4: .rE ~ .t‘g +~ r; «P t «~24 t Mgr; ltirt 390 QUADRA’TIC FORMS AND DEHNITE MArRicrs It 6} on the constraint set X3 ‘i’ X}, *7 X4 : (l. XI '4 9x: + X4 3 (l. form the bordered matrix ll ll 1 l l} t) l
It,h : o 1 I l o o ~l
1 ~‘) I (l ,, l 3 (l
1 ll l ll 2 l (l
l l l """ l (l (l l
Since this problem has n i: 4 variables and m 4: 2 constraints. we need to check the largest n  m r 2 leading, principal submzitrices oil/(t: Hi, itself and o o l u t 1
t} t) l t A) o “>3: 0 l t l o o
I Mi) 1 n ~i 2
i u l o 2 l : Since m 2: 2 and (A l )2 2 +1. we need detHh (l and detH; t) to verity
‘ positive detiniteness. Since n I 4 and ( l)” I: ”s— l. we need detH’h (l and
i detH; < ()to verify negative detiniteness. ln fuchdetllh ,5 Munddetlk ; 77: so Q its positive definite on the constraint set, and x z 0 in it min on restricted
3 to the constraint set. Remark If the test for constrained deﬁniteness‘ of "l‘heorem 10.4 fails only he
eause one or more of the last I: A m leading principal minors; is zero. then we
would like a test for semideﬁniteness. analogous to the statement of Theorem In}.
for the unconstrained problem. Unfortunately, tests for constrained semidetinite~
ness are much more tedious to state than the criteria described in Theorem 16.2.
Fortunately, such tests are rarely required in applications. One Constraint Constrained maximization problems with just one effective conx‘trtiint are common
in economic theory. For the problem of checking the detiniteness ot' n quadratic
Q subject to a single constraint/11.x“, L ... l~ rink); ll, ”l‘heoreni 16.4 states that
one needs to check the lust n w 1 leading principal minors of ill A; A” A} it” ~ :1“, [Irv”l r" . . ‘ .  (l0) An all! ' ' ' ”in: [16.31 UNEAR CONSTRAINTS AND BORDERED MATRICES 391 The only omitted leading principal submatrices are: H, 2 (II) and H; z ( U "“l ) fill {11}
Let‘s suppose that A; i 0. (One of the Afs must be nonzero!) Thent detHg 2
“Ai < 0. Since m x l and ( l)I r: _ l, the criterion for constrained positive
deﬁniteness is that the last it A 1 leading principal minors of (16) are negative.
Since detHg < i). this criterion is equivalent to the statement that the last it lead
ing principal minors have the same sign. The criterion for constrained negative
deﬁniteness is that det H,, i; have the sign of(* l )” and that detHjt, . . . , detHM;
alternate in sign. This means, in this case. that detH} must be positive. It follows
that the condition for constrained negative detiniteness is equivalent to the condi‘
tion that the last 11 leading principal minors 0an t , alternate in sign. The following
theorem summarizes this discussion for m : l and yields an easierto~remember
approach for the problem of determining deﬁniteness when there is only one linear
constraint. r“ l Theorem 16.5 To determine the deﬁniteness of a quadratic Q(x1,...,.r,,)
l subject to one linear constraint, form the usual (/1 + l) X (n + l) bordered
l matrix H, as in ( 16). Suppose thatA; $ t). lfthe last 11 leading principal minors
l of 11,, it have the same sign, Q is positive deﬁnite on the constraint set (and
l x 3 0 is a constrained min on). lfthe last it leading principal minors off!" ii
: alternate in sign, Q is negative definite on the constraint set (and x r: 0 is a
l constrained max on). l
' wWWWWWﬂWWMMMAMWWW Other Approaches For the sake of completeness. we mention two alternative approaches to the
problem of determining the deﬁniteness of a quadratic form of n variables subject
to m linear equations. "l‘he statement of Theorem 16.4 focuses on the sign of the
largest submatrix H,,, .1; as the cornerstone of the algorithm. Some presentations
locus instead on the smallest of the last 12 ~ or leading principal submatrices:
[13,". I. the (2m + llth order leading principal suhrnatrix. Theorem loA implies
the following alternative checks: (A) To verify positive delinitencssi check that det [13mg 1 has the same sign as
( ~ l)"’ and that all the larger leading principal minors have this sign too. (8) To verify negative detiniteness, check that det H3,” t i has the sign of
(61)” ' and that the leading principal minors of larger order alternate in
sign. Some texts prefer to construct the bordered matrix H by bordering the matrix
A of the quadratic form Q(x) E x’Ax below and to the rig/t! by the matrix B for 392 QUADRATIC FORMS AND DEFINITE MATRICES [161 the linear equations Bx : 0: M A B?”
Hm+rt“"(8 0) In this situation, one must still Check n  m principal minors. However, the
corresponding principal submatrices are no longer leading ones, but “border
preserving” ones. One removes from HM”, one at a time, the n  m « 1 rows and
columns which contain the last it w m  1 rows and columns of the matrix A, that
is, rows and columns n, 21 ~ 1, . . . , m + 2 of HmM. Example 16.8 To use this approach for the problem in Example 16.7, form the
bordered matrix l
t
l 100%].
i 2 i 0 1 0 wt 0 I i «“9 A 0 2 l 0 t i 0
He —~1 o 0 1 t t 1 and then form the submatrix H5 by removing row 4 and column 4 from H. the
row and column just before the border of H: 1 u 0 I o t
o r t t «9
F15 : 0 :2 t l 0 ~— h.) 0 l l l l) (l
,1 “=9 (l l (l 0 Note that detH a 24 and detlig a 77, just as we found for the corresponding
% minors in Example 16.7. EXERCISES 16.6 Determine the deﬁniteness of the following constrained quadratics: a) Q(x1, 1:3) == xi + lax2 M 9:33, subject to x, + x; Z 0. b) Q(x;, xz) = 43:? + 2x133 ~ x53, subject to x, + x3 x (l. c) Q(x1, x3,x3) 3 x12 + x; w Vi + 4x,x3  21mg, subject to xi 4: x3 + x; m l) and
x1 + 3:3 ‘ .r;; t 0. d) Q(x1,x3, x3) 3 ref + x: + xi + 4hr; « lxlxz, subject to x; + £3 + x3 x (l and
x1 + x; w x3 2: 0. e) {206}, x3, x3) 3 x? w x3“ + skew;  6.363363, subject to x; + x: w x; 'I: (l. {16.4} APPENDIX 393 16.7 Prove that statements A and 8 above are equivalent to the statement of Theorem
16.4. 16.8 Use the theory of determinants to show why the corresponding minors in Examples
16.7 and 16.8 have the same values. 16.9 Use the techniques of Theorem 16.3 to verify Theorem 16.4 for the general problem
with: a) three variables and one constraint, b} three variables and two constraints. 16.4 APPENDIX This section presents the proof of Theorem 16.1. This proof has two major ingredi
ents: the principle of induction and the theory of partitioned matrices as developed
in Section 8.7. We will prove Theorem 16.1 for positive deﬁnite matrices and leave
the proof for negative deﬁnite matrices as an exercise. First, we need two simple
lemmas. lemma 16.1 if A is a positive or negative deﬁnite matrix, then A is nonsingular. Proof Suppose that such an A is singular. Then, there exists a nonzero vector x
such thatAx m 0. But then, j xfiAxmxT‘GEO, s , a contradiction to the deﬁniteness of A. i Lemma 16.2 Suppose that A is a symmetric matrix and that Q is a nonsingular
matrix. Then, Q1r AQ is a symmetric matrix, and A is positive (negative) deﬁnite if
and only if QTAQ is positive (negative) deﬁnite. Proof To see that QTAQ is symmetric, one checks directly that it equals its own
transpose: {QTAQV e drifter)?” e 911470 e Q’eQ. % Suppose that QTAQ is positive deﬁnite. Let x e f) be an arbitrary nonzero
earn in R“. Since Q is nonsingular, there exists a nonzero vector 3‘ such that
m Qy. Then xrex 3 (lereer) e rTQTAQy e ritQTAQli/z which is positive, since QTAQ is positive deﬁnite. Therefore, A is positive
deﬁnite. On the other hand, if A is positive deﬁnite and z is an arbitrary nonzero
vector, then Q2 will he nonzero too, since Q is nonsingular. Therefore, 0 < thifet’in e foil/4Q: e zrtQTAQﬂ, and QTAQ is positive deﬁnite. I 394 QUADRATIC FORMS AND DEFlNlTE MATRlCES E16] and only if all its leading principal minors are positive. E J l
E Theorem 16.1 Let A be a symmetric matrix. Then, A is positive deﬁnite if E
l
l Proof We will prove this result by using induction on the size n ofA. The result
E is trivially true for l X 1 matrices. We proved it for 2 X 2 matrices in Section
, 16.2. We suppose that the theorem is true for n X n matrices and prove it true
E for (n + l) X (n + 1) matrices.
Let A be an (n + l) X (n + l) symmetric matrix. Write Aj for the j X j
E leading principal submatrix ofA for j = ,, . . ., n + l.
E We ﬁrst prove that if all the Ali’s have positive determinants, then A is
E positive deﬁnite. The leading principal submatrices off/Lln are/ill, . . . , A”, which
are positive deﬁnite by hypothesis, since they are the ﬁrst 11 leading principal
submatrices of A. By the inductive hypothesis that the theorem is true for n X n
matrices, the n X a symmetric matrix A,2 is positive definite. By Lemma 16.1
above, A” is invertible. Partition A as A (An I a ) , 7
, x , , (E )
3f i an+l,n+l where :1 denotes the n X 1 column matrix al,n+l answer} E Let d 3 {install  aT(A,,)“Ea, let l,t denote the n "X :2 identity matrix, and let
0,, denote the n X 1 column matrix of all Os. Then, the matrix/t in (17) can be
‘ written as l l a~( 1,, t (LEM/ts l taxi" ! Alia E ‘ hiya)?” t l a}: l d of l '1 ) (18)
l E a QTBQ.
(Exercise) By properties of the determinant, detQ z (16th x 1 and detB 2: a « item”.
Therefore, detA r: d * detain. (19) Since detA > 0 and Clem,2 > 0, then d > 0.
Let X be an arbitrary (n + l)—vector. Write X as {16.4} APPENDBX 395 i where x is an nvector. Then, XTBXXCKT rim) ~—,~——————~ X
f r
. 0 l d ”1+1 (20) H ; Since A... is positive deﬁnite by inductive hypothesis and d > if), this last
. expression is strictly positive. Therefore. 8 : QTAQ is positive deﬁnite. By
Lemma 16.2,.4 is positive deﬁnite. To prove the converse “A positive deﬁnite implies that all the kid’s are
positive — we use induction once more. We have seen that this result is true for
l, X l and 2 X 2 matrices. Assume that it is true for n X n symmetric matrices.
and letA be an (n + I) X (n + 1) positive deﬁnite symmetric matrix. We ﬁrst
show that all the Aj’s are positive deﬁnite. Let x; be a nonzero j—veetor, and let i 0* be the zero (n + 1)  jvector. Then
l l l i 7 g = X Ans + dxgﬂ.
i l i l 5 xi 0<(x{ 0*)A(0. .. T g.
)~ arrAfar), and A, is positive deﬁnite. In particular, since A” is positive deﬁnite, the inductive hypothesis tells us
that A1,...,A,, all have positive determinants. We need only prove that the
determinant of A itself is positive. Since A... is invertible, we can once again
writesi as QTBQ as in (18) and conclude that (19) still holds. Since A is positive
deﬁnite, B is positive definite by Lemma 16.2. Choose X in (20) so that x == 0
and in,“ m 1,.Then, i
i
5
i
l
i
i
i
i
l
g
t
i 0 < X173}: 3 d. i Since data. > o and d > o, detA > o. I EXERCISES 16.10 Show that the block decomposition (18) is correct.
16.11 Prove the corresponding theorem for negative deﬁnite matrices. ...
View
Full Document
 Spring '10
 fallahi

Click to edit the document details