6. Example Suppose that we have a production function Q = kxa yb . Then,
Q
= akxa1 yb ,
x
2Q
= abkxa1 yb1 ,
x y
Q
= bkxa yb1
y
2Q
= abkxa1 yb1
y x
14 Some Linear Algebra
14.1 Deniteness of Quadratic Forms (16.2)
Denition 27. Let A be an m m symmetric matrix, then A is
(a) positive denite if xT A x > 0 for all x = 0 in Rm ,
(b) positive semidenite if xT A x 0 for all x in Rm ,
(c) negative denite if xT A x < 0 for all x = 0 in Rm ,
(d) negative semidenite if xT A x 0 for all x in Rm,
(e) indenite if xT A x > 0 for some x in Rm and < 0 for some other x in Rm .
Q(x, y) = 3x2 6y2 0 and strictly negative except at 0. Hence, it is negative
denite, and (x, y) = 0 is its global maximizer. A =?
Q3 (x, y) = 3x2 6y2 is indenite. A =?
Q4 (x, y) = x2 + 2xy + y2 is positive semidenite. A =?
Q5 (x, y) = (x y)2 is negative semidenite. A =?
14.2 Symmetric Matrices (23.7)
Denition 28 (p.579, 582). Let A be a m m matrix, v = 0 be a m-vector and r be a
number. If Av = rv, then v is called an eigenvector (or characteristic vector) of A, and
r is called an eigenvalue (or characteristic value) of A.
Example 7. Consider A =
Av =
2
2
24
42
= 2v, and Aw =
,v =
6
6
1
1
,w =
1
1
. Then,
= 6w. Hence, the eigenvalues of A are -2
and 6.
Theorem 44 (Thm 23.16, p.621). Let A be a symmetric m m matrix. Then there is a
nonsingular matrix P whose columns w1 , , wm are eigenvectors of A such that
(i) w1 , , wm are mutually orthogonal to each other
40
(ii) P1 = PT , and
(iii)
P1 AP
= PT AP
=
r1 0 0
0 r2 0
. . ..
.
..
..
..
.
0 0 rm
If we write the diagonal matrix as D, then we have A = PDPT .
Hence, xT Ax = xT PDPT x = yT Dy, where y = PT x.
A is positive(negative) denite D is positive(negative) denite.
Denition 29 (p.165). Let A be an m m matrix. A is said to be nonsingular if there
exists an m m matrix A1 , called the inverse of A, such that
A1 A = AA1 = I
Theorem 45 (Thm 8.5, p.165). If A has an inverse, it is unique.
Theorem 46 (Thm 23.16, p.621). Let A be a symmetric m m matrix. Then there
exists a nonsingular matrix P such that
(i) P1 = PT , and
(ii) P1 AP = PT AP =
r1 0 0
0 r2 0
. . ..
.
..
..
..
.
0 0 rm
If we write the diagonal matrix as D, then we have A = PDPT .
Hence, xT Ax = xT PDPT x = yT Dy, where y = PT x.
A is positive(negative) denite D is positive(negative) denite.
15 Unconstrained Optimization 2 (ch. 17)
15.1 Second Order Conditions
Sufcient Conditions
Theorem 47 (Thm 17.2, p.399). Let F : U R1 be a C2 function dened on an open
set U Rm . Suppose that DF (x ) = 0. (Such x is called a critical point of F.)
(1) If D2 F (x ) is negative denite, then x is a strict local max of F;
(2) If D2 F (x ) is positive denite, then x is a strict local min of F;
41
(3) If D2 F (x ) is indenite, then x is neither a local max nor a local min of F.
Denition 30 (p.399). A critical point x for which D2 F (x ) is indenite is called a
saddle point of F.
Q3 (x, y) = 3x2 6y2 is indenite. (0, 0) is a saddle point.
15.2 Proof of the Sufciency
Proof. When DF (x ) = 0, we have
F (x + h) = F (x ) +
where
R2 (h)
h
2
1 T2
h D F (x )h + 2R2 (h)
2
0 as h 0.
There exists a nonsingular matrix P such that P1 = PT and D2 F (x ) = PDPT ,
where D is a diagonal matrix with all negative (positive) diagonal elements.
Since
hT h
hT I h
R2 (h)I
R2 (h) = R2 (h)
= R2 (h)
= hT
h,
2
2
h
h
h2
we have
hT D2 F (x )h + 2R2 (h) = hT D2 F (x )h + 2R2 (h)
2R2 (h)I T
= hT PDPT h + hT PPT
PP h
h2
= hT P D +
= yT D +
2R2 (h)PT IP
h
2R2 (h)I
h
2
2
PT h
y < (>)0
(See the Laws of Matrix Algebra in the note 0923p)
16 More Linear Algebra (ch.11 and Rudin ch.9)
16.1 Determinant of a Matrix (Rudin, ch.9)
Denition 31 (p.240). Vectors v1 ,v2 , , vk Rm are linearly dependent if there exists
c1 , c2 , , ck R, not all zero, such that c1 v1 + c2 v2 + + ck vk = 0. They are linearly
independent if c1 v1 + c2 v2 + + ck vk = 0, c1 , c2 , , ck R implies that c1 = c2 =
= ck = 0.
42
Denition 32 (Rudin p.232). For an ordered m-tuple of integers ( j1 , j2 , , jm), dene
s( j1 , j2 , , jm) = sgn( j p jq ),
p >q
where sgn x = 1 if x > 0, = 1 if x < 0, = 0 if x = 0.
Example 8. s(1, 3, 2) = sgn[(2 1)(2 3)(3 1)] = 1
s(1, 3, 2, 4) = sgn[(4 1)(4 3)(4 2)]s(1, 3, 2) = 1
s( j1, , jh, jh+1 , , jm) = s( j1, , jh+1, jh , , jm)
s( j1, , jh, , jk , , jm) = s( j1 , , jk, , jh , , jm)
Denition 33 (Rudin p.232). For an m m matrix A = (ai j ), dene
det A = s( j1, j2 , , jm)a1 j1 a2 j2 am jm ,
where the sum is taken over all ordered m-tuple of integers ( j1 , j2 , , jm) with 1
j r m.
Example 9.
det
a11 a12
a21 a22
= s(1, 1)a11a21 + s(1, 2)a11a22 + s(2, 1)a12a21 + s(2, 2)a12a22
= a11 a22 a12 a21
j h = j k s( j 1 , j 2 , , j n ) = 0.
Hence, the sum contains n! (the number of ways to order n different integers)
non-zero terms.
The determinant is the sum of all the multiplications of one element in each row
and each column multiplied by the signs dened by s( j1, j2 , , jn).
Example 10.
a11 a12 a13
det a21 a22 a23
a31 a32 a33
= s(1, 2, 3)a11a22 a33 + s(1, 3, 2)a11a23 a32 + s(2, 1, 3)a12a21 a33 + s(2, 3, 1)a12a23 a31
+ s(3, 1, 2)a13a21 a32 + s(3, 2, 1)a13a22 a31
= a11 a22 a33 a11 a23 a32 a12 a21 a33 + a13 a22 a31 + a13 a21 a32 a13 a22 a31
Lemma 3. If A is a matrix obtained from A by switching two columns, then det A =
det A.
43
Proof. Suppose that A is obtained by switching h-th and k-th columns. Then
det A = s( j1, , h, , k, , jn )a1 j1 aih h aik k an jn
= s( j1, , h, , k, , jn )a1 j1 aih k aik h an jn
= s( j1, , k, , h, , jn )a1 j1 aih k aik h an jn
= s( j1 , , k, , h, , jn )a1 j1 aih k aik h an jn
= det A
Lemma 4. If two columns of A are identical, then det A = 0.
Proof. det A = det A = det A. Hence, det A = 0.
Theorem 48. det (a1 , , am) is linear a function of each column vector a j . That is
det (a1 , , ka j + hb j , , am ) = k det (a1 , , a j , , am ) + h det (a1 , , b j , , am)
Proof.
det (a1 , , ka j + hb j , , an )
= s( j1 , , j, , jn )a1 j1 ai1 ji1 (kai j + hbi j )ai+1 ji+1 an jn
= s( j1 , , j, , jn )a1 j1 ai1 ji1 (kai j )ai+1 ji+1 an jn
+ s( j1 , , j, , jn )a1 j1 ai1 ji1 (hbi j )ai+1 ji+1 an jn
= k det (a1 , , a j , , an ) + h det (a1 , , b j , , an )
Corollary 3. If A is obtained from A by adding a column a j of matrix A multiplied by
some number r to another column ah , then det A = det A. That is
det (a1 , , ah + ra j , , am ) = det (a1 , , ah , , am)
Proposition 1. Let A = (a1 , a2 , , am). If a1 , a2 , , am are linearly dependent, then
det A = 0.
Proof. If a1 , a2 , , am are linearly dependent, then there exist c1 , , cm , with ch =
0 for some h, such that j c j a j = 0. Then, one can write as
cj
aj
j=h ch
ah =
c
Hence, det A = det (a1 , , ah , , am) = det a1 , , j=h chj a j , , am = 0
Theorem 49 (Rudin Thm 9.35, p.233). If A and B are m m matrices, then det AB =
det A det B.
We do not prove this theorem. The proof is in Rudin.
44
Theorem 50 (Rudin Thm 9.36, p.233). An m m matrix A is invertible if and only if
det A = 0.
Proof. A is invertible there is A1 such that A1 A = I det A1 det A = det I = 1.
Hence det A = 0. This proves the only if part.
The if part will be proved later.
16.2 Basis
Denition 34 (Rudin pp.206-207). Given vectors v1 , , vk Rm and scalars c1 , , ck
R, the vector c1 v1 + + ck vk is called a linear combination of v1 , , vk .
A nonempty set X Rm is a linear space if v1 , , vk X implies that any linear
combination of v1 , , vk belongs to X
If S Rm and if E is the set of all linear combinations of elements of S, we say that
S spans E, or that E is the span of S.
If a linear space X contains an independent set of r vectors but contains no independent set of r + 1 vectors, we say that X has dimension r, and write dim X = r. The
set consisting of 0 is a linear space; its dimension is 0.
An independent subset of a linear space X which spans X is called a basis of X.
Proposition 2. If B = {x1 , , xr } is a basis of X, then every x X has a unique
representation of the form x = ci xi
Proof. Since B spans X , any x X can be written as a linear combination of xi s,
x = ci xi .
(2)
x = ci xi .
(3)
Suppose it is also written as
Subtracting equation (3) from equation (2), one obtains
0 = (ci ci )xi.
Since xi s are linearly independent, this implies ci = ci for all i.
Theorem 51. If a linear space X is spanned by a set of r vectors, then dim X r.
Proof. According to the denition of the dimension it is enough to show that any set
of r + 1 vectors in X is linearly independent. Suppose that {y1 , , yr+1 } is linearly
independent. Let {x1 , , xr } span X . Then y1 is written as a linear combination
of xi s, y1 = ci xi . Some ci is not 0, since y1 = 0. Without loss of generality lets
assume that c1 = 0. Then x1 can be written as a linear combination of y1 , x2 , , xr .
Hence, S1 {y1 , x2 , , xr } spans X . Then y2 is written as a linear combination of
vectors in S1 , y2 = b1 y1 + r=2 ci xi . Some ci is not 0, because ci = 0, i = 2, , r
i
implies y2 = b1 y1 which means {y1 , , yr+1} is linearly dependent. Without loss
of generality we can assume that c2 = 0. Then, we can replace x2 by y2 in S1 and
span X by S2 {y1 , y2 , x3 , , xr }. We can continue this process and show that Sr
{y1 , , yr } spans X . But then yr+1 can be written as a linear combination of y1 , , yr ,
a contradiction.
45
Corollary 4. dim Rm = m.
Proof. Consider e1 , , em . They are linearly independent. Hence, dim Rm m. They
also span Rm . Hence, dim Rm m.
Theorem 52. Suppose X is a linear space, and dim X = m.
a) A set E of m vectors in X spans X if and only if E is independent.
b) X has a basis and every basis consists of m vectors.
Proof. a) only if part. Suppose that E spans X and that E is dependent. Then one of
the vectors in E is written as a linear combination of the others, hence one can delete
that vector and the remaining vectors can still span X , which implies dim X < m, a
contraction.
if part. Consider x X . The set E {x} is linearly dependent according to the
denition of the dimension. That is there are scalars c0 , c1 , , cm not all equal to 0
such that c0 x + m 1 ci xi = 0, where xi E . We must have c0 = 0, otherwise E is
i=
ci
dependent. Hence x = c0 xi . Since x is chosen arbitrarily, E spans X .
b) According to the denition of dimension there is an independent set E of m
vectors in X . By a) E spans X , hence it is a basis of X . A basis cannot contain more
than m vectors because it would not be independent. A basis cannot contain less than
m vectors, because dim X would be less than m according to Theorem 51
Given a vector a Rm , the function f (x) = a x is a linear function from Rm to
R.
Given a k m matrix A = (a1 , a2 , , am ), F (x) Ax is a linear function from
Rm into Rk ,
A(rx + sy) = rAx + sAy.
Denition 35 (p.297). A function F : X Y is said to be one-to-one (or injective)
if x, y X and x = y implies F (x) = F (y). We say F is onto (or surjective) if F (X )
{y| y = F (x) for some x X } = Y .
Question: When is an m m matrix A = (a1 , a2 , , am) onto as a linear function?
Answer: When the column vectors a1 , a2 , , am are linearly independent.
Reason: The set {y Rm | y = Ax = x1 a1 + + xm am, x Rm } spans Rm if and
only if a1 , a2 , , am are linearly independent.
Question: When is an m m matrix A = (a1 , a2 , , am ) one-to-one?
Answer: When the column vectors a1 , a2 , , am are linearly independent.
Proof? A is one-to-one x = y implies Ax = Ay Ax = Ay implies x = y
Ac = 0 implies c = 0 a1 , a2 , , am are linearly independent
46