SOLUTION III. DEVELOPMENT: PROBLEM 1
This section deals with the solution properties of Rx(t ) = Y (t ) ! T U (t ) via a series of questions and
answers.
QUESTION 5.1: When does there exist a solution?
ANSWER: Whenever the equations are consistent, i.e. whenever [Y (t ) ! T U (t )] " col ! sp( R) or
equivalently whenever rank ([ R, Y ! T * U ]) = rank ( R) .
QUESTION 5.2: If there exists a solution is it ever unique?
ANSWER: Yes, if and only if rank(R) = n. In the SISO case, this equivalently requires that det(R) 0 in
which case
x(t ) = R !1 [Y (t ) ! TU (t )]
QUESTION 5.3: If the solution is unique, how does one compute it for a rectangular matrix R ?
ANSWER: Since R has at least as many rows as columns and rank[R] = n, the unique solution is given by
x(t ) = R ! L [Y (t ) ! TU (t )]
where R ! L is any left inverse of R .
QUESTION 5.4: OK, so what is a left inverse of R? If R is an rn ! n and rank(R) = n, then any matrix
denoted R ! L for which R ! L R = I is a valid left inverse of R . Recall that in general the equations are over
determined and hence one can extract different sets of n-equations which yield the same solution. So one
would expect non-unique left inverses.
DEFINITION 5.2: The Moore-Penrose (pseudo) left inverse of R is
!1
R ! L = " RT R $ RT
#
%
Exercise. Show that this is a valid left inverse of R , assuming RT R is nonsingular, i.e., invertible.
QUESTION 5.5: Why didnt you use the SVD stuff to compute the left inverse?
!1
ANSWER: Great question since the formula R ! L = " RT R $ RT is numerically ill-conditioned, i.e., it is
#
%
prone to round off error and other numerical faux pas. For small hand calculations it sometimes proves
T
adequate. In regards to the SVD, the MP (pseudo) left inverse is given by R ! L = V1S !1U1 . This is easily
(
)(
)
T
seen since if one multiplies the matrices in the SVD for R, we obtain R ! L R = V1S !1U1 U1SV1T = I , that
satisfies the required definition.
QUESTION 5.6: Hey, arent measurements always noisy? So what about perturbations and errors in data
due to noise or bad/approximate differentiation?
ANSWER: Although the (sensor) data is often inaccurate resulting in equations that may not be consistent,
they are typically almost consistent.
QUESTION 5.7: Almost consistent is still inconsistent dude. So there doesnt exist a solution. How are you
going to get around that PROFESSOR?
ANSWER: Not going to get around it. Just going to do the best the math lets me do. How about trying to
get the best approximate answer? That means I have to quantify almost consistent.
QUESTION 5.8: OK Mr. Professor, what does almost consistent mean?
ANSWER: Almost consistent means there exists a vector v ! R rn such that for an appropriately small scalar
v
2
< ! Y (t ) " TU (t ) 2
and
[Y (t ) ! TU (t )] + v !"! col ! sp[ R]
Loosely speaking, a small change in the data vector leads to consistent equations.
QUESTION 5.9: How does one go about approximating the answer to inconsistent equations?
ANSWER: Well daahhh, a least squares solution:
min
x (t )!R n
[Y (t ) " TU (t )] " Rx(t ) 2
QUESTION 5.10: So are we supposed to do this kind of calculation by hand?
ANSWER: No, let MATLAB do the crunching:
(i)
x = pinv(R)*(Y - T*U) where pinv(R) is MATLABs computation of use the MoorePenrose pseudo inverse of R yields a least squares solution and the correct solution if the
equations are consistent;
(ii)
a more convenient way to compute x when R has full column rank (and thus a valid left
inverse) is:
x = R\(Y - T*U)
EXAMPLE 5.6: In MATLAB.
Part 1: Inconsistent equations.
>> R = [1 0;1 0;1 0;1 0]
R=
1
0
1
0
1
0
1
0
>> b = [1 0 1 1]';
>> x =R\b
Warning: Rank deficient, rank = 1, tol = 1.7764e-15.
x=
0.750000000000000
0
Note: the backslash approach produces a possible error warning.
>> x = pinv(R)*b
x=
0.750000000000000
0
Both cases yield a least squares solution.
Part 2: Consistent equations
>> R = [1 0;1 1;1 0;1 1]
R=
1
0
1
1
1
0
1
1
Note: full rank R.
>> b = [1 -1 1 -1]';
>> pinv(R)
ans =
0.500000000000000 0.000000000000000 0.500000000000000 0.000000000000000
-0.500000000000000 0.500000000000000 -0.500000000000000 0.500000000000000
>> inv(R'*R)*R'
ans =
0.500000000000000
0 0.500000000000000
0
-0.500000000000000 0.500000000000000 -0.500000000000000 0.500000000000000
Note: pinv(R) and inv(R*R)*R yield same LEFT inverse.
>> x = pinv(R)*b
x=
0.999999999999999
-1.999999999999999
>> x = R\b
x=
0.999999999999999
-1.999999999999999
Note: the following is a get rid of so many decimals technique.
>> x = round(10e6*x)/10e6
x=
1
-2
EXAMPLE 5.7: Solve the equation:
Rx(t ) = Y (t ) ! T U (t )
given as
'!
!C$
10
#
#
&
#0 1
#
&
'#1 1
# CA & !
# 0 (1
#
&
#
#
&
#
& ' # (1 1
2
# (1 0
#
&
" CA %
"
1
1
0
1
0
2
$
&
& ! x1
&# x
&# 2
& # x3
&"
&
%
!
#
$#
&#
&=#
&#
%#
#
"
2
2
2
0
0
1
$
&
&
&
&
&
&
&
%
OBJECTIVE: demonstration of two facts:
(i) All left inverses give the same unique answer when the equations are consistent; and
(ii) The Moore-Penrose pseudo left inverse produces the least squares error solution for inconsistent
equations.
Step 1: The Moore-Penrose pseudo left inverse of R is
R = [1 0 1;0 1 1;1 1 0;
0 -1 1;-1 1 0;-1 0 2]
R=
1
0
1
0
1
1
1
1
0
0
-1
1
-1
1
0
-1
0
2
>> RL1 = pinv(R)
RL1 =
0.2963 0.0370 0.2593 0.0370 -0.2593
0.0 0.250
-0.1852
0.2500 -0.2500 0.2500
0.1852 0.1481 0.0370 0.1481 -0.0370
0
0.2593
" 0.2963 0.037 0.2593 0.037 !0.2593 !0.1852
=$
0
0.25
0.25
!0.25
0.25
0
$
$ 0.1852 0.1481 0.037 0.1481 !0.037 0.2593
#
%
'
'
'
&
Step 2: A second left inverse computed by inverting the first three rows of R and appending zeros is
!
R2 L
" 0.5 !0.5 0.5 0 0 0
= $ !0.5 0.5 0.5 0 0 0
$
$
# 0.5 0.5 !0.5 0 0 0
%
'
'
'
&
Step 3: Both left inverses produce the solution: (x1, x2, x3) = (1, 1, 1)
Step 4: Now consider a variation on the theme: Suppose the data vector is perturbed by noise to
[Y ! TU ] + v = "
#
1.5 2 2.5 !0.5 0.5 1.5 $
%
(a) Now each left inverse produces a different solution:
!
#
#
! 0.73335 $
1#
& = R' L #
x=
1#
# 1.357 &
#
# 0.93335 &
"
%
#
#
"
(b) A second solution is given by:
1.5
2
2.5
'0.5
0.5
1.5
$
&
&
&
&
&
&
&
%
T
"
$
$
"0%
2$
' = R! L $
( x = !2
2$
$
'
$
$
'
# 0.5 &
$
$
#
1.5
2
2.5
!0.5
0.5
1.5
%
'
'
'
'
'
'
'
&
(b) A simple error analysis ( Rx ! [Y ! TU ] = error ) indicates how closely Rx1 and Rx 2 match the
measurement v:
(i) Rx1 ! [Y ! TU ] = 0.6589
2
and
(ii) Rx 2 ! [Y ! TU ] = 7
2
Hopefully, if the perturbed data vector is close to the actual (non-noisy) data vector, then the solution Rx1
will be a best approximation to the actual one.
IV. PROBLEM 2: SOLUTION DEVELOPMENT
!
!
1. RECALL PROBLEM 2: Given x(t ) = Ax(t ) + Bu (t ) , x (t 0 ) = x0 , where A is n ! n and B is n ! m , find an
impulsive vector input
n #1
!
u (t ) = $ !i" (i ) (t # t 0 ) = !0" (t # t 0 ) + !1" (t # t 0 ) + "!n#1" ( n#1) (t # t 0 )
i=0
!
+
which will (instantaneously) drive x(t 0 ) to a desired x(t 0 ) where !i " R m and ! (i ) (t ) is the "i-th
derivative of the -function". Recall equation 5.7:
" &0
'
'&
+
!
x(t 0 ) ! x(t 0 ) = " B AB ! A n!1B $ ' 1
#
%
"
'
' &n!1
#
$
" &0
(
'
(
' &1
( # Q' "
(
'
(
' &n!1
%
#
$
(
(
(
(
(
%
QUESTION 5.8: When does equation 5.7 (above) have a solution?
+
!
ANSWER: A solution exists whenever " x (t 0 ) ! x (t 0 ) $ lies in the column space of Q , i.e.,
#
%
(
)
+
!
rank " x (t 0 ) ! x (t 0 ) $ = rank (Q ) .
#
%
+
!
REMARK: Sometimes " x (t 0 ) ! x (t 0 ) $ will not lie in the column space of Q .
#
%
+
!
QUESTION 5.9: For arbitrary x(t 0 ) and x(t 0 ) when will there always exist a solution?
+
!
ANSWER: There will always exist a solution regardless of the values of x(t 0 ) and x(t 0 ) whenever rank(Q)
= n.
QUESTION 5.10: Why?
ANSWER: If rank(Q) = n there are always n linearly independent columns which span R n ; for arbitrary
+
!
+
!
values of x(t 0 ) and x(t 0 ) , " x(t 0 ) ! x(t 0 ) $ & R n is also Thus arbitrary. there is always a linear combination
#
%
+
!
of the columns of Q equal to " x (t 0 ) ! x (t 0 ) $ .
#
%
QUESTION 5.11: If rank(Q) = n, is the solution ever unique?
ANSWER: Yes, whenever Q is nxn, i.e. m = 1, in which case Q !1 exists. Otherwise the solution is NOT
unique, and that non-uniqueness is a good thing.
QUESTION 5.12: How does one compute the solution when m > 1 and rank(Q) = n?
ANSWER: When rank(Q) = n and m > 1, a solution is
" &0
'
!R "
+
! $ ' &1
Q # x (0 ) ! x (0 ) % = '
!
'
' &n!1
#
$
(
(
(
(
(
%
where Q ! R is any right inverse of Q .
QUESTION 5.13: OK, first the inverse, then the left inverse, and now a right inverse? Right! So, whats a
right inverse as opposed to a wrong inverse?
ANSWER: Well, first right means like on the RIGHT as opposed to on the LEFT. We do try to be nonjudgmental about our mathematical entities. Getting to the right answer, if rank(Q) = n, (the condition for
the existence of a right inverse) a right inverse of Q is any matrix Q ! R which satisfies QQ ! R = I . Notice
how Q ! R appears on the right side of Q as in figure 5.2.
Figure 5.2. The idea of a right inverse of a rectangular matrix Q.
QUESTION 5.14: Any relation to the Moore-Penrose pseudo inverse, pray tell?
ANSWER: Such a wonderful question, truly wonderful. Yes, and
(
Q = Q ! R = QT QQT
)
!1
which as with the left inverse in nice for simple hand calculations but numerically ill-conditioned; in terms
of our beloved SVD,
T
Q ! R = V1S !1U1
Exercise. Show that both of the above formulas are valid right inverses.
QUESTION 5.15: If rank(Q) = n and m > 1 (leading to non-unique soluitions) is it possible to characterize all
possible solutions to
" &0
'
&
+
!$
" x(t 0 ) ! x(t 0 ) = Q ' 1
'!
#
%
'
' &n!1
#
$
(
(
(
(
(
%
ANSWER: Yes.
QUESTION 5.16: Yes what?
!
!
+
"
ANSWER: Recall, N [Q ] = {v !|!Qv = 0} . Hence, for any particular solution ! to Q! = x(t 0 ) " x(t 0 ) , ! + v
is also a solution since .
!
!
+
"
Q ! + v = Q! = x(t 0 ) " x(t 0 )
(
QUESTION 5.12B: How does this help?
)
ANSWER: Recall that the usual SVD of Q would produce a V2 -matrix whose columns are a basis for N [Q ] .
Thus every solution has the form
# !0
%
% !1
%!
%
% !n"1
$
&
(
"
(
+
"
= Q " R # x(t 0 ) " x(t 0 ) & + V2) = ! + V2)
(
$
'
(
(
'
where
!
+
"
(a) ! = Q " R # x(t 0 ) " x(t 0 ) % is a particular solution, and
$
&
(b) V2! represents an arbitrary vector in N [Q ] .
Before illustrating the above material with an example, we note that any basis for the null space of Q
will work; one does not need to necessarily compute the SVD of Q. It is often easy enough to find a basis
for N[Q] by inspection.
!
EXAMPLE 5.8. Suppose x = Ax + Bu where
"0 1 0
A = $ 0 0 !1
$
$0 0 1
#
%
"0 1%
', B = $
'
'
$0 0'
'
$1 1'
&
#
&
Part 1. Find an impulsive input which drives x(0-) = 0
T
to x(0 + ) = ! 1 1 1 # . Specifically find any solution to
"
$
the equation, Q! = x(0 + ) " x(0 " ) .
Step 1. Construct Q = ! B AB A 2 B # .
"
$
" 0 1 0 0 !1 !1
Q = $ 0 0 !1 !1 !1 !1
$
$
#1 1 1 1 1 1
%
'
'
'
&
Step 2. Observe first three columns of Q are linearly independent; one possible right inverse of Q is
Q ! R1
"
$
$
=$
$
$
$
$
#
!1 1
10
0 !1
00
00
00
1
0
0
0
0
0
%
'
'
'
'
'
'
'
&
T
Hence ! = # 1 1 "1 0 0 0 % .
$
&
PART 2. Find the minimum energy input. Specifically, find the coefficient vector of minimum norm
which satisfies Q! = x(0 + ) " x(0 " ) .
Step 1. Computes the Moore-Penrose pseudo right inverse of Q , i.e.,
(
Q ! R2 = Q + = QT QQT
)
!1
. It follows that
Q ! R2
"
$
$
1$
=$
12
$
$
$
#
!4 10 8
4
24
4 !7 !2
4 !7 !2
!4 1 2
!4 1 2
Step 2. Solve using Q ! R2 or directly in MATLAB to obtain:
%
'
'
'
'
'
'
'
&
#
%
%
!%
! =%
%
%
%
$
1.1667
"0.8333
"0.4167
"0.4167
"0.0833
"0.0833
&
(
( # !0
(=% !
(%2
( % !3
$
(%
(
'
&
(
(
(
(
'
PART 3. Error Analysis
Step 1. Consider solution from part 1:
(1!!1!! !1!! 0 !! 0 !! 0 )T
= 1.732
Step 2. Consider
" 1.1667 0.8333 !0.4167 !0.4167 !0.0833 !0.0833
#
T
$
%
= 1.5546
2
PART 4. Characterization of all solutions
Step 1. Find a basis for N [Q ] = span {v1, v2 , v3 } where
"
$
$
v1 = $
$
$
$
$
#
0
0
0
0
1
!1
%
"0
'
$
'
$0
', v = $ 1
' 2 $ !1
'
$
'
$0
'
$
&
#0
%
"
'
$
'
$
', v = $
'3$
'
$
'
$
'
$
&
#
Step 2. Write down general solution
!3
! = ! + # " i vi
i =1
!1
1
0
!1
1
0
%
'
'
'
'
'
'
'
&
Discussion Questions
1. Answer each of the questions below from the given data: from Matlab you obtain the SVD of a
particular matrix M as:
[U,S,V] = svd(M)
U=
0.5000 -0.5000 -0.6304 -0.3203
0.5000
0.5000
0.3203 -0.6304
0.5000 -0.5000
0.3203
0.5000 -0.3203
0.5000
0.6304
0.6304
S=
2.4495
0
0
0
1.4142
0
0
0
0.0000
0
0
0
V=
0.4082 -0.7071
0.5774
0.4082
0.5774
0.7071
0.8165 -0.0000 -0.5774
True-False:
(i) Question 1: M has a left inverse. __________
(ii) Question 2: M has a right inverse. __________
(iii) Question 3: M has a pseudo inverse. __________
(iv) Question 4: If M x = b is an equation which is inconsistent, then a least squares solution is given by x
= pinv(M)*b. __________ where pinv means Moore-Penrose pseudo inverse.
(v) Question 5: The matrices U and V satisfy, U*UT = I and VT*V = I. __________
(vi) Question 6: The spectral norm of M is: __________
(vii) Question 7: A basis for the column space of M is:
(viii) Question 8: A basis for the null space of M is:
Worked Problems
1. The controllable canonical form of a systm represented by a scalar differential equation is given by
010
0
x= 0 0 1 x+ 0 u
-a -b 0
1
y=
1 0 -1 x + [1] u
(a) If a = 0, for what values of "b" is it NOT possible to drive an arbitrary x(0-) to an arbitrary x(0+).
Explain!!
T
(b) If a = 0, find an inpulsive input (if possible) which will drive x(0-) = 0 to x(0 + ) = ! 1 1 1 # . Write
"
$
out the structure of this input.
(c) If b = 0, for what values of "a" will there always exist a unique solution to the equation
R x(t) = [Y(t) - T U(t)].
(d) Suppose b = 0, a = 1, and
y(0)
4
y(0) = 0 , and
2
y(0)
u(0)
3
u(0) = 1
2
u(0)
determine a possible initial state of the system. Hint: THINK!
(e) Given your answer to part (d), describe the space of all solutions to R x(t) = [Y(t) - T U(t)]. Hint:
Borrow ideas from generalized problem 2.
(f) Determine the differential equation of the system having the state model given above.
Solution: (a) None. All values of b are permissible. The system is in controllable canonical form which
implies Q is always nonsingular.
(b) Since
001
010
1 0 -b
2
Q = [B AB A B] =
by inspection
Q
-1
b01
= 010
100
Further
!0
b01
! 1 = Q [x(0 ) - x(0 )] = 0 1 0
100
!2
-1
+
-
1
b+1
1= 1
1
1
The desired input then is u(t) = (b+1)d(t) + d'(t) + d"(t).
(c) Since
C
R=
CA
CA
=
2
1 0 -1
a10
0a1
it follows that det[R] = 1 - a2 which implies that a = 1 produces a singular matrix.
(d) x(0) = [1 1
0]T. The answer is not unique.
(e) N[R] = span[1, -1, 1]T. Hence
1
1
x(0) = 1 + ! -1
0
1
(f) Since
y = b0y
(3)
+ b1 y + b 2 y + b 3 y
then
y = (b3 - b0 a3) x1 + (b2 - b0 a2) x2 + (b1 - b0 a1) x3 + b0 u
It follows that b0 = 1, a3 = a, a2 = b, a1 = 0, b3 = 1+a, b2 = b, b1 = -1.
2. A particular physical process has the linear time varying state model
!
x (t) = A x(t) + B u(t)
y(t) = C(t) x(t) + D(t) u(t)
where x(t) R2 and u(t) R for each t.
(a) Derive a set of equations from input-output measurements whose solution would yield x(t). Put
equations in matrix form.
(b) Suppose u(t) = t 1+(t) and
A=
11
0
;B=
; C(t) = [1
01
1
t] ; D(t ) = [1 2t]
!
It was found that after exhaustive measurements and checks that at t = 1 second, y(1) = 0 and y (1) = (2e 5) where e = 2.7182818. Determine x(1). Leave answer in terms of e.
Solution:
(a) (i) y(t) = C(t)x(t) + D(t)u(t)
(ii)
y(t ) = C(t)x( t) + C(t )x(t) + D(t)u(t) + D(t)u(t)
= C(t)Ax(t ) + C(t) Bu( t) + C(t )x(t) + D(t)u(t) + D(t)u(t)
(iii) In matrix form
y(t)
C(t)
D(t)
=
x(t ) =
y(t)
C( t)A + C(t)
C(t )B + D(t)
(b)
0
=
2e 5
1 1 x(1) =
13
1 0
1 1
1
1
or equivalently
1
2e 3 =
1 1 x(1)
13
Hence
x(1) = 0.5
3 1
1 1
1
3e
2e 3 = e 2
0
D(t )
u(t)
u(t)