Lecture 12
August 11, 2015
First wrap up some stu from last lecture.
Conditional density/joint density/conditional expectation.
Joint distribution: so far we have considered only the distribution of
a single random variable; now we are interested in th

Homework 2
July 9, 2015
1 2
0 2
1. Let A =
0 0
0 0
fundamental subspaces
0 3
2 2
. Find the basis and dimension of the four
0 0
0 4
(the column space and null space of A and A ).
2. What do you know about C(A) when the number of solutions to Ax = b
is (a

HW1
July 1, 2015
1.
Let
2
3
2 3
1 0 1
4
A = 4 1 1 0 5 , b = 4 35 .
1 1 1
6
Calculate Ab, and solve the equation Ax = b by Gauss-Jordan elimination,
LU decomposition, and by calculating A 1 b.
2. The Fibonacci sequence is dened by F1 = 1, F2 = 1, Fn+2 = Fn

2
3
2 3
10
1
4 7 5. Ax = b has solution x = 425.
1. Ab =
13
3
2. Observe that:
Fn+2
1 1 Fn+1
=
.
Fn+1
1 0
Fn
n
1 1
F
Fn
Note that
= n+1
. This can be proven by induction. Then
1 0
Fn F n 1
our expression for Fm+n+1 is given by:
m
Fm+n+1
1 1
Fn+1
=
Fm+n
1

1. The monthly worldwide number of plane crashes is 3.5. What is the probability that there will be (a) at least 2 such accidents in the next month?
(b) at most 1 accident in the next month?
2. Let X be a binomial random variable with parameters (n, p). F

Today well be taking a much more rigorous look at probability.
Recall that the sample space (denoted by ) is the set of all possible
outcomes. Consider for example = [0, 1], then the probability of
any particular outcome happening is zero. So we cannot

An important result is that for aperiodic, irreducible Markov chains,
there is a unique stationary distribution that solves = P . In this
case the invariant distribution and the limiting distribution are the
same.
Clearly there are dierent questions tha

MSM502: Linear Algebra
Date and location TBA
Instructor: Leon Cui (leon.c.cui@gmail.com) Phone: 908-240-5288
Course Objectives: The goal of this course is to give an introduction to linear algebra. A thorough
understanding of this subject is fundamental f

1. (1pt) If A is a 6417 matrix of rank 11, how many independent vectors
x satisfy Ax = 0? How many independent vectors satisfy A x = 0?
Proof. Since the rank is 11, there are 11 pivots and so there are 17-11=6
free variables, meaning that 6 independent ve

Lecture 1
June 28, 2015
Matrices are a bunch of numbers put into an array:
Ex:
6 5 4
.
3 2 1.
A=
A is a matrix. It has 2 rows and 3 columns, i.e. A is a 2 3 matrix.
Notation: Ajk is the entry in the jth row and kth column.
A23 =? A32 =?
Special case: A

Lecture 11
August 10, 2015
Review of last time: lagrange multipliers.
Problem: minimize f (x) such that g(x) = 0. Solve it by setting up
the Lagrangian L = f (x) g(x) and set all partial derivatives equal
to zero. We must solve f = g, g(x) = 0.
Intuiti

Lecture 10
August 5, 2015
Introduction to matrix calculus: dierentiating linear and quadratic
matrix functions.
Suppose y = f (x) where y is a vector in Rm and x is a vector in Rn .
dy
Dene the rst derivative dx by the matrix D where Dij =
is called th

Lecture 8
July 27, 2015
SVD continued: recall that the SVD factorization of a matrix A is
given by A = U V , where U contains the eigenvectors of AA in its
columns, V contains the eigenvectors of A A in its columns, and
contains the square roots of the

Lecture 9
August 2, 2015
Linear programming: simplex method, duality, application to zerosum games.
A linear program is an optimization problem that has a linear objective
function and linear constraints.
min c x,
x
s.t. Ax b,
x 0.
The set of all point

Lecture 7
July 21, 2015
Consider nding the minimum point of a quadratic form f (x, y) =
ax2 + 2bxy + cy 2 .
If the quadratic form has a minimum point, it is called positive denite.
Note that clearly solving fx = 0, fy = 0 gives (x, y) = (0, 0). We want

Lecture 6
July 20, 2015
Recall the dierential equation we solved earlier: du = Au where
A = 2 1 1 2 . The rst step is to nd the eigenvalues (-1 and
1
1
.
and
3) and the eigenvectors
1
1
The solution was a combination of the eigenvalues and eigenvectors:

Lecture 5
July 7, 2015
Diagonalization: Form a matrix S with its columns as the eigenvectors
of A. Form matrix with diagonal entries equal to its eigenvalues
and 0 everywhere else. Then if all the eigenvectors are independent,
S 1 AS = . To see why, just

Lecture 4
July 8, 2015
First talk about how to calculate a determinant before showing what
it is good for.
The determinant of a square matrix A is a number. For a 2 2 matrix
a b
, det(A) = ad bc.
A=
c d
The determinant for an n n matrix A is given by t

The length (or the norm) of a vector x, denoted by x is given by
the Pythagorean theorem: x =
of vector x. Note that x
the 0 vector.
2
n 2
1 xi ,
where n is the dimension
= x x. The only vector with length 0 is
Given vectors x and y, how to test if they

First wrap up some stu from last lecture.
Conditional density/joint density/conditional expectation.
Joint distribution: so far we have considered only the distribution of
a single random variable; now we are interested in the distribution of
two varia

Review of last time: lagrange multipliers.
Problem: minimize f (x) such that g(x) = 0. Solve it by setting up
the Lagrangian L = f (x) g(x) and set all partial derivatives equal
to zero. We must solve f = g, g(x) = 0.
Intuition: imagine we are travelli

Introduction to matrix calculus: dierentiating linear and quadratic
matrix functions.
Suppose y = f (x) where y is a vector in Rm and x is a vector in Rn .
dy
Dene the rst derivative dx by the matrix D where Dij =
is called the Jacobian matrix.
yi
xj .

1. Just plug in R R for A and apply the regular Cauchy-Schwarz inequality:
|x Ay| = |x R Ry| = |(Rx) (Ry)| Rx Ry
= |x R Rx|y R Ry| = |x Ax|y Ay|.
2 1 1
1 is positive denite. Easiest way is to check
2. The matrix 1 2
1 1
2
2 1 1
2 1
1 >
the top left determ

1. If A = R R, prove the generalized Schwarz inequality: |x Ay|2 (x Ax)(y Ay).
2 1 1
1 . Also,
2. Decide for or against the positive deniteness of 1 2
1 1
2
1 2 4
for what values of b is the matrix 2 b 8 positive denite?
4 8 7
0 1 0
3. Find the SVD for th

1. Let f (x) = a + bx + cx2 . Find the dierentiation matrix, i.e. the matrix
such that
a
b
b = 2c .
D
c
0
Compute D3 and interpret the result in terms of derivatives. What are the
eigenvalues and eigenvectors of D?
0 1 0
2. Let P = 0 0 1. Calculate P 2

0 1 0
1. The dierentiation matrix is given by D = 0 0 2. The third power
0 0 0
3 is a matrix of all zeroes; taking the third derivative of a second order
D
polynomial gives you all zeroes. The eigenvalue is 0 and the eigenvector is
(1, 0, 0). Applying th

HW1
June 2, 2014
1.
Let
4
1 0 1
A = 1 1 0 , b = 3 .
6
1 1 1
Calculate Ab, and solve the equation Ax = b by Gauss-Jordan elimination,
LU decomposition, and by calculating A1 b.
2. The Fibonacci sequence is dened by F1 = 1, F2 = 1, Fn+2 = Fn + Fn+1
(so th