Regression with dummy variables
In many occasions we use categorical variables as explanatory variables. Since these are NOT
numerical variables, they have to be handled with the proper methodological care.
For some (unjustified) reasons such variables (i

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 4
1. eigenvalue decomposition
recall our two fundamental problems:
Ax = b
and Ax = x
even if we are just interested to solve Ax = b and its variants, we will need to understand
eigenvalues and eige

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 9
1. backsolve
backsolve refers to a simple, intuitive way of solving linear systems of the form Rx = y or
Lx = y where R is upper-triangular and L is lower-triangular
take Rx = y for illustration

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 6
1. solving least squares problems
given A
Cmn
and b Cm , the least squares problem ask to find x Cn so that
kb Axk22
is minimized
note that x minimizes kb Axk22 iff it minimizes kb Axk2 , so whe

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 3
1. errors
to a vector x are
three commonly used measures of the error in an approximation x
the absolute error
k
abs = kx x
the relative error
k
kx x
rel =
kxk
the point-wise error
x
i xi
ele

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 12
1. orthogonalization using householder reflections
it is natural to ask whether we can introduce more zeros with each orthogonal rotation and
to that end, we examine Householder reflections
cons

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 8
1. qr and complete orthogonal factorization
poor mans svd
can solve many problems on the svd list using either of these factorizations
but they are much cheaper to compute there are direct algor

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 1
1. laundry list
web site: http:/www.stat.uchicago.edu/~lekheng/courses/309/
notes: http:/www.stat.uchicago.edu/~lekheng/courses/309/notes/
books: http:/www.stat.uchicago.edu/~lekheng/courses/309/bo

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 17
1. Richardson method
unlike the splitting methods in the previous lecture, the iterative methods here do not
require splitting A into a sum of two matrices but they are a bit like sor in that the

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 11
1. least squares with linear constraints
suppose that we wish to fit data as in the least squares problem, except that we are using
different functions to fit the data on different subintervals

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 7
1. finding closest unitary/orthogonal matrix
let U (n) be the set of all n n unitary matrices
given A Cnn , we wish to find the matrix X U (n) that satisfies
min kA XkF
XU (n)
let A = U V be the

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 13
1. need for pivoting
last time we showed that under proper circumstances,
1
0
0
0
u11
0
0
`21 1
0
.
.
.
. , U = 0
L = `31 `32
.
.
.
.
.
.
.
.
.
0
.
`n1 `n2 `n,n1 1
0
what exactly are prop

Box Plot (Tukey, 1977)
A box plot is a graphical display of the center, spread, and
shape of the data using the median, the quartiles, and
their ranges.
Box Plot: Centers and Hinges
Median Q2
Mean
Lower hinge = Q1
Upper hinge = Q3
Box Plot: Whiskers
Uppe

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 10
1. full rank least squares via normal equations
the second approach is to define
1
(x) = kAx bk22
2
which is a differentiable function of x
we can minimize (x) by noting that (x) = A (Ax b) which

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 14
1. another look at Cholesky
instead of considering an elementwise algorithm, we can also derive a vectorized version
this is analogous to our discussions of qr and lu
let F = [f1 , . . . , fn ]

Regression Prescriptions
General Multivariate analysis with continuous target variable and continuous explanatory
variables
1. Stability of data. Check that Mean > SD for all variables
2. Examine and interpret the correlations between the target variable

Some regression issues
1. Regression and correlation
Recall the functions of rx,y and of b1
rx , y
Cov ( X , Y )
,
SD( X ) SD(Y )
Cov ( X , Y )
b1
SD 2 ( X )
(1)
The major difference between the two coefficients is that r is symmetric while b 1 is not. T

1. In an experiment concerning a roll of two fair dice, let A be the event
The outcome of die I is greater than the outcome of die II. Let B be the
event the outcome of die II is greater than 3. Are these two events
independent, positively dependent or n

An example of a problem on the expectation of the Geometric
Distribution
Geometric distribution.
In an amusement park you see a shooting game that offers you the following
seemingly tempting option. You get 10 shooting trials. Once you hit the bulls
eye t

Quantifying a binary categorical variable: The Bernoulli solution
Consider an experiment with only two possible outcomes, denoted without loss of
generality, by HEADS and TAILS (abbreviated to H and T, respectively). Let H occur
with probability p, so tha

Optimizing a portfolio
This document outlines algorithms for finding the optimal mixture of stocks in a portfolio where
"optimum" is defined as follows.
Here, given the stocks you want to have in your portfolio, the following algorithms determine the
rela

Hypotheses Testing Bailout
In this document I address basic concepts and statistical procedures associated with hypotheses
testing. It is supposed to clarify the underlying subject and to give you a simple yet useful
summary guide into inference on means.

Journal of Expcnmentnl Psychology: General
2003. Vol. 132. No. l. 322
Copyright 2003 by the American Psychological Association. Inc.
0096-3445/03/512.(X) 001' 10 10371009634451.1113
The Psychology of the Monty Hall Problem: Discovering Psychological
Mecha

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 2
norms are of great importance in numerical computations because they allow us to measure
the size of errors
that is, how far are we from the true solution that we are seeking
this is important b

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 15
1. multiple right-hand sides and inverse
let A Rmn and b1 , . . . , bp Rm
suppose we need to solve p linear systems with the same coefficient matrix but different
right-hand sides
Ax1 = b1 , Ax2

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 5
1. existence of svd
Theorem 1 (Existence of SVD). Every matrix has a singular value decomposition (condensed
version).
Proof. Let A Cmn . We define the matrix
0 A
W =
C(m+n)(m+n) .
A 0
It is easy

STAT 309: MATHEMATICAL COMPUTATIONS I
FALL 2015
LECTURE 16
1. why iterative methods
if we have a linear system Ax = b where A is very, very large but is either sparse or
structured (e.g., banded, Toeplitz, banded plus low-rank, semiseparable, Hierarchica