Introduction to Microeconomic Concepts and Formulas
ECON 115

Winter 2006
Lecture 4 Notes: Chebyshevs Inequality
1
Let Z1, , Zn R be i.i.d. random variables. Were interested in bounds on n
Zi EZ.
(1) Jensens inequality: If is a convex function, then (EZ) E(X).
EZ
(2) Chebyshevs inequality: If Z 0, then P (Zt) t . Proof:
EZ = EZ
Introduction to Microeconomic Concepts and Formulas
ECON 115

Winter 2006
Lecture 1 Notes: Intro to Course
Consider a family of weak classifiers
H = cfw_h : X cfw_1, +1.
Let the empirical minimizer be
1
n
i
n
h0= argmin
I(h(Xi)= Yi)
=1
and assume its expected error,
1
2> = Error(h0), >0
Examples:
d
d
X = R , H = cfw_sign(wx +
Introduction to Microeconomic Concepts and Formulas
ECON 115

Winter 2006
Lecture 7 Notes: Sauers Dilemma
Theorem 13.1. Assume F is a VCsubgraph class and V C(F) =V . Suppose 1f(x)1 for all f F and x X .
1 n
Let x1, . . . , xn X and define d(f, g) =n
i=1 f(xi)g(xi). Then
D(F, , d)
(which is
K V+
8 e 7 V
log .
for some .)
P
Introduction to Microeconomic Concepts and Formulas
ECON 115

Winter 2006
Lecture 2 Notes: Maximizing Margins
d
As in the previous lecture, consider the classification setting. Let X = R , Y = cfw_+1,1, and
d
H = cfw_x + b, R , b R
where  = 1.
We would like to maximize over the choice of hyperplanes the minimal distance from t
Introduction to Microeconomic Concepts and Formulas
ECON 115

Winter 2006
Lecture 6 Notes: Rate Changes
Last time we proved the Pessimistic VC inequality:
P
n
C
1
n
sup
i P
i=1
I(X
C)
(C)
C
sup
V
Hence, the rate is
1
2
nt
e
,
8
2en
log 4 + V log
n
i
n
V
4
8
t=
P
2en
t
which can be rewritten with
as
V
P
n
nlog 4 + Vlog
8
i=1
V
Introduction to Microeconomic Concepts and Formulas
ECON 115

Winter 2006
Lecture 3 Notes: Average Generalization Error
Assume we have samples z1 = (x1, y1), . . . , zn = (xn, yn) as well as a new sample zn+1. The classifier trained
on the data z1, . . . , zn is fz1,.,zn .
The error of this classifier is
Error(z , . . . , z ) =
Introduction to Microeconomic Concepts and Formulas
ECON 115

Winter 2006
Lecture 5 Notes: Bennetts Profit Theorem
2
2
Last time we proved Bennetts inequality: EX = 0, EX = , X < M = const, X1, , Xn independent copies of
X, and t 0. Then
P
n
Xi t
2
exp
M
i=1
tM
n
2
2
n
,
where (x) = (1 + x) log(1 + x) x.
2
x
2
2
2
x
) x = x
Introduction to Microeconomic Concepts and Formulas
ECON 115

Winter 2006
Lecture 10 Notes: Uniform Economics
In a classification setup, we are given cfw_(xi, yi) : xi X , yi cfw_1, +1i=1,n, and are required to construct a
classifier y = sign(f (x) with minimum testing error. For any x, the term y f(x) is called margin can be
c
Introduction to Microeconomic Concepts and Formulas
ECON 115

Winter 2006
Lecture 8 Notes: Variable Changes
j=1 R(j (f ) j1(f ). We first show how to control R on the links. Assume
Hoedings
Lj1,j . Then by
inequality
P
n
i=1
1
i i
exp
t
1 2
n 2 i
2
n
2
t
nt2
i=1 i2
2 n1
n
= ex p
2
exp
nt
222j+4
Note that
2
cardLj1,j cardFj1 c
Introduction to Microeconomic Concepts and Formulas
ECON 115

Winter 2006
Lecture 9 Notes: General VC Inequality
In Lecture 8, we proved the following Generalized VC
inequality
P f F,
Efn
Ex
f(xi) n 0 log1/2 D(F, , d)d + 27/2 Ex n
d(0,
d(0,
29/
1 n
2
f)2t
f)
i=1
d(f, g)
n
=
1 n
1/
2
2
(f(xi) f(xi ) g(xi) + g(xi )
i
=
1
Definit