Learning Active and
Optimized Information Gathering
Lecture 8 Active Learning
CS 101.2 Andreas Krause
Announcements
Homework 1: Due today Office hours
Come to office hours before your presentation! Andreas: Monday 3pm-4:30pm, 260 Jorgensen Ryan: Wednesday 4:00-6:00pm, 109 Moore
2
Outline
Background in learning theory Sample complexity Key challenges Heuristics for active learning Principled algorithms for active learning
3
Spam or Ham?
Spam x2
+ +
o o o o o o +
Ham
o o o o o o o o
x1 label = sign(w0 + w1 x1 + w2 x2) (linear separator) Labels are expensive (need to ask expert) Which labels should we obtain to maximize classification accuracy?
4
Recap: Concept learning
Set X of instances, with distribution PX True concept c: X {0,1} Data set D = {(x1,y1),,(xn,yn)}, xi PX, yi = c(xi) Hypothesis h: X {0,1} from H = {h1, , hn, } Assume c H (c also called target hypothesis) errortrue(h) = EX |c(x)-h(x)| errortrain(h) = (1/n) i |c(xi)-h(xi)| If n large enough, errortrue(h) errortrain(h) for all h
5
Recap: PAC Bounds
How many samples n to we need to get error with probability 1- ? No noise: n 1/ ( log |H| + log 1/ ) Noise: n 1/2 ( log |H| + log 1/ )
Requires that data is i.i.d.!
Today: Mainly no-noise case (more next week)
6
Statistical passive/active learning protocol
Data source PX (produces inputs xi) Active learner assembles data set Dn = {(x1,y1),,(xn,yn)} by selectively obtaining labels Learner outputs hypothesis h errortrue(h) = Ex~P[h(x) c(x)] Data set NOT sampled i.i.d.!!
7
Example: Uncertainty sampling
Budget of m labels Draw n unlabeled examples Repeat until weve picked m labels
Assign each unlabeled data an uncertainty score Greedily pick the most uncertain example
One of the most commonly used class of heuristics!
8
Uncertainty sampling for linear separators
9
Active learning bias
10
Active learning bias
If we can pick at most m = n/2 labels, with overwhelmingly high probability, US pick points such that there remains a hypothesis with error > .1!!! With standard passive learning, error 0 as n
11
Wish list for active learning
Minimum requirement
Consistency: Generalization error should go to 0 asymptotically
Wed like more than that:
Fallback guarantee: Convergence rate of error of active learning at least as good as passive learning
What were really after
Rate improvement: Error of active learning decreases much faster than for passive learning
12
From passive to active
Passive PAC learning
1.
2. 3.
Collect data set D of n 1/ ( log |H| + log 1/ ) data points and their labels i.i.d. from PX Output consistent hypothesis h With probability at least 1-, errortrue(h)
Key idea
Sample n unlabeled data points DX={x1,,xn} i.i.d. Actively query labels until all hypotheses consistent with these labels agree on the labels of all unlabeled data
13
Why might this work?
14
Formalization: Relevant hypothesis
Data set D = {(x1,y1),,(xn,yn)}, Hypothesis space H Input data: DX = {x1,,xn} Relevant hypothesis H(DX) = H = Restriction of H on DX Formally: H = {h: DX {0,1} h H s.t. x DX: h(x)=h(x)}
15
Example: Threshold functions
16
Version space
Input data DX = {x1,,xn} Partially labeled: Have L = {(xi ,yi ),,(xi ,yi )} 1 1 m m The (relevant) version space is the set of all relevant hypotheses consistent with the labels L Formally:
Why useful?
Partial labels L imply all remaining labels for DX |V|=1
17
Version space
Input data DX = {x1,,xn} Partially labeled: Have L = {(xi ,yi ),,(xi ,yi )} 1 1 m m The (relevant) version space is the set of all relevant hypotheses consistent with the labels L Formally: V(DX,L) = V = {h H(DX): h(xi )=yi for 1 j m}
j j
Why useful?
Partial labels L imply all remaining labels for DX |V|=1
18
Example: Binary thresholds
19
Pool-based active learning with fallback
1. 2.
3.
Collect n 1/ ( log |H| + log 1/ ) unlabeled data points DX from PX Actively request labels L until there remains a single hypothesis h H thats consistent with these labels (i.e., |V(H,L)| = 1) Output any hypothesis hH consistent with the obtained labels. With probability 1- errortrue(h) Get PAC guarantees for active learning Bounds on #labels for fixed error carry over from passive to active Fallback guarantee
20
Wish list for active learning
Minimum requirement
Consistency: Generalization error should go to 0 asymptotically
Wed like more than that:
Fallback guarantee: Convergence rate of error of active learning at least as good as passive learning
What were really after
Rate improvement: of Error active learning decreases much faster than for passive learning
21
Pool-based active learning with fallback
1. 2.
3.
Collect n 1/ ( log |H| + log 1/ ) unlabeled data points DX from PX Actively request labels L until there remains a single hypothesis h H thats consistent with these labels (i.e., |V(H,L)| = 1) Output any hypothesis hH consistent with the obtained labels. With probability 1- errortrue(h)
22
Example: Threshold functions
23
Generalizing binary search [Dasgupta 04]
Want to shrink the version space (number of consistent hypotheses) as quickly as possible. General (greedy) approach:
For each unlabeled instance xi compute vi,1 = vi,0 = vi = min {vi,1, vi,0 } Obtain label yi for xi where i = argmaxj {vj}
24
Ideal case
25
Is it always possible to half the version space?
26
Typical case much more benign
27
Query trees
A query tree is a rooted, labeled tree on the relevant hypothesis H Each node is labeled with an input x DX Each edge is labeled with {0,1} Each path from root to hypothesis h H is a labeling L such that V(DX,L) = {h} Want query trees of minimum height
28
Example: Threshold functions
29
Example: linear separators (2D)
30
Number of labels needed to identify hypothesis
Depends on target hypothesis! Binary thresholds (on n inputs D_X)
Optimal query tree needs O(log n) labels!
For linear separators in 2D (on n inputs D_X)
For some hypotheses, even optimal tree needs n labels On average, optimal query tree needs O(log n) labels!
Average-case analysis of active learning
31
Average case query tree learning
Query tree T Cost(T) = 1/|H| h` H depth(h,T) Want T* = argminT Cost(T) Superexponential number of query trees Finding the optimal one is hard
32
Greedy construction of query trees [Dasgupta 04]
Algorithm GreedyTree(DX, L) V = H(DX) If V={h} return Leaf(h) Else For each unlabeled instance xi compute vi,1 = |V(H,L {(xi,1)}| and vi,0 = |V(H,L {(xi,0)}| vi = min {vi,1, vi,0} Let i = argmaxj {vj} LeftSubTree = GreedyTree(DX, L {(xi,1)}) RightSubTree = GreedyTree(DX, L {(xi,0)}) return Node xi with children LeftSubTree (1) and RightSubTree(0)
33
Near-optimality of greedy tree [Dasgupta 04]
Theorem: Let T* = argminT Cost(T) Then GreedyTree constructs a query tree T such that Cost(T) = O(log |H|) Cost(T*)
34
Limitations of this algorithm
Often computationally intractable
Finding most-disagreeing hypothesis is difficult
No-noise assumption Will see how we can relax these assumptions in the talks next week.
35
Bayesian or not Bayesian?
Greedy querying needs at most O(log |H|) queries more than optimal query tree on average Assumes prior distribution (uniform) on hypotheses If our assumption is wrong, generalizat...