This preview shows page 1. Sign up to view the full content.
Unformatted text preview: Acknowledgement: Material derived from slides for the book
Machine Learning, Tom M. Mitchell, McGraw-Hill, 1997
http://www-2.cs.cmu.edu/~tom/mlbook.html 11s1: COMP9417 Machine Learning and Data Mining Radial Basis Function Networks
April 5, 2011 Aims Introduction This lecture will enable you to describe and reproduce the approach of
radial basis function networks. Following it you should be able to: • Radial Basis Function Networks are a local learning method
• Combines ideas from a number of approaches in machine learning • describe radial basis function networks
• describe key concepts and their relation to other learning approaches • Example of how to design new approaches Reading: Mitchell, Chapter 8 COMP9417: April 5, 2011 RBF Networks: Slide 1 COMP9417: April 5, 2011 RBF Networks: Slide 2 Radial Basis Function Networks Radial Basis Function Networks f(x)
• Global approximation to target function, in terms of linear combination
of local approximations
• Used, e.g., for image classiﬁcation w0 • A diﬀerent kind of neural network
• Closely related to distance-weighted regression, but “eager” instead of
“lazy” w1 1 wk ...
... a 1 (x) a2 (x)
COMP9417: April 5, 2011 RBF Networks: Slide 3 In the diagram ar (x) are the attributes describing instance x. RBF Networks: Slide 4 Q1: What xu (subsets) to use for each kernel function Ku(d(xu, x)) The learned hypothesis has the form:
COMP9417: April 5, 2011 Training Radial Basis Function Networks Radial Basis Function Networks f ( x) = w 0 + a n (x) • Scatter uniformly throughout instance space
• Or use training instances (reﬂects instance distribution) wuKu(d(xu, x)) • Or prototypes (found by clustering) u=1 where each xu is an instance from X and the kernel function Ku(d(xu, x))
decreases as distance d(xu, x) increases. Q2: How to train weights (assume here Gaussian Ku) One common choice for Ku(d(xu, x)) is • First choose variance (and perhaps mean) for each Ku
– e.g., use EM Ku(d(xu, x)) = e − 12 d2 (xu ,x)
2σ • Then hold Ku ﬁxed, and train linear output layer u – eﬃcient methods to ﬁt linear function i.e. a Gaussian function.
COMP9417: April 5, 2011 RBF Networks: Slide 5 COMP9417: April 5, 2011 RBF Networks: Slide 6 ...
View Full Document