5.2.Radial Basis Function Networks

5.2.Radial Basis Function Networks - Acknowledgement:...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Acknowledgement: Material derived from slides for the book Machine Learning, Tom M. Mitchell, McGraw-Hill, 1997 http://www-2.cs.cmu.edu/~tom/mlbook.html 11s1: COMP9417 Machine Learning and Data Mining Radial Basis Function Networks April 5, 2011 Aims Introduction This lecture will enable you to describe and reproduce the approach of radial basis function networks. Following it you should be able to: • Radial Basis Function Networks are a local learning method • Combines ideas from a number of approaches in machine learning • describe radial basis function networks • describe key concepts and their relation to other learning approaches • Example of how to design new approaches Reading: Mitchell, Chapter 8 COMP9417: April 5, 2011 RBF Networks: Slide 1 COMP9417: April 5, 2011 RBF Networks: Slide 2 Radial Basis Function Networks Radial Basis Function Networks f(x) • Global approximation to target function, in terms of linear combination of local approximations • Used, e.g., for image classification w0 • A different kind of neural network • Closely related to distance-weighted regression, but “eager” instead of “lazy” w1 1 wk ... ... a 1 (x) a2 (x) COMP9417: April 5, 2011 RBF Networks: Slide 3 In the diagram ar (x) are the attributes describing instance x. RBF Networks: Slide 4 Q1: What xu (subsets) to use for each kernel function Ku(d(xu, x)) The learned hypothesis has the form: k ￿ COMP9417: April 5, 2011 Training Radial Basis Function Networks Radial Basis Function Networks f ( x) = w 0 + a n (x) • Scatter uniformly throughout instance space • Or use training instances (reflects instance distribution) wuKu(d(xu, x)) • Or prototypes (found by clustering) u=1 where each xu is an instance from X and the kernel function Ku(d(xu, x)) decreases as distance d(xu, x) increases. Q2: How to train weights (assume here Gaussian Ku) One common choice for Ku(d(xu, x)) is • First choose variance (and perhaps mean) for each Ku – e.g., use EM Ku(d(xu, x)) = e − 12 d2 (xu ,x) 2σ • Then hold Ku fixed, and train linear output layer u – efficient methods to fit linear function i.e. a Gaussian function. COMP9417: April 5, 2011 RBF Networks: Slide 5 COMP9417: April 5, 2011 RBF Networks: Slide 6 ...
View Full Document

Ask a homework question - tutors are online