UnsupervisedLearning_2

UnsupervisedLearning_2 - A UNIFIED PERSPECTIVE ON...

Info iconThis preview shows pages 1–15. Sign up to view the full content.

View Full Document Right Arrow Icon
A UNIFIED PERSPECTIVE ON SUPERVISED, UNSUPERVISED AND SEMI-SUPERVISED LEARNING
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Problem Types
Background image of page 2
Problem Types Supervised learning Regression Given input descriptions and target values Infer a function Usually x 2 R n y 2 R k f : X ! Y k << n x y x y x y x y
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Problem Types Supervised Unsupervised learning Regression Dimensionality reduction Given input descriptions Simultaneously infer a function and a guess at the missing labels Z Unsupervised regression usually called dimensionality reduction because x 2 R n f : X ! Y k << n y x y x y x y x
Background image of page 4
Problem Types Supervised learning Classification Given inputs and target values Infer a function Usually x 2 R n f : X ! Y k << n y 2 f 1 ;:::;k g x y x y x y x y
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Problem Types Supervised Unsupervised learning Classification Clustering Given inputs Simultaneously infer a function and a guess at the missing labels Z Unsupervised classification usually called clustering because the guessed classifications Z are emphasized over f x 2 R n f : X ! Y y x y x y x y x
Background image of page 6
Problem Types Semi-supervised learning Some training labels are given, but most are not x y x y y x y x y x y x
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Training Principles Are unsupervised and supervised training problems really so unrelated? Does supervised and unsupervised learning require distinct training principles, distinct motivations, distinct theory? Maybe not
Background image of page 8
Key Challenge Relating unsupervised to supervised learning 1. If both unified under a common training principle Get semi-supervised training principle for free Get a head-start on theory and algorithms 2. If not, then need different supervised and unsupervised training principles Proliferation of semi-supervised combinations No head-start on theory and algorithms Understanding is difficult Unfortunately, #2 dominates current literature But we will see that #1 might be possible
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Overview Will demonstrate a unification of supervised, unsupervised and semi-supervised training under least squares The same training principle will yield Supervised least squares Principal components analysis K-means clustering Normalized graph-cut clustering and semi-supervised versions thereof Only differences are constraints imposed on labels given/missing, continuous/discrete The unification is based on a non-standard but equivalent view of least squares learning
Background image of page 10
A revealing case study Supervised Least Squares
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Supervised Least Squares Given t £ n data matrix X t £ k label matrix Y t = # training instances n = # features k = # targets Assume X is full rank n Y is full rank k k < n X Y
Background image of page 12
Supervised Least Squares Learn: parameters W ( n £ k ) for a model Objective A convex quadratic, so just solve for a critical point: Thus Test prediction Given test vector x , predict : W f X Y min W tr (( XW ¡ Y )( ¡ Y ) 0 ) d dW = 2 X 0 ( ¡ Y ) = 0 X 0 = X 0 Y W = ( X 0 X ) ¡ 1 X 0 Y = X y Y ^ y = W 0 x
Background image of page 13

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Supervised Least Squares Regularization Kernelization Instance weighting 11
Background image of page 14
Image of page 15
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 06/16/2011 for the course CS 5141 taught by Professor Chenenhong during the Spring '10 term at USTC.

Page1 / 52

UnsupervisedLearning_2 - A UNIFIED PERSPECTIVE ON...

This preview shows document pages 1 - 15. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online