{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

UnsupervisedLearning_2

# UnsupervisedLearning_2 - A UNIFIED PERSPECTIVE ON...

This preview shows pages 1–15. Sign up to view the full content.

A UNIFIED PERSPECTIVE ON SUPERVISED, UNSUPERVISED AND SEMI-SUPERVISED LEARNING

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Problem Types
Problem Types Supervised learning Regression Given input descriptions and target values Infer a function Usually x 2 R n y 2 R k f : X ! Y k << n x y x y x y x y

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Problem Types Supervised Unsupervised learning Regression Dimensionality reduction Given input descriptions Simultaneously infer a function and a guess at the missing labels Z Unsupervised regression usually called dimensionality reduction because x 2 R n f : X ! Y k << n y x y x y x y x
Problem Types Supervised learning Classification Given inputs and target values Infer a function Usually x 2 R n f : X ! Y k << n y 2 f 1 ;:::;k g x y x y x y x y

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Problem Types Supervised Unsupervised learning Classification Clustering Given inputs Simultaneously infer a function and a guess at the missing labels Z Unsupervised classification usually called clustering because the guessed classifications Z are emphasized over f x 2 R n f : X ! Y y x y x y x y x
Problem Types Semi-supervised learning Some training labels are given, but most are not x y x y y x y x y x y x

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Training Principles Are unsupervised and supervised training problems really so unrelated? Does supervised and unsupervised learning require distinct training principles, distinct motivations, distinct theory? Maybe not
Key Challenge Relating unsupervised to supervised learning 1. If both unified under a common training principle Get semi-supervised training principle for free Get a head-start on theory and algorithms 2. If not, then need different supervised and unsupervised training principles Proliferation of semi-supervised combinations No head-start on theory and algorithms Understanding is difficult Unfortunately, #2 dominates current literature But we will see that #1 might be possible

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Overview Will demonstrate a unification of supervised, unsupervised and semi-supervised training under least squares The same training principle will yield Supervised least squares Principal components analysis K-means clustering Normalized graph-cut clustering and semi-supervised versions thereof Only differences are constraints imposed on labels given/missing, continuous/discrete The unification is based on a non-standard but equivalent view of least squares learning
A revealing case study Supervised Least Squares

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Supervised Least Squares Given t £ n data matrix X t £ k label matrix Y t = # training instances n = # features k = # targets Assume X is full rank n Y is full rank k k < n X Y
Supervised Least Squares Learn: parameters W ( n £ k ) for a model Objective A convex quadratic, so just solve for a critical point: Thus Test prediction Given test vector x , predict : W f X Y min W tr (( XW ¡ Y )( XW ¡ Y ) 0 ) d dW = 2 X 0 ( XW ¡ Y ) = 0 X 0 XW = X 0 Y W = ( X 0 X ) ¡ 1 X 0 Y = X y Y ^ y = W 0 x

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Supervised Least Squares Regularization Kernelization
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 52

UnsupervisedLearning_2 - A UNIFIED PERSPECTIVE ON...

This preview shows document pages 1 - 15. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online