Lecture 9
Oct. 4 -2006
Data Visualization
STAT 442 / 890, CM 462
Lecture: Ali Ghodsi
1
LLE continued.
Previously, we minimized the following:
n
min
w
such that
k
i=1
k
wij xNi |2
|xi
i=1
j=1
wij = 1. Now the other cost function for LLE must be minimized.
Lecture 10, 11
Oct. 11, 13 -2006
Data Visualization
STAT 442 / 890, CM 462
Lecture: Ali Ghodsi
1
Metric Multidimensional Scaling (MDS)
An alternative perspective on dimensionality reduction is oered by Multidimensional scaling
(MDS). MDS is another classi
Lecture 24 and 25
Nov. 13 and 15 -2006
Data Visualization
STAT 442 / 890, CM 462
Lecture: Ali Ghodsi
1
Learning a Metric from Class-Equivalence Side Information
Given a set of t points, cfw_xi t Rn , we identify two kinds of class-related side information
Logistic regression
Matthias Schonlau, Ph.D.
Overview
Logistic regression
Interpretation of Coefficients
Prediction
Estimation
Example: Pharmacy
Witten et al. Chapter 4.3
Hastie et al. Chapter 4.4
Many other books on logistic regression and generaliz
Statistical Learning Classification
Stat441
Matthias Schonlau, Ph.D.
Statistical Learning Classification
Stat441
Overview
What is statistical learning?
Overfitting/ Train-test split
Example Dutch income vs age
Bias-variance tradeoff
Some concepts
Pr
Stat 441 - Fall 2016
Instructor: Dr. Schonlau
Email: [email protected]
Office: M3, room 4111
Lectures:
Tuesday and Thursday 14:30p.m.-15:50p.m., MC4045
It is your responsibility to know what happened in the class, whether you miss a class or
not. If
On maps
Location, location, location
R.W. Oldford
University of Waterloo
Fall 2016
Location information in two dimensions
Our earliest data visualization of two dimensional relationships are maps, for
example:
I
star maps
I
moon maps
I
town maps (3,500 ye
Decomposing time series
R.W. Oldford
University of Waterloo
Fall 2016
Time series data - Sunspot activity
Recall the measurements of monthly mean relative sunspot numbers from 1749 to 1983
plotted as
200
50 100
0
Sunspot numbers
Monthly sunspot activity
1
Transforming data
R.W. Oldford
University of Waterloo
Fall 2016
Transforming data
CO2 data: changed the aspect ratio (changes relative scales)
aspect ratio = 0.14
360
350
co2(ppm)
360
340
355
co2(ppm)
370
365
380
aspect ratio = 1
1985
1990
1995
Time
Revea
Moving to 2 dimensions
R.W. Oldford
University of Waterloo
Fall 2016
Getting to two dimensions
0. Zero dimensions
I
have a single y
I
I
visualization identified with an encoding of this single value
I
I
I
minimally it has identity, its value could be cate
Lecture 22,23
Nov. 8,10 -2006
Data Visualization
STAT 442 / 890, CM 462
Lecture: Ali Ghodsi
1
Scribes: Stefan Pintilie
Mixture Models
In this section the data points come from a density function. Lets assume that the data
comes from a Gaussian, or a mixtu
Lecture 20,21
Nov. 3,6 -2006
Data Visualization
STAT 442 / 890, CM 462
Lecture: Ali Ghodsi
1
Scribes: Stefan Pintilie
Combinatorial Algorithms:
Suppose we have n data points which are indexed 1 . . . n and suppose we would like to cluster
these points int
Lecture 5
Sept. 22-2006
Data Visualization
STAT 442 / 890, CM 462
Lecture: Ali Ghodsi
1
Alternative Derivation
Another nice property of PCA, closely related to the original discussion by Pearson [1], is
that the projection onto the principal subspace mini
Lecture 7,8
Sept. 27,29 -2006
Data Visualization
STAT 442 / 890, CM 462
Lecture: Ali Ghodsi
0.1
Centering
In the derivation of the kernel PCA we assumed that (X) has zero mean.
n
(xi ) = 0
i=0
The problem is that we must ensure that this condition is sati
Lecture 6
Sept. 25-2006
Data Visualization
STAT 442 / 890, CM 462
Lecture: Ali Ghodsi
1
Dual PCA
It turns out that the singular value decomposition also allows us to formulate the principle
components algorithm entirely in terms of dot products between da
Lecture 12
Oct. 16 -2006
Data Visualization
STAT 442 / 890, CM 462
Lecture: Ali Ghodsi
1
Scribes: Stefan Pintilie
Landmark MDS:
Landmark MDS is based on the MDS algorithm. We can rst have a quick look at how MDS
1
works. For a given distance matrix D(X) w
Lecture 13
Oct. 18 -2006
Data Visualization
STAT 442 / 890, CM 462
Lecture: Ali Ghodsi
1
Scribes: Stefan Pintilie
Landmark MDS cont.
Previously we saw that we could approximate the symmetric matrix K by knowing only the
A and B parts.
B
A
K=
B T B T A1 B
Lecture 3 and 4
Sept. 18 and Sept.20-2006
Data Visualization
STAT 442 / 890, CM 462
Lecture: Ali Ghodsi
1
Principal Components Analysis
Principal components analysis (PCA) is a very popular technique for dimensionality reduction. Given a set of data on n
Lecture 19
Nov. 1 -2006
Data Visualization
STAT 442 / 890, CM 462
Lecture: Ali Ghodsi
1
Scribes: Stefan Pintilie
Clustering
Clustering (also known as segmentation) is the task of grouping a collection of objects into
groups or clusters such that objects w
Lecture 14, 15 and 16
Oct. 20,23 and 25 -2006
Data Visualization
STAT 442 / 890, CM 462
Lecture: Ali Ghodsi
1
Unied Framework
All of the algorithms presented so far can be cast as kernel PCA,
A straightforward connection between LLE and Kernel PCA has bee
Lecture 17
Oct. 27 -2006
Data Visualization
STAT 442 / 890, CM 462
Lecture: Ali Ghodsi
1
Scribes: Stefan Pintilie
Landmark SDE
One of the major problems with SDE is that it requires the use of semidenite programming
which is computationally intensive. SDE
Categorical data
Two way
R.W. Oldford
University of Waterloo
Fall 2016
Categorical data
Categorical variates are either
I
nominal, that is the values that a categorical variate takes are simply
names
I
I
I
I
country might take one of Canada, United States