Problem Set 3 Solutions
Statistics 315A
Problem 1
1(a) Since it is known that column rank(X ) = row rank(X ) = N , the columns of X span RN . By
denition, it is therefore always possible to nd a satisfying X = y , and so there is a LS
solution with zero r
Statistics 315a
Homework 3, due Wednesday March 12, 2014.
1. ESL 5.7.
2. Let X be the n p regression matrix for a logistic regression, with
p
n, and y the response vector. Assume X has row-rank n. Let
X = UDVT be the full SVD of X. That is, U is nn orthog
Statistics 315a
Homework 2, due Wednesday February 13, 2013.
1. ESL 3.12 & 3.30
2. ESL 3.15
3.
(a) Suppose that we run a ridge regression with parameter on a
single variable X , and get coecient a. We now include an exact
copy X = X , and ret our ridge re
Statistics 315a
Homework 2, due Wednesday February 12, 2014.
1. ESL 3.12 & 3.30
2.
(a) Suppose that we run a ridge regression with parameter on a
single variable X , and get coecient a. We now include an exact
copy X = X , and ret our ridge regression. Sh
HW 3 Solutions
March 18, 2013
Grade distribution: Problems 1 - 5: 12 points each, Problem 6: 15 points
for writeup, 15 points for computation.
Problem 1
a) Let cfw_X, y denote the full original dataset, and let cfw_X(i) , y(i) denote the dataset with the
STATS 315A
Winter 2007
Homework 1
Solutions
Prob. #1 (Thanks to Wei Zhen) (a) The function mixG takes a centroid matrix mu, a vector N specifying the number of samples in each group and the noise variance v.
mixG <- function (mu, N, v)cfw_ mu <- rbind(mu)
Stats 315A HW2 Solutions
February 17, 2014
If there are any questions regarding the solutions or the grades of HW 2, please contact
Austen (ahead@stanford.edu) with Stats315A-hw2-grading in the subject line.
Grade Distribution: Total 100 Points
Problem 1:
Statistics 315a
Homework 2, due Wednesday February 12, 2014.
1. ESL 3.12 & 3.30
2.
(a) Suppose that we run a ridge regression with parameter on a
single variable X , and get coecient a. We now include an exact
copy X = X , and ret our ridge regression. Sh
Statistics 315a
Homework 1, due Wednesday January 29, 2014.
ESL refers to the course textbook, and ESL 2.4 refers to exercise 2.4 in
ESL. Since the homework assignments count 70% of your nal grade, you
must do them on your own. Problem 1 is computing inte
STAT 315A Homework 2 Solutions
Problem 1: ESL 4.2
a) For LDA,
k = x 1 k 1/2k 1k + log (Nk /N ).
Substitute this formula into the requirement that 2 (x) 1 (x) > 0 and we get the
solution
x 1 (2 1 ) + 1/21 1 1 1/22 1 2 + log (N2 /N ) log (N1 /N ) > 0.
Statistics 315a
Homework 1, due Wednesday January 30, 2013.
ESL refers to the course textbook, and ESL 2.4 refers to exercise 2.4 in
ESL. Since the homework assignments count 70% of your nal grade, you
must do them on your own. Problem 1 is computing inte
STAT 315a
Midterm winter 2013- Solutions
Duration- 1 hour and 15 min
Aids allowed: class text, class notes, and calculators
The questions below require fairly short answers and are of equal value. They are in
no particular order. No one is expected to ans
Statistics 315a
Homework 3, due Wednesday Dec 3, 2008
1. Consider a linear regression problem where p
N , and assume the
T = RVT , where R is
row rank of X is N . Let the SVD of X = UDV
N N nonsingular, and V is p N with orthonormal columns.
(a) Show that
Elements of Statistical Learning
Solutions to the Exercises
Yu Zhang, sjtuzy@gmail.com
November 25, 2009
Exercise 2.6 Consider a regression problem with inputs xi and outputs yi ,
and a parameterized model f (x) to be t by least squares. Show that if
ther
Stats315a: Statistical Learning
1
Statistics in the news
How IBM built Watson, its Jeopardy-playing
supercomputer by Dawn Kawamoto DailyFinance 02/08/2011
Learning from its mistakes
According to David Ferrucci
(PI of Watson DeepQA technology for IBM Resea
Link between LDA and OLS
Jean-Philippe Vert
June 9, 2011
This is the solution to exercise 4.2 of [1] which shows a link between
linear discriminant analysis (LDA) and ordinary least squares (OLS) in the
binary case.
We have features x Rp and a two-class r
1
STAT 315a
Midterm Winter 2009
Duration- 1 hour and 15 min
Aids allowed: class text, class notes, and calculators
The questions below require fairly short answers and are of equal value.
They are in no particular order. No one is expected to answer all o
Elements of Statistical Learning
Andrew Tulloch
Contents
Chapter 2.
Overview of Supervised Learning
Chapter 3.
Linear Methods for Regression
12
Chapter 4.
Linear Methods for Classication
23
Chapter 5.
Basis Expansions and Regularization
28
Chapter 13.
4
S
Problem Set 1 Solutions
Statistics 315A
Problem 1
(a)
0.05
Both the k -nearest neighbor (KNN) and linear regression classiers are implemented in the code
included in the appendix. The performance of the methods, along with the performance of the
hybrid me
Statistics 315a
Homework 1, due Wednesday Oct 15 , 2008.
1. Compare the classication performance of linear regression and k
nearest neighbor classication on the zipcode data. In particular, consider only the 2s and 3s, and k = 1, 3, 5, 7 and 15. The zipc
668
18. High-Dimensional Problems: p
N
rst dierences to the left, right, above and below the target pixel. This
can be useful for denoising or classifying images. Friedman et al. (2007)
develop fast generalized coordinate descent algorithms for the one- a
ESL Chapter 3 Linear Methods for Regression
Trevor Hastie and Rob Tibshirani
Linear Methods for Regression
Outline
The simple linear regression model
Multiple linear regression
Model selection and shrinkagethe state of the art
1
ESL Chapter 3 Linear Me
ESL Chapter 5 Basis Expansions and Regularization
Trevor Hastie and Rob Tibshirani
Basis Expansions and Regularization
For a vector X , we consider models of the form
M
f (X ) =
m hm (X )
m=1
Examples of hm :
2
hm (X ) = Xj , Xj X , . . .
hm (X ) = |X |
ESL Chapter 4 Linear Methods for Classication
Trevor Hastie and Rob Tibshirani
Linear Methods for Classication
Linear regression
linear and quadatric discriminant functions
example: gene expression arrays
reduced rank LDA
logistic regression
separat
Stats 315A HW1 Solutions
6th February, 2012
If there are any questions regarding the solutions or the grades of HW 1, please contact
Gourab (gourab@stanford.edu) with Stats315A-hw1-grading in the subject line. Common mistakes are highlighted in blue.
Grad
Statistics 315a
Midterm Exam
4:15-5:30pm, February 29, 2012.
You may use the class text, notes and calculators, but not computers or any
device that connects to the Internet. The questions below require fairly short
answers and are of equal value. They ar
Stats 315A HW2 Solutions
8th February, 2012
If there are any questions regarding the solutions or the grades of HW 2, please contact
Austen (ahead@stanford.edu) with Stats315A-hw2-grading in the subject line.
Grade Distribution: Total 100 Points
Problem 1