CS178 Homework #1 Solution
Machine Learning & Data Mining: Winter 2014
Problem 1: Getting connected
Hopefully you did this.
Problem 2: Data Exploration
iris = load ( 'data/iris.txt' ); % load the text file
y = iris (: , end );
% target value is last colum
CS178 Homework #3 Solution
Machine Learning & Data Mining: Winter 2014
Problem 1: Logistic Regression
(a) Plotting the data:
% Run the code provided to extract the data into XA, XB
figure(1); plotClassify2D([],XA,YA); % plot XA
figure(2); plotClassify2D([
CS178 Homework #2 Solution
Machine Learning & Data Mining: Winter 2014
Problem 1: Bayes Classiers
(a) You can most easily do this by hand, but since I have to type it I will put it in Matlab format:
p_y = 4/10;
% p(y) = 4/10
%p(xi  y=1)
p_x1_y0 = 3/6; p
Machine Learning & Data Mining: Winter 2015
Due: Tuesday January 13th, 2015
Write neatly (or type) and show all your work!
This may be your rst homework using
Matlab
tutorials to help you start using it. In matlab, help
; please see the course webpage for
Reinforcement Learning
Machine Learning & Data Mining
Prof. Alexander Ihler
Notes
Due
HW5 due Friday
Kaggle uploads close Sunday
Reports due Tuesday
Discussion Thursday & Lecture Friday
Review for Final
Bonus points
Peformance on Ka
Linear Regression
What is Linear Regression?
Linear Regression: an approach for modeling the relationship between target y and one
or more features x.
Benefits
Linear Regression is a useful technique for protection that will also help us illustrate
many
Linear Classification
Perceptron Classifier (2 features)
Compute the linear response and then convert the output into a discrete class by taking
the sign of a linear response: f =1 X 1 +2 X 2+ 0=T ( f ) =^c (x)

c^ ( x )= cfw_1,+1
Binary Output Predicti
Nearest Neighbor methods
Nearest neighbor regression
Find training data x(i) closes to x(new); predict y(i)
If we evaluate that procedure at each possible future x, we get this function:
 Whenever were close to a data point, we predict its y value the mo
Decision Trees
Decision Trees
Decision Trees: represents a function consisting of a series of comparisons or if/else
statements
 Each branch may lead to:
Another comparison and branch
An output value
The Tree Diagram
 Leaf nodes (red and blue circles)
Regularization
Regularization
Can modify our cost function J to add preference for certain parameter values
New Solution (derive the same way)
 Problem is wellposted for any degree
Ridge Regression: the overall effect to shrink the values towards 0 by
Gradient Descent
What is Gradient Descent?
Gradient Descent: a firstorder optimization algorithm that finds a local minimum of a
function by taking steps proportional to the negative ( ) of the gradient of the
function at the current point
How to change
Bias and Variance
Inductive Bias
Inductive Bias: the assumptions needed to predict examples we havent seen
Bias: makes us prefer one model over another
Examples
 Polynomial functions
 Smooth functions
Some bias is necessary for learning!
Inductive B
KNearest Neighbors
KNearest Neighbor (kNN) Classifier
Find the knearest neighbors to x in the data
1. Rank the feature vectors according to the Euclidean distance from the new point x
2. Select the k vectors which are have smallest distance to x
3. Us
CS178 Midterm Exam
Machine Learning & Data Mining: Winter 2017
Wednesday February 15th, 2017
Use Row/Seat:
Your name:
Your ID # and UCINetID:
(e.g., 123456789, [email protected])
Total time is 50 minutes. READ THE EXAM FIRST and organize your time; dont spe
Learning
Learning the Classifier Parameters
Learning from Training Data
 Training data = labeled feature vectors
 Find parameter values that predict well (low error)
error is estimated on the training data
true error will be on future test data
Defi
AI & Machine Learning
Artificial Intelligence (AI)
Generally considered to be a subfield of AI
Machine Learning (ML)
Is both more specific and broadly applied than AI
Focused on:
 Making predictions or decisions
 Getting better with experience
Is fu
+
Machine Learning and Data Mining
Linear regression
Prof. Alexander Ihler
Supervised learning
Notation
Features
x
Targets
y
Predictions
Parameters
Training data
(examples)
Features
Feedback /
Target values
Program (Learner)
Characterize
Final Review
Supervised Learning
Classification vs. Regression
 Classification
Discrete cfw_0, 1 or cfw_1, +1
Loss Error Rate
 Regression
Real Number 
Loss Mean Squared Error (MSE)
Overfitting
 Complexity
 Variance, Bias
Underfitting
 Simple

CS178 Homework #4 Solution
Machine Learning & Data Mining: Winter 2015
Problem 1: Decision Trees
You can most easily do this by hand, but since I have to type it I will put it in Matlab format:
Xy = [0 0 1 1 0 1
1 1 0 1 0 1
0 1 1 1 1 1
1 1 1 1 0 1
0 1
CS178 Homework #1 Solution
Machine Learning & Data Mining: Winter 2015
Problem 0: Getting connected
Hopefully you did this.
Problem 1: Data Exploration
iris = load ( 'data/iris.txt' ); % load the text file
y = iris (: , end );
% target value is last colum
08178 Midterm Exam
Machine Learning 85 Data Mining: Winter 2015
Tuesday February 10th, 2014
Your name:
oo obk
Your UCINetllD (e.g., mynameuci.edu):
Your seat (row and number):
o Total time is 80 minutes. READ THE EXAM FIRST and organize your time; dont sp
5'.
08178 Midterm Exam
Machine Learning & Data Mining: Winter 2017
Wednesday February 15th, 2017
Your name: S O KQbs 3 Use Row/ Seat:
ucf
Your ID # and UCINetID:
(e.g., 123456789, mynam'@uci.edu)
emafgr Qg . (Ah
0 Total time is 50 minutes. READ THE EXAM F
08178 Final Exam
Machine Learning & Data Mining: Winter 2016
Thursday March 17th, 2016
Your name: gbwqbp
Your ID Number and UCINetID ,
(e.g., 12345678 / mynameuci.edu): 3H 41.0 /PM+MO QM' '
Your seat row and number :
0 Total time is 1 hour 50 minutes. RE
CS178 Final Exam
Machine Learning & Data Mining: Winter 2016
Thursday March 17th, 2016
Your name:
Your ID Number and UCINetID
(e.g., 12345678
/
[email protected]):
Your seat (row and number):
Total time is 1 hour 50 minutes. READ THE EXAM FIRST and organize
VC Dimension
Learners and Complexity
Weve seen many versions of underfit/overfit tradeoff
 Complexity of the learner = Representational Power
Its ability to
o Learn a wide variety of inputoutput relationships
o Memorize or overfit to the data
Differ
+
Machine Learning and Data Mining
Clustering (1): Basics
Prof. Alexander Ihler
Unsupervised learning
Supervised learning
Predict target value (y) given features (x)
Unsupervised learning
Understand patterns of data (just x)
Useful for many reasons
+
Machine Learning and Data Mining
Ensembles of Learners
Prof. Alexander Ihler
Ensemble methods
Why learn one classier when you can learn many?
Ensemble: combine many predictors
(Weighted) combina<ons of predictors
May be same type of learner or d
C HAPTER
1
FT
Introduction
D
RA
Machine learning began as an offshoot of the field of artificial intelligence, built around the idea that an
intelligent system should improve with experience, over time. However, it has since found numerous
applications in
C HAPTER
4
FT
Linear Regression
RA
In this chapter, we focus on the regression task, specifically applying one of the simplest possible regression models: the linear model. We use this class of functions to explore a number of fundamental
tools that will