Inference in Markov Networks
Doug Downey
Northwestern EECS 395/495 Fall 2011
Markov Network Inference
P(x) = c c(xc)
Z
Z = x c c(xc)
Grades
Trivia
Knowledge
TV
Grades
TV
(A, B)
Low
Little
2.0
Good
Little
Low
Good
TV
Trivia
Knowledge
(A, B)
3.0
Little
Lit
Todays lecture
Exact inference in graphical models.
Variable Elimination
Elimination as Graph Transformation
Treewidth
Sum-product belief propagation
Max-product belief propagation
Junction tree algorithm (during tutorial)
Figures from Koller book as well
Quantum Deep Learning
Nathan Wiebe, Ashish Kapoor, and Krysta M. Svore
Microsoft Research, Redmond, WA (USA)
arXiv:1412.3489v1 [quant-ph] 10 Dec 2014
In recent years, deep learning has had a profound impact on machine learning and artificial
intelligence.
A Scalable Lock-free Stack Algorithm
Danny Hendler
Nir Shavit
Lena Yerushalmi
School of Computer Science
Tel-Aviv University
Tel Aviv, Israel 69978
Tel-Aviv University &
Sun Microsystems
Laboratories
School of Computer Science
Tel-Aviv University
Tel Aviv
DATA SCIENCE
FELLOWS PROGRAM
Employers are increasingly looking to an elite program
called the Insight Data Science Fellows Program.
Big Data's High-Priests of Algorithms
The Wall Street Journal | August 2014
Insight is an intensive, seven week postdocto
3-Colouring graphs without triangles or induced paths on seven
vertices
Flavia Bonomo1 , Maria Chudnovsky2 , Peter Maceli2 , Oliver Schaudt3 , Maya Stein4 ,
and Mingxian Zhong2
1
CONICET and Departamento de Computaci
on, FCEN, Universidad de Buenos Aires,
arXiv:1312.6199v4 [cs.CV] 19 Feb 2014
Intriguing properties of neural networks
Christian Szegedy
Wojciech Zaremba
Ilya Sutskever
Joan Bruna
Google Inc.
New York University
Google Inc.
New York University
Dumitru Erhan
Ian Goodfellow
Rob Fergus
Google Inc.
Playing Atari with Deep Reinforcement Learning
Volodymyr Mnih
Koray Kavukcuoglu
David Silver
arXiv:1312.5602v1 [cs.LG] 19 Dec 2013
Daan Wierstra
Alex Graves
Ioannis Antonoglou
Martin Riedmiller
DeepMind Technologies
cfw_vlad,koray,david,alex.graves,ioanni
version 1.1
April 1st, 06
m moves the cursor, or denes
the range for an operator
direct action command,
if red, it enters insert mode
0 erator requires a motion afterwards,
p operates between cursor &
destination
special functions,
e ra requires extra inp
The First Level of Super Mario Bros. is Easy with Lexicographic
Orderings and Time Travel . . . after that it gets a little tricky.
Dr. Tom Murphy VII Ph.D.
1 April 2013
Abstract
paper is mainly as a careful record of the current status for repeatability
Java theory and practice: Thread pools and work
queues
Thread pools help achieve optimum resource utilization
Brian Goetz ([email protected])
Principal Consultant
Quiotix Corp
01 July 2002
One of the most common questions posted on our Multithreaded Java
Java Caching with Guava
Charles Fry ([email protected])
Google Confidential and Proprietary
Introduction
The Guava project is an open-source release of
Google's core Java libraries
Stuff like collections, primitives support, concurrency
libraries, string p
Practice Problems for CSC 412/2506 Midterm
1. Let p(k) be a one-dimensional discrete distribution that we wish to approximate, with support on
nonnegative integers. One way to fit an approximating distribution q(k) is to minimize the KullbackLeibler diver
Todays lecture
Approximate inference in graphical models.
Forward and Backward KL divergence
Variational Inference
Mean Field: Naive and Structured
Marginal Polytope
Local Polytope
Relaxation methods
Loopy BP
LP relaxations for MAP inference
Figures from
Practice Problems for CSC 412/2506 Midterm
1. Let p(k) be a one-dimensional discrete distribution that we wish to approximate, with support on
nonnegative integers. One way to fit an approximating distribution q(k) is to minimize the KullbackLeibler diver
The Trinity Tutorial
by Avi Kak
ML, MAP, and Bayesian The Holy
Trinity of Parameter Estimation and Data
Prediction
Avinash Kak
Purdue University
January 4, 2017
11:19am
An RVL Tutorial Presentation
originally presented in
Summer 2008
(minor changes in: Ja
CSC 412 (Lecture 4): Undirected Graphical Models
Raquel Urtasun
University of Toronto
Feb 2, 2016
R Urtasun (UofT)
CSC 412
Feb 2, 2016
1 / 37
Today
Undirected Graphical Models:
Semantics of the graph: conditional independence
Parameterization
Clique
Poten
Assignment #2
In this assignment, well fit both generative and discriminative models to the MNIST dataset of handwritten numbers. Each datapoint in the MNIST [http:/yann.lecun.com/exdb/mnist/] dataset is a
28x28 black-and-white image of a number in cfw_0
CSC 412/2506 Spring 2017
Probabilis7c Graphical Models
Lecture 2: Genera7ve Classiers
Slides based on Rich Zemels
All lecture slides will be available on the course website:
www.cs.toronto.edu/~duvenaud/courses/CS412
Some of the Aigures ar
Implementing
autograd
Slides by Matthew Johnson
Autograds implementation
github.com/hips/autograd
Dougal Maclaurin, David Duvenaud, Matt Johnson
differentiates native Python code
handles most of Numpy + Scipy
loops, branching, recursion, closures
arrays,
Introduction to Probability for
Graphical Models
CSC 412
Kaustav Kundu
Thursday January 14, 2016
*Most slides based on Kevin Swerskys slides, Inmar Givonis slides, Danny
Tarlows slides, Jasper Snoeks slides, Sam Roweis s review of probability,
Bishops boo
Matrix Approach to Linear
Regression
Dr. Frank Wood
Frank Wood, [email protected]
Linear Regression Models
Lecture 11, Slide 1
Random Vectors and Matrices
Lets say we have a vector consisting of three
random variables
The expectation of a random ve
CSC 412/2506 Spring 2017
Probabilis7c Graphical Models
Lecture 3: Directed Graphical Models
and Latent Variables
Based on slides by Richard Zemel
Joint Probabili7es
Chain rule implies that any joint distribution equals
p(x1:D ) = p(x1
No more mini-languages:
Autodiff in full-featured Python
David Duvenaud, Dougal Maclaurin, Matthew Johnson
Our awesome new world
TensorFlow, Stan, Theano, Edward
Only need to specify forward model
Autodiff + inference / optimization done for you
Our aw
Variational Inference
for Machine Learning
Shakir Mohamed
DeepMind
shakirm.com
@shakir_za
18 February 2015, Imperial College, London
Abstract
Variational inference is one of the tools that now lies at the heart of the modern data analysis lifecycle.
Varia
CSC321 Lecture 6: Backpropagation
Roger Grosse
Roger Grosse
CSC321 Lecture 6: Backpropagation
1 / 21
Overview
Weve seen that multilayer neural networks are powerful. But how can
we actually learn them?
Backpropagation is the central algorithm in this cour
CS281B/Stat241B: Advanced Topics in Learning & Decision Making
Linear & Ridge Regression and Kernels
Lecturer: Michael I. Jordan
1
Scribes: Dave Latham
Kernel Definitions Reviewed
Let us review the denition of a kernel function.
The denition given before
Assignment #1
Due: 11:59pm February 10th, 2017
Problem 1 (Variance and covariance, 6 marks)
Let A and B be two continuous independent random variables.
(a) Starting from the definition of independence, show that if A and B are independent then their
covar
Tutorial: Stochastic Variational Inference
David Madras
University of Toronto
March 16, 2017
David Madras (University of Toronto)
SVI Tutorial
March 16, 2017
Variational Inference (VI) - Setup
Suppose we have some data x, and some latent variables z (e.g.
Bayesian Optimization
CSC 412/2506 Tutorial
Geoffrey Roeder
Mar 31, 2017
Slides from Kevin Swersky, Nando de Freitas
Course of tutorial
What problem are we solving with BayesOpt?
Gaussian Process review: notebook
Acquisition functions
EI Example
What
How to build an automatic statistician
James Robert Lloyd1 , David Duvenaud1 , Roger Grosse2 ,
Joshua Tenenbaum2 , Zoubin Ghahramani1
1: Department of Engineering, University of Cambridge, UK
2: Massachusetts Institute of Technology, USA
November 19, 2014
CSC 412/2506 Spring 2016
Probabilis6c Graphical Models
Lecture 7: Sampling
Based on slides by Richard Zemel
Dicult Inference Problems
Undirected graphical model
Belief propagation is fast if argument lists of js are small,
and form