This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Machine Learning Srihari 1 Graphical Models Sargur Srihari [email protected] Machine Learning Srihari 2 What are Graphical Models? • They are diagrammatic representations of probability distributions – marriage between probability theory and graph theory • Also called probabilistic graphical models • They augment analysis instead of using pure algebra Machine Learning Srihari 3 What is a Graph? • Consists of nodes (also called vertices) and links (also called edges or arcs) • In a probabilistic graphical model – each node represents a random variable (or group of random variables) – Links express probabilistic relationships between variables Machine Learning Srihari 4 Probability Theory • Probability theory can be expressed in terms of two simple equations – Sum Rule • probability of a variable is obtained by marginalizing or summing out other variables – Product Rule • joint probability expressed in terms of conditionals ) ( )  ( ) , ( ) , ( ) ( a p a b p b a p b a p a p b = = ∑ All probabilistic inference and learning amounts to repeated application of sum and product rule Machine Learning Srihari 5 Graphical Models in Engineering • Natural tool for handling Uncertainty and Complexity – which occur throughout applied mathematics and engineering • Fundamental to the idea of a graphical model is the notion of modularity – a complex system is built by combining simpler parts. Machine Learning Srihari 6 Why are Graphical Models useful in Engineering? • Probability theory provides the glue whereby – the parts are combined, ensuring that the system as a whole is consistent – providing ways to interface models to data. • Graph theoretic side provides: – Intuitively appealing interface • by which humans can model highlyinteracting sets of variables – Data structure • that lends itself naturally to designing efficient generalpurpose algorithms Machine Learning Srihari 7 Graphical models: Unifying Framework • View classical multivariate probabilistic systems as instances of a common underlying formalism – mixture models, factor analysis, hidden Markov models, Kalman filters and Ising models – Encountered in systems engineering, information theory, pattern recognition and statistical mechanics • Advantages of View: – Specialized techniques in one field can be transferred between communities and exploited – Provides natural framework for designing new systems Machine Learning Srihari 8 Role of Graphical Models in ML 1. Simple way to visualize structure of probabilistic model 2. Insights into properties of model Conditional independence properties by inspecting graph 3. Complex computations required to perform inference and learning expressed as graphical manipulations Machine Learning Srihari 9 Graph Directionality • Directed graphical models – directionality associated with arrows • Bayesian networks – Express causal relationships between random variables • More popular in AI...
View
Full Document
 Spring '09
 Machine Learning, Probability theory, graphical models

Click to edit the document details