Summary
Structured representations are important
Abstract
Recursive
Generative
New primitive concepts can be learned
Learning the most parsimonious theory
How to combine structured representations and
statistical inference?
Statistical parsing in l
Outline for today
Learning a theory and new concepts in firstorder logic.
The debate about structure in peoples
mental representations of concepts
Hierarchies or hidden units?
Logical relations or hidden units?
Definitions or prototypes?
First-order
Outline for today
Grammars
First-order logic
Learning a theory and new concepts in firstorder logic.
Finite-state grammar
The minimal linguistic theory.
lucky
tired
s2
s0
the
a
boy
girl
saw
tasted
treats
defeat
s1
s3
s4
s5
Figure by MIT OCW.
E.g., The
Where weve been, where were going
Two classes ago: classic model of learning
concepts based on combinations of features.
[Is it really learning? Fodors challenge.]
Theories of when learning is possible:
Identifiability in the limit: Subset principle
Goodmans problem
Why do some hypotheses receive
confirmation from examples but not others?
All piece of copper conduct electricity: yes
All men in this room are third sons: no
Distinguishing lawlike hypotheses from
accidental hypotheses is not easy:
9.66 / 9.914 Computational
Cognitive Science
Josh Tenenbaum
What is this class?
An attempt to see how recent work in
computation (AI, machine learning,
statistics) can inform some of the core
questions of cognitive science.
and vice versa.
The questions
Outline
Bayesian concept learning: Discussion
Probabilistic models for unsupervised and
semi-supervised category learning
Discussion points
Relation to Bayesian classification?
Relation to debate between rules / logic /
symbols and similarity / co
Outline
Limits of Bayesian classification
Bayesian concept learning
Probabilistic models for unsupervised and
semi-supervised category learning
Limitations
Is categorization just discrimination among mutually
exclusive classes?
Overlapping concepts?
Problem sets
Late policy (5% off per day, but the
weekend counts as only one day). E.g.,
Friday: -5%
Monday: -15%
Tuesday: -20%
Thursday: -30%
Outline
Final thoughts on hierarchical Bayesian
models and MCMC
Bayesian classification
Bayesian concept lea
Outline
Bayesian parameter estimation
Hierarchical Bayesian models
Metropolis-Hastings
A more general approach to MCMC
Coin flipping
Comparing two simple hypotheses
P(H) = 0.5 vs. P(H) = 1.0
Comparing simple and complex hypotheses
P(H) = 0.5 vs. P
Compression in Bayes nets
A Bayes net compresses the joint
probability distribution over a set of
variables in two ways:
Dependency structure
Parameterization
Both kinds of compression derive from
causal structure:
Causal locality
Independent causal
So.
why do we keep having this debate:
rules/symbols vs. prototypes/connections?
So.
The real problem: a spurious contest between
logic and probability.
Neither logic nor probability on its own is
sufficient to account for human cognition:
Generativity
Knowledge Representation:
Spaces, Trees, Features
Announcements
Optional section 1: Introduction to Matlab
Tonight, 8:00 pm
Problem Set 1 is available
The best statistical graphic ever?
Image removed due to copyright considerations. Please see:
Tufte, Edw