CS223B-JHeitz-Guest

CS223B-JHeitz-Guest - IntegratingVisionModelsfor...

Info iconThis preview shows pages 1–14. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Integrating Vision Models for  Holistic Scene  Understanding Geremy Heitz CS223B March 4 th , 2009
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 Scene/Image Understanding What’s happening in these pictures?
Background image of page 2
3 Human View of a “Scene” “A car passes a bus on the road, while people walk past a building.” ROAD BUILDING CAR BUS PEOPLE WALKING
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
4 Computer View of a “Scene” BUILDING ROAD STREET SCENE Can we integrate all of these subtasks,  so that whole > sum of parts ? 
Background image of page 4
5 Outline Overview Integrating Vision Models CCM: Cascaded Classification Models Learning Spatial Context TAS: Things and Stuff Future Directions [Heitz et al. NIPS 2008a] [Heitz & Koller ECCV 2008]
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
6 Image/Scene Understanding “a man and a dog are walking  on a sidewalk in front of a building” Man Dog Backpack Cigarette Primitives Objects Parts Surfaces Regions Interactions Context Actions Scene  Descriptions Established  techniques  address  these in  isolation. Reasoning  over image  statistics Complex web of  relations well  represented by  graphical models. Reasoning over  more abstract  entities. Building Sidewalk
Background image of page 6
7 Why will integration help? What is this object?
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
8 More Context Context is key!
Background image of page 8
9 Outline Overview Integrating Vision Models CCM: Cascaded Classification Models Learning Spatial Context TAS: Things and Stuff Future Directions [Heitz et al. NIPS 2008a]
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
10 Human View of a “Scene” ROAD BUILDING CAR BUS PEOPLE WALKING Scene Categorization Object Detection Region Labelling Depth Reconstruction Surface Orientations Boundary/Edge  Detection Outlining/Refined  Localization Occlusion Reasoning ...
Background image of page 10
11 Intrinsic Images [Barrow and Tenenbaum, 1978], [Tappen et al., 2005] Hoiem et al., “Closing the Loop in Scene Interpretation” , 2008 We want to focus more on “semantic” classes We want to be flexible to using outside models We want an extendable framework, not one engineered for a particular set of tasks Related Work = + =
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
12 How Should we Integrate? Single joint model over all variables Pros: Tighter interactions, more designer control Cons: Need expertise in each of the subtasks Simple, flexible combination of existing models Pros: State-of-the-art models, easier to extend Limited “black-box” interface to components DETECTION Dalal & Triggs, 2006 REGION LABELING Gould et al., 2007 DEPTH RECONSTRUCTION Saxena et al., 2007
Background image of page 12
13 DET 1 REG 1 REC 1 Cascaded Classification  Models Image Features f DET Object  Detection Region Labeling DET 0 Independent Models f REG REG 0 f REC REC 0 3D Reconstruction Context-aware Models
Background image of page 13

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 14
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 01/24/2010 for the course CS 223B taught by Professor Thrun,s during the Winter '09 term at Stanford.

Page1 / 53

CS223B-JHeitz-Guest - IntegratingVisionModelsfor...

This preview shows document pages 1 - 14. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online