lecture19 - Lucent Technologies Bell Labs Innovations...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Lucent Technologies Bell Labs Innovations Image­Based Rendering to Image­Based Rendering to Accelerate Interactive Walkthroughs Daniel G. Aliaga Visual Communications Research Lucent Technologies Bell Labs <aliaga@bell­labs.com> 3D Models 2.0M tris 1.7M tris 1.0M tris 0.9M tris E2 Why Use Images? • Independent of scene complexity 640x480 pixels 640x480 pixels E3 Flight Simulators • Mid­1980’s – E&S CT­6 one of first to use real­time photo textures • Hand­selected objects: – Terrain, trees, airplanes, buildings, etc. • 30­60 Hz – High visual fidelity E4 Overview • Replacing Geometry with Images • Displaying Images – – – Texture­mapping and error metrics Geometry and image warping Meshes, Lightfield/Lumigraph • Image Placement – Automatically Bounding Model Complexity – Cells and Portals • Conclusions E5 Replacing Geometry with Images • Algorithm – – – Select subset of model Create image of the subset Cull subset and replace with image • Why? – Image displayed in (approx.) constant time – Image reused for several frames E6 Simple Example E7 Simple Example E8 Simple Example E9 Overview • Replacing Geometry with Images • Displaying Images – – – Texture­mapping and error metrics Geometry and image warping Meshes, Lightfield/Lumigraph • Image Placement – Automatically Bounding Model Complexity – Cells and Portals • Conclusions E10 Geometric Discontinuity • If we move from the center­of­projection, discontinuities appear at the border E11 Temporal Discontinuity • While moving, if we switch between geometry and image, a sudden pop occurs E12 Approaches • Geometric and Temporal Continuity – – – – Error metrics Geometry warping Image warping Lightfield/Lumigraph E13 Error Metrics • Use an error metric to control amount of discontinuity c1 α c2 [Maciel95][Shade96][Schaufler96] E14 Error Metric • Relies on “angular­deviation” measuring the visual quality of using the (same) image E15 Video Segment I • Pre­rendered Impostors – Maciel95 • Dynamic Image­Caching – Shade96, Schaufler96 E16 Geometry Warping [Aliaga96] E17 Geometry Warping E18 Geometry Warping E19 Geometry Warping E20 Geometry Warping E21 Geometry Warping E22 Geometry Warping E23 Geometry Warping E24 Geometry Warping Surrounding geometry warped (incorrect perspective) Correct perspective E25 Geometry Warping • Surrounding geometry warped to match image Viewpoint Center-of-projection Surrounding Geometry E26 Video Segment II • Geometry Warping – Aliaga96 E27 Image Warping • Change the image itself – Re­project the image to the current viewpoint • [Chen93][McMillan95][Max95][Shade98] – Display image as a (simplified, textured) mesh • [Darsa97][Sillion97] E28 Image Warping eye Reference COP • A raster scan of each sheet produces a back­to­front ordering of warped pixels E29 Image Warping • McMillan and Bishop’s Warping Equation x2 = δ(x1) P2-1 (c1 - c2) + P2-1 P1 x1 (c Move pixels based on distance to eye ~Texture mapping d c1 x1 x2 c2 E30 Example... • Image outlined in yellow • Viewed from image’s center­of­projection E31 3D Image Warp • Single sample per pixel E32 Layered Depth Image Warp • Multiple samples per pixel – Previous occlusions are filled­in [Popescu98] E33 Meshes • (Simplified) Textured Depth Mesh – Per­pixel depth creates mesh that approximates 3D parallax effects – Image is texture­mapped onto mesh E34 Video Segment III • 3D Image Warping – McMillan95 • Textured Meshes – Darsa97, Sillion97 E35 Lightfield/Lumigraph • Flow of light at all positions and directions – [Levoy96][Gortler96] • Large number of images are used as 2D slices of a 4D light function (u,v) (s,t) 36 E Video Segment IV • Light field – Levoy96 • Lumigraph – Gortler96 E37 Overview • Replacing Geometry with Images • Displaying Images – – – Texture­mapping and error metrics Geometry and image warping Meshes, Lightfield/Lumigraph • Image Placement – Automatically Bounding Model Complexity – Cells and Portals • Conclusions E38 Automatic Image­Placement • As a preprocess – Select geometry to replace with an image in order to limit the number of primitives to render for any frame • At run time – – Display selected geometry as a (depth) image Render remaining geometry normally E39 Automatic Image­Placement Model Automatic ImagePlacement Create Images Preprocess Run time Model Cull Geometry Replaced with Images Render Geometry and Images Display E40 Example Rendering Geometry + Image = Final Scene E41 - Overview - Image Placement - Displaying Images - Conclusions Key Observation • Example model • Too much geometry in view frustum Viewpoint E42 Key Observation • Geometry is replaced by image to limit the number of primitives to render Viewpoint E43 Key Observation Eye • Less geometry is in the view frustum from the eye than the one from the grid viewpoint Grid viewpoint E44 2k Recursive Subdivision Algorithm 2k+1 Even to odd 2k+1 2k+2 Odd to even E45 Example Grid 1 0.5 0 -0.5 -1 1 0.5 1 0 0.5 0 -0.5 -0.5 -1 Wireframe rendering -1 3D grid of 1557 viewpoints E46 Sample Path View Frustum Culling Images Present 700000 600000 Polygons 500000 400000 300000 200000 100000 0 0 50 100 Power Plant, 2M tris, SGI Onyx2 w/ IR 150 Frame 200 250 E47 300 Images Per Triangle Power Plant Torpedo Room Brooks House Pipes Best Case Worst Case Images Per Model Triangle 0.004 0.003 0.002 0.001 0 0 10 20 30 40 Maximum Rendered Triangles (% of Model) E48 50 Preprocessing Summary Model No. of Images Prep. Time Estimated (hours) Space (MB) 239-5815 1.2-21.7 156-3802 Power Plant (2M) Torpedo Room 181-2333 (850k) Brooks House 561-2492 (1.7M) 282- 893 Pipes (1M) 1.1-11.8 72- 933 11.4-28.4 388-1725 2.4- 4.6 175- 554 E49 Video Segment V • Automatically Bounding Geometric Complexity by Using Images – Aliaga99 E50 Cells and Portals [Airey90, Teller91, Luebke95] E51 Portal Images [Aliaga97] E52 Creating Portal Images Ideal portal Ideal image would be one sampled from the current eye position eye eye portal E53 Creating Portal Images Reference COPs Display one of a large number of pre-computed images (~120) portal E54 Creating Portal Images Reference COPs or… Warp one of a much smaller number of reference images portal E55 Brooks House View Frustum Culling Portal Culling Static Portal Images 0.5 0.45 0.4 Rendering Time (sec.) 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 0 Onyx IR 50 100 Frame 150 200 E56 Brooks House Portal Culling Warped Portal Images Rendering Time (sec.) 0.5 0.4 0.3 0.2 0.1 0 0 100 200 300 400 Frame Onyx IR E57 500 Video Segment VI • Architectural Walkthroughs using Portal Images – Aliaga97, Rafferty98 E58 Overview • Replacing Geometry with Images • Displaying Images – – – Texture­mapping and error metrics Geometry and image warping Meshes, Lightfield/Lumigraph • Image Placement – Automatically Bounding Model Complexity – Cells and Portals • Conclusions E59 Image Quality • What about measuring quality? – Need a perceptual quality metric! • We know – Texture­mapping: bad perspective, small distortions believable (geometry warping) – IBR: correct perspective, disocclusions – Meshes: stretching of skins E60 Limitations • Diffuse illumination – Deferred shading? • Static models – Incremental updating? • Cannot sample all visible surfaces – Smarter reconstruction/resampling? • Can only sample surfaces at a fixed resolutions – Multi­resolution reference images? E61 Acknowledgments • Authors of the Video Segments • Models – Discreet Logic, UNC Walkthrough Group • UNC­Chapel Hill – Walkthrough, PixelFlow, ImageFlow • NSF, NIH, DARPA • Lucent Technologies Bell Labs E62 ...
View Full Document

This note was uploaded on 12/09/2011 for the course SP 108 taught by Professor Whittenburg during the Summer '11 term at Montgomery College.

Ask a homework question - tutors are online