Unformatted text preview: Augmented Reality 1 Definition of Augmented Reality Virtual Environments (VE): Completely replaces the real world Augmented Reality (AR): User sees real environment; combines virtual with real Supplements reality, instead of completely replacing it Photorealism not necessarily a goal 2 Differences between VR and AR environments Virtual Reality immerse a user inside a virtual world that completely replaces the real world outside. AR augments the user's view of the real world by composing 3D virtual objects with their real world counterparts, necessitating that the user maintains a sense of presence in that world.
3 Augmented Reality (AR) A growing area in VR research It is a form of humanmachine interaction where information is presented in the file of view of an individual augmenting the real world scene necessitating that the user maintains a sense of presence in that world. the virtual images are merged with the real view to create the augmented display. 4 5 Characteristics
1. 2. 3. Blends real and virtual, in real environment Realtime interactive Registered in 3D Applies to all senses (auditory, haptic?) Not an HMDspecific definition Includes idea of removing part of real environment 6 Key Technologies of AR accurate registration of the virtual objects with the real world image
camera position, viewing parameters digital composition, scene lighting, object texture mapping realtime processing constrain user viewing method 7 must be related in an augmented reality system Figure shows the multiple reference frames that register the virtual frame of reference with what the user is seeing 8 See diagram in previous slide The scene is viewed by an imaging device, which in this case is depicted as a video camera. The camera performs a perspective projection of the 3D world onto a 2D image plane. The virtual objects are modeled in an object reference frame. The graphics system requires information about the imaging of the real scene so that it can correctly render these objects. This data will control the synthetic camera that is used to generate the image of the virtual objects. This image is then merged with the image of the real scene to form the augmented reality image 9 Display technologies in AR Systems
1. 2. 3. Monitor Based Augmented Reality Video Seethrough Augmented Reality Display Optical Seethrough Augmented Reality Display Uses to categorize augmented reality systems
10 Monitor Based Augmented Reality combining of real and virtual images into a single image pose technical challenges to AR designers Monitorbased AR provides a simplest approach 11 Video Seethrough Augmented Reality Display The architecture is similar to previous except that now user has a heightened sense of immersion in the display. The user's view of the real world is provided for the video cameras. The scene generator creates graphic images that are combined with the video, merging the real and virtual. The result is sent to the monitors.
12 Video Seethrough Augmented Reality Display 13 The video composition can be done through Video Seethrough Augmented Reality Display Chromakeying. The background of the images is set to a specific color, for example red, which none virtual objects use. Then, the combining set replaces all red areas with the corresponding parts from video of the real world. The result is then superimposing the virtual objects over the real world. If the system had depth information at each pixel for the real world images, this method could combine the real and virtual images by a pixelbypixel depth comparison. This would allow real objects to cover virtual objects and vice versa.
14 Depth Information. Optical SeeThrough HMD One way to implement Augmented Reality is with an optical seethrough Head Mounted Display. Eliminates the video channel Instead, the merging of real world and virtual augmentation is done on the HMD display, rather than the cockpit window etc This device places optical combiners in front of the user's eyes. The combiners let light in from the real world, and they also reflect light from monitors displaying graphic images. The result is a combination of the real world and a virtual world drawn by the monitor 15 Optical SeeThrough HMD 16 Applications of Augmented Reality 17 Finger Tracking Finger Tracking is one of the simplest application of Augmented Reality, The computer can visually track the user's finger, there is no need to use a pen, a mouse or other devices 18 Annotation and visualization AR could be used: to annotate objects and environments with public or private information to aid general visualization tasks. One might be able to look out a window and see how a proposed new building would change her view. 19 Augmented Museum The system detects the ID of a picture, and generates a description of it. NaviCam identifies which picture the user is looking at and displays relevant information on the screen.
20 Medical This domain is viewed as one of the more important for AR Systems. AR could aid the doctors in the visualization and training for surgery. AR may provide an internal view of the real patient. Through noninvasive sensors like Magnetic Resonance Imaging (MRI), Computed Tomography scans (CT), or ultrasound imaging, 3D datasets of a patient in real time could be collected. These datasets are rendered and combined in real time with a view of the patient, giving a "XRay vision" of the patient for doctor.
21 Enhanced Reality Visualization We can peel back the MRI skin and see where the internal structures are located relative to the viewpoint of the camera. Thus the surgeon has xray vision, a capability which will be needed more and more as we continue moving towards minimallyinvasive surgeries. 22 Medical "Xray vision" for surgeons Aid visualization, minimallyinvasive operations. Training. MRI, CT data. Ultrasound project, UNC Chapel Hill.
Courtesy UNC Chapel Hill 23 Medical AR may be used for general medical visualization in the surgical room. The information from the noninvasive sensors would be directly displayed on the patient, showing exactly where to perform the operation. AR may be used for Training purposes (see next slide).
24 Medical Augmentedreality based therapeutic device designed to overcome gait (a way of walking) problems associated with Parkinson's Disease (PD). A normal model of gait is proposed followed by a model of Parkinsonian gait with the goal of construction of a gait enabling therapeutic device. The fundamental underlying belief of the model is that vision pathology is responsible for the majority of Parkinsonian gait pathology. 25 Medical Liver surgery planning system (TU Graz)
Seethrough HMD tracked PEN and tracked Personal Interaction Panel (transparent plexiglass) 26 Medical MedARPa
Freely moveable AR display Multimodal interaction technologies Foot switch visual and audio feedback 27 Manufacturing and repair AR technology could provide instructions that might be easier for user to understand an equipment. These instructions are not available in manuals with text and pictures, but as superimposed 3D drawings upon the actual equipment. These drawings show stepbystep the tasks that need to be done and how to do them. Instructions for assembly, maintenance and repair of complex equipment: Aircraft Printers Engines Automobile assembly 28 Manufacturing and repair Boeing wire harness assembly. Adam Janin wearing HMD. Courtesy David Mizell, Boeing Prototype of the application, displaying how to remove the paper tray 1993 S. Feiner, B. MacIntyre, & D. Seligmann, Columbia University 29 The Touring Machine Augmentedreality displays will overlay computer generated graphics onto the real world 30 Mobile Augmented Reality Systems Explore the synergy of two promising fields of user interface research: Augmented reality (AR), in which 3D displays are used to overlay a synthesized world on top of the real world, and mobile computing, in which increasingly small and inexpensive computing devices and wireless networking allow users to have access to computing facilities while roaming the real world 31 ARToolKit A software library for building AR applications Open Source, multiplatform These applications involves overlays 3D virtual objects on real markers (in real world) Uses single tracking marker Determines camera pose information (6 DOF) Includes utilities for markerbased interaction ARToolKit Website http://www.hitl.washington.edu/artoolkit/
32 ARToolKit Free computer vision based tracking library
33 Tangible Interaction ARToolKit supports physically based interaction
34 Research Activities in AR Revolves around the following issues: To develop methods to register two distinct sets of images and keep them registered in real time In display technology for merging both the real and computer generated images. AR System performance 35 Also note AR is an Ultimate Computer Science problem! Computer Vision Computer Graphics Human/Computer Interaction Database Networking Security 36 Reference Ronald T. Azuma, A Survey of Augmented Reality, In Presence: Teleoperators and Virtual Environments 6, 4 (August 1997), 355385. (web) What is Augmented Reality, Kiyoshi Kiyokawa, HIT Lab, University of Washington (web) Augmented Reality, Daniel Thalmann, EPFL VRlab (web) Introduction to Augmented Reality, http://www.se.rit.edu/~jrv/research/ar/introduction.ht ml#Section1.3.3
37 Project 3 (Individual 15%) You are required to explore the area of augmented reality research/applications and write a report (3000 5000 words) on any of its topic that interest you the most. Then based on your written report, you are required to produce around 15 power point slides. Project must be completed before 13 October 2006. Report (hardcopy) Power point slides (softcopy) Assessment is based on the following criteria: Relevancy and accuracy 7% Content Coherent (logical, well organised, easy to understand and clear) 8%
View Full Document
- Winter '09
- Augmented reality