audio-haptics - 3D User Interface Hardware 3D Lecture #5:...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 3D User Interface Hardware 3D Lecture #5: Audio/Haptic Displays Spring 2008 Joseph J. LaViola Jr. Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Interaction Workflow Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 1 Introduction To Displays Display: device which presents perceptual Display information Goal: display devices which accurately Goal: represent perceptions in simulated world Displays do not have to be just visual Displays auditory auditory haptic, tactile haptic, olfactory olfactory Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Lecture Outline Audio Displays Audio 3D sound localization depth cues 3D 3D sound generation 3D sound configurations sound audio and 3DUIs audio Haptic/Tactile Displays Haptic/Tactile haptic cues haptic display characteristics display display types display haptics and 3DUIs haptics Olfactory Displays Olfactory Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 2 Audio Displays Present sound to the user Present Spatialized 3D sound Spatialized sound surrounds user sound take advantage of localization take Localization – psychoacoustic Localization process of determining location and direction from which a sound emanates Many applications Many Spring 2008 http://www.loonygames.com/content/2.4/feat/ CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3D Sound Localization – Binaural Cues Comparison of sound waves Comparison received by each ear Interaural time difference – Interaural time between arrival of sound at each ear Interaural intensity Interaural difference – difference in sound intensity at each ear Cue does have problems Cue http://www.jeroenbreebaart.com/research_binaural.htm ambiguous situations ambiguous when two or more sources when where ITD and IID are the same Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3 3D Sound Localization – Spectral and Dynamic Cues Dynamic head movement or sound source Dynamic moving source is fairly weak cue moving Utilize spectral content Utilize interaction of sound wave with outer ear interaction occur at relatively high frequencies (above occur 6KHz) Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3D Sound Localization – Head Related Transfer Functions Spatial filters that describe Spatial how sound waves interact with listener’s body listener specific listener lack reverberation info lack Build in echo free Build chamber with head model Spring 2008 http://www.tonmeister.ca/main/textbook/node320.html CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 4 3D Sound Localization – Reverberation Many factors affect a sound source Many objects objects atmospheric properties atmospheric Sound hits listener directly and indirectly Sound Aids in perception of distance, not direction Aids Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3D Sound Localization – Others Intensity (i.e., loudness) Intensity simple cue simple common in 3D audio displays common Vision and Environment Vision sounds in FOV make spatial percepts sounds Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 5 3D Sound Generation 3D sound sampling and synthesis 3D take samples from real environment take binaural audio recording binaural generate HRTFs for each ear generate Auralization Auralization rendering a sound field rendering good for reverberation effects good wave-based modeling waveray-based modeling raySpring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Sound System Configurations Headphones (stereophonic) Headphones High level of channel separation High avoid crosstalk avoid isolate listener from external sounds isolate problems with inside the head localization problems External Speakers External place speakers around the room place no need to wear anything no problems with crosstalk problems Two approaches –transaural audio and amplitude Two panning Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 6 Audio in 3DUIs Localization Localization wayfinding aids wayfinding Sonfication Sonfication turning information into sounds turning Ambient Effects Ambient adding realism adding Sensory Substitution Sensory sound for touch sound Annotation and Help Annotation Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Haptic Displays Provide user with sense of Provide touch simulate interaction simulate between virtual objects and user force (joint/muscle) force tactile (skin-based) tactile Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality www.novint.com ©Joseph J. LaViola Jr. 7 Haptic Cues – Tactile Cues Taction – perceived by variety of Taction cutaneous receptors under skin surface surface texture surface temperature temperature pressure pressure pain pain Found in varying concentrations on the Found body Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Haptic Cues – Kinesthetic Cues Perceived by receptors in muscles, joints Perceived and tendons help to determine movement, position and help torque of different body parts relationship between user and object via relationship muscular tension Both active (movement self-induced) and Both passive (movement by external force) Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 8 Haptic Display Characteristics Presentation capability – what types of Presentation output Resolution Resolution Spatial – minimum proximity of stimuli Spatial Temporal – refresh rate Temporal Ergonomics Ergonomics Don’t want to break anyone Don Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Haptic and Tactile Displays “For every action there is an equal and opposite reaction” reaction” Sir Isaac Newton Sir Main forms of feedback Main ground referenced ground body referenced body tactile tactile combination combination passive – physical props passive Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 9 Haptic Displays Motionware device Motionware Provides vestibular Provides stimulation Sends signals to the Sends 8th cranial nerve Gives user a sense of Gives motion No longer exits No Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Other Haptic Devices Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 10 PainPong PainStation PainStation two person console two electro shock, heat, electro whipping: bad performance pain pain PainStation - Pong Courtesy of www.painstation.de Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Hall of Pain (www.painstation.de) Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 11 Haptics – Interface Design Useful for object Useful manipulation Problem with these Problem devices is they are very intimidating Mimic real world Mimic interaction Untapped area for Untapped games scary too!! scary Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Olfactory Output Least developed area Least maybe for good reason! maybe Has practical applications Has fire fighting fire surgical training surgical Number of practical problems Number Spring 2008 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 12 Next Class Input Device Hardware Input Readings Readings 3DUI Book – Chapter 3, pages 59-86 3DUI 59- Spring 2008 CAP6938 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 13 ...
View Full Document

This note was uploaded on 06/13/2011 for the course CAP 6938 taught by Professor Staff during the Spring '08 term at University of Central Florida.

Ask a homework question - tutors are online