audio-haptics - 3D User Interface Hardware Lecture #6:...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 3D User Interface Hardware Lecture #6: Audio/Haptic Displays Audio/Haptic Spring 2011 Joseph J. LaViola Jr. Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Interaction Interaction Workflow Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 1 Introduction To Displays Display: device which presents perceptual information Goal: display devices which accurately represent perceptions in simulated world Displays do not have to be just visual Spring 2011 auditory haptic, tactile olfactory CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Lecture Outline Audio Displays Displays Haptic/Tactile Displays 3D sound localization depth cues 3D sound generation sound configurations audio and 3DUIs haptic cues display characteristics display types haptics and 3DUIs Olfactory Displays Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 2 Audio Displays Present sound to the user Spatialized 3D sound 3D sound sound surrounds user take advantage of localization Localization – psychoacoustic process of determining location and direction from which and direction from which a sound sound emanates Many applications Spring 2011 http://www.loonygames.com/content/2.4/feat/ CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3D Sound Localization – Binaural Cues Comparison of sound waves received by each ear Interaural time difference – time between arrival of sound at each ear Interaural intensity difference – difference in sound intensity at each ear Cue does have problems Spring 2011 http://www.jeroenbreebaart.com/research_binaural.htm ambiguous situations when two or more sources where ITD and IID are the same CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3 3D Sound Localization – Spectral and Dynamic Cues Dynamic head movement or sound source head movement or sound source moving source is fairly weak cue Utilize spectral content Spring 2011 interaction of sound wave with outer ear occur at relatively high frequencies (above 6KHz) CAP6121 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3D Sound Localization – Head Related Transfer Functions Spatial filters that describe Spatial filters that describe how how sound waves interact with listener’s body listener specific lack reverberation info Build in echo free chamber with head model Spring 2011 http://www.tonmeister.ca/main/textbook/node320.html CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 4 3D Sound Localization – Reverberation Many factors affect a sound source factors affect sound source objects atmospheric properties Sound hits listener directly and indirectly Aids in perception of distance, not direction Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3D Sound Localization – Others Intensity (i.e., loudness) (i loudness) simple cue common in 3D audio displays Vision and Environment Spring 2011 sounds in FOV make spatial percepts CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 5 3D Sound Generation 3D sound sampling and synthesis sound sampling and synthesis take samples from real environment binaural audio recording generate HRTFs for each ear Auralization Spring 2011 rendering a sound field sound field good for reverberation effects wavewave-based modeling rayray-based modeling CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Sound System Configurations Headphones (stereophonic) (stereophonic) High level of channel separation avoid crosstalk isolate listener from external sounds problems with inside the head localization External Speakers Spring 2011 place speakers around the room no need to wear anything problems with crosstalk Two approaches –transaural audio and amplitude panning CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 6 Audio in 3DUIs Localization Sonfication adding realism Sensory Substitution turning information into sounds Ambient Effects wayfinding aids sound for touch Annotation and Help Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Haptic Displays Provide user with sense of Provide user with sense of touch touch Spring 2011 simulate interaction between virtual objects and user force (joint/muscle) (j tactile (skin-based) (skin- CAP6121 – 3D User Interfaces for Games and Virtual Reality www.novint.com ©Joseph J. LaViola Jr. 7 Haptic Cues – Tactile Cues Taction perceived by variety of Taction – perceived by variety of cutaneous cutaneous receptors under skin surface surface texture temperature pressure pain Found in varying concentrations on the body Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Haptic Cues – Kinesthetic Cues Perceived by receptors in muscles joints Perceived by receptors in muscles, joints and and tendons help to determine movement, position and torque of different body parts relationship between user and object via muscular tension Both active (movement self-induced) and selfpassive (movement by external force) Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 8 Haptic Display Characteristics Presentation capability what types of Presentation capability – what types of output output Resolution Spatial – minimum proximity of stimuli Temporal – refresh rate Ergonomics Spring 2011 Don’t want to break anyone CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Haptic and Tactile Displays “For every action there is an equal and opposite reaction” reaction” Sir Isaac Newton Main forms of feedback Spring 2011 ground referenced body referenced tactile tactile combination passive – physical props CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 9 Haptic Displays Motionware device device Provides vestibular stimulation Sends signals to the 8th cranial nerve Gi Gives user a sense of sense of motion motion No longer exits Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Other Haptic Devices Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 10 PainPong PainStation two person console person console electro shock, heat, whipping: bad performance pain PainStation - Pong Courtesy of www.painstation.de Spring 2011 CAP6121 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Hall Hall of Pain (www.painstation.de) Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 11 Haptics – Interface Design Useful for object manipulation manipulation Problem with these devices is they are very intimidating Mimic real world interaction Untapped area for games Spring 2011 scary too!! CAP6121 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Olfactory Output Least developed area developed area maybe for good reason! Has practical applications fire fighting surgical training surgical training Number of practical problems Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 12 Next Class Input devices devices Readings Spring 2011 3DUI Book – Chapter 3, pages 59-86 59- CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 13 ...
View Full Document

Ask a homework question - tutors are online