MODELING THE INTERACTION BETWEEN SPEECH AND GESTURE
Justine Cassell Mark Steedman Norm Badler Catherine Pelachaud Matthew Stone Brett Douville Scott Prevost Brett Achorn Computer & Information Science University of Pennsylvania Philadelphia, PA 19104
Introduction
Numerous approaches have been applied to the problem of visual interpretation of gestures for Human-Computer Interaction (HCI).
Gesture Recognition
Xiaojun Qi
Many of those approaches have been chosen and implemented so that they foc
To Gesture or Not to Gesture: What is the Question?
Center for Human Modeling and Simulation University of Pennsylvania Philadelphia, PA 19104-6389 215-898-5862 badler@central.cis.upenn.edu
Computer synthesized characters are expected to make approp
VL2 PRESENTATION SERIES AT A GLANCE
About the VL2 Presentation Series
In 2008-2009, Gallaudets NSF Science of Learning Center on Visual Language and Visual Learning will host the VL2 Presentation Series featuring distinguished researchers renowned fo
A Salience-Based Approach to Gesture-Speech Alignment
Jacob Eisenstein and C. Mario Christoudias MIT Computer Science and Articial Intelligence Laboratory 32 Vassar Street Cambridge, MA 02139 {jacobe+cmch}@csail.mit.edu
Abstract
One of the rst steps
TEP 290: Distributed Learning Visit with Distributed Cognition & Human-Computer Interaction Lab at UCSD (Ed Hutchins & Jim Hollan) 11.07.2005 Original Issue to Discuss: The multimodal nature of interaction in distributed learning interactions
Speake
GPs under abnormal conditions
An implication of the GP hypothesis is that, by speaking, gestures are generated as an integral component of the process. Without imagery, of which gestures are a materialization, there could not be speech. Thus, if we
Chapter 4: The Nonverbal Messages
Interpreting Nonverbal Messages - Four types of communication: - verbal/vocal communication ~ communication through the spoken word (eg. discussion) - verbal/nonvocal communication ~ words are involved but no speakin
Tribal Communication Each member of the tribe, despite being a culturally diverse group, maintains an understanding of one another. Although English is the dominant language within the tribe, not all members use it consistently. Without focusing too
Ch. 7 Communicating Nonverbally Introduction We all communicate without saying a word. 65% of overall message meaning is conveyed nonverbally. 93% of emotional meaning is conveyed nonverbally. Using nonverbal behaviors, we draw conclusions about
Lane 1
Alex Lane Aaron Shield LIN 312: Language in the Body 13 Sept. 2007 Discussion Notes #1 One of the thirteen defining characteristics of human language that Charles Hockett identified in 1960 was specialization meaning that organs used for pro
Culture & language
Modern Homo sapiens Evolved ~200 kya in Africa Traits of human language Communication in primates Language in an evolutionary light What is culture? Dispersed ~ 50 kya More gracile, 1400 cc brains "Modern" intelligence?
The Origins of Language
Primate Communication
Vocal + Nonvocal Expresses immediate mental/emotional state Species-wide Innate Examples:
Yikesa snake! Give me some of that I am dangerous! I see food!
Human Spoken Language
Distinguishing Traits
D
The Origins of Speech
Bio 222 April 9, 2008 Ron Hoy
Reading Assignment Purves, textbook Page 688-689, Box 27A, "Speech"
How does it Happen so Fast?
Oral behavior as the precursors of speech
Yawning
See yawn
Do yawn
Pre-human primate origins
Chapter 7
THOUGHT & LANGUAGE
Lecture Outline
Units of Thought Problem Solving Implicit Thought Language
Thought
Thought is an extension of perception and memory
We form mental representations We recall representations using memory We ment