SongDD2011a

SongDD2011a - Tracking Body and Hands for Gesture...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Tracking Body and Hands for Gesture Recognition: NATOPS Aircraft Handling Signals Database Yale Song, David Demirdjian, and Randall Davis MIT Computer Science and Artificial Intelligence Laboratory 32 Vassar Street, Cambridge, MA 02139 { yalesong,demirdj,davis } @csail.mit.edu Abstract We present a unified framework for body and hand tracking, the output of which can be used for under- standing simultaneously performed body-and-hand gestures. The framework uses a stereo camera to collect 3D images, and tracks body and hand together, combining various existing techniques to make tracking tasks efficient. In addition, we introduce a multi-signal gesture database: the NATOPS aircraft handling signals. Unlike previous gesture databases, this data requires knowledge about both body and hand in order to distinguish gestures. It is also focused on a clearly defined gesture vocabulary from a real-world scenario that has been refined over many years. The database includes 24 body-and- hand gestures, and provides both gesture video clips and the body and hand features we extracted. I. INTRODUCTION Human gesture is most naturally expressed with body and hands, ranging from the simple gestures we use in normal conversations to the more elaborate gestures used by baseball coaches giving signals to players; soldiers gesturing for tactical tasks; and police giving body and hand signals to drivers. Current technology for gesture understanding is, however, still sharply limited, with body and hand signals typically considered separately, restricting the expressiveness of the gesture vocabulary and making interaction less natural. We have developed a multi-signal gesture recognition system that attends to both bodies and hands, allowing a richer gesture vocabulary and more natural human-computer interaction. In this paper, we present the signal processing part of the system, a unified framework for tracking bodies and hands to obtain signals. The signal understanding part (i.e., learning to recognize patterns of multi-signal gestures) is described in a companion paper [16]. There has been extensive work in human pose tracking, including upper or full body, hand, head, and eye gaze. In [3], for example, Buehler et al . presented an arm-and-hand tracking system that enabled the extracted signals to be used in sign language recognition. Hand poses were estimated us- ing histograms of oriented gradients (HOG) [5] features, but not classified explicitly. Also, body poses were reconstructed in 2D space, losing some of the important features in gesture recognition (e.g., pointing direction). In [15], Nickel et al . developed a head-and-hand tracking system for recognizing pointing gestures. The system tracked 3D positions of head and hands based on skin-color distribution. The extracted signals were used for recognizing pointing gestures using an HMM. However, their application scenario included static pointing gestures only, a task too simple to explore the complex nature of multi-signal gestures.complex nature of multi-signal gestures....
View Full Document

Page1 / 7

SongDD2011a - Tracking Body and Hands for Gesture...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online