1568981719.pdf - 14th European Signal Processing Conference(EUSIPCO 2006 Florence Italy September 4-8 2006 copyright by EURASIP ICANDO INTELLECTUAL

1568981719.pdf - 14th European Signal Processing...

This preview shows page 1 - 2 out of 5 pages.

ICANDO: INTELLECTUAL COMPUTER ASSISTANT FOR DISABLED OPERATORS Alexey Karpov and Andrey Ronzhin Speech Informatics Group, St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences SPIIRAS, 39, 14 th line, 199178, St. Petersburg, Russia phone: +7 (812) 328-7081, fax: +7 (812) 328-4450, email: {karpov,ronzhin}@iias.spb.su web: ABSTRACT The paper describes a prospective multimodal system ICanDo (Intellectual Computer AssistaNt for Disabled Op- erators) developed in SPIIRAS and intended for assistance to persons without hands or with disabilities of their hands or arms in human-computer interaction. This system com- bines the modules for automatic speech recognition and head tracking in one multimodal system. The architecture of the system, methods for recognition and tracking, multimo- dal information fusion and synchronization, experimental conditions and obtained results are described in the paper. The developed system was applied for hands-free work with Graphical User Interface in such tasks as Internet commu- nication and work with documents. 1. INTRODUCTION Many people are unable to operate a personal computer by a standard computer mouse or a keyboard because of disabili- ties of their hands or arms. One possible alternative for these persons is a multimodal system, which allows controlling a computer without traditional control devices, but using: (1) head (or face) movements to control the mouse cursor on a monitor screen; (2) speech input for giving the control commands. Speech and head-based multimodal control sys- tems have a great potential in improving the life comfort of disabled people as well as independence of their living from other persons. Disability may affect also the person's neck and head move- ments along with hands and arms. Thus a human can have problems with activity of neck and hence reduced ability to move the head in one or more directions. In many of such cases an eye tracking system can be successfully used in- stead of head tracking system. Moreover eye blinking can give the signal for click of mouse button. Among examples of hardware-software eye tacking systems Visual Mouse [1] and Eyegaze System [2] can be mentioned. However, the usage of eye tracking systems is worse than head tracking systems in such parameters as: task performance, human’s workload and comfort both for untrained and experienced users [3]. Of course, speech input is only one acceptable alternative instead of keyboard for motion-impaired users which cannot move their hands. In the following sections of the paper the assistive multimo- dal system ICanDo, which uses the head movements tracking for mouse cursor control on a monitor screen and the auto- matic speech recognition to press the buttons of a keyboard or a mouse, is presented. Section 2 describes the applied automatic speech recognition system, Section 3 presents the head tracking system, Section 4 gives the description of the method for audio and video data synchronization as well as
Image of page 1
Image of page 2

You've reached the end of your free preview.

Want to read all 5 pages?

  • Summer '18
  • Muchiri
  • Speech recognition, Multimodal interaction, EURASIP, ICanDo, Speech Informatics Group

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Stuck? We have tutors online 24/7 who can help you get unstuck.
A+ icon
Ask Expert Tutors You can ask You can ask You can ask (will expire )
Answers in as fast as 15 minutes