Multi-touch Interaction for Robot Control
, Jill L. Drury
, Brenden Keyes
, and Holly A. Yanco
University of Massachusetts Lowell
1 University Avenue, Lowell, MA 01854 USA
The MITRE Corporation
202 Burlington Road, Bedford, MA 01730 USA
Recent developments in multi-touch technologies have
exposed fertile ground for research in enriched human-
Although multi-touch technologies have
been used for virtual 3D applications, to the authors'
knowledge, ours is the first study to explore the use of a
multi-touch table with a physical robot agent.
study explores the control of a single agent with a multi-
touch table using an adapted, previously studied, joystick-
We performed a detailed analysis of users’
interaction styles with two complex functions of the multi-
touch interface and isolated mismatches between user
expectations and interaction functionality.
ACM Classification Keywords
H5.2. Information systems, information interfaces and
presentation: User interfaces.
There is much excitement surrounding multi-touch tables
and displays (e.g., Dietz, 2001; Han, 2005) because of the
potential for enhanced interactivity.
Removing the joystick,
mouse, or keyboard can remove a layer of interface
abstraction and increase the degree of direct manipulation
In the case of human-robot
interaction, this technology should allow users more direct
interaction with the robot to influence its behavior.
A joystick interface limits the user to a relatively small set
of interaction possibilities.
Digital buttons, analog gimbals,
and analog sliders are the three common input modes.
multi-touch surface is quite different, allowing for almost
limitless interaction methods on a 2D plane.
joystick limits the user through mechanical and physical
constraints, the multi-touch surface serves as the blank
canvas on which control surfaces are dynamically created.
But the flexibility and freedom of the interface also presents
problems for designers.
Designers must carefully choose
control methods that give appropriate feedback and
extremely clear affordances (affordances mean that
perceived and actual functionality match; see Norman,
Users are accustomed to haptic feedback, such as
spring-loaded buttons and gimbals, and auditory feedback,
such as clicks, even from a non-force-feedback joystick
controller: feedback not provided by multi-touch surfaces.
While multi-touch table technology seems to offer promise
for human-robot interaction, we could not find any previous
work that characterized human performance when directing
physical agents (robots) via multi-touch table interaction.
We wanted to understand how to best take advantage of the