Olsen - Metrics for Evaluating Human-Robot Interactions Dan...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
Metrics for Evaluating Human-Robot Interactions Dan R. Olsen, Michael A. Goodrich Computer Science Department Brigham Young University Provo, Utah 84602 {olsen, mike}@cs.byu.edu ABSTRACT Metrics for evaluating the quality of a human-robot interface are introduced. The autonomy of a robot is measured by its neglect time. The robot attention demand metric measures how much of the user’s attention is involved with instructing a robot. The free-time and fan-out metrics are two ways to measure this demand. Each of them leads to estimates of the interaction effort. Reducing interaction effort without diminishing task effectiveness is the goal of human-robot interaction design. Keywords: Neglect-time, Fan-out, Free-time, metrics 1. I NTRODUCTION Autonomous robots that can perform a variety of tasks with no human intervention are an interesting but ultimately marginal goal. What we really want are robots that can do what we want when we want it, not whatever they want whenever they want it. We are not interested in producing alternate life-forms. We are interested in effective servants. We want devices that will leverage human attention and human ability. In this paper we ignore the leveraging of human physical abilities and focus on the leverage of human attention. In this paper we present a series of metrics for measuring the effectiveness of robots as servants of their human masters. In particular we are looking for measures of interface effectiveness that capture our desires to leverage human attention The first metrics are those that measure task effectiveness (TE). Task effectiveness is some measure of how well a task is actually performed. At the end of the day we care mostly about getting some task done. In driving or navigation scenarios we might measure effectiveness as the time required to get from point A to point B. In search tasks we could measure the time to find all targets or the number of targets found in a given amount of time. In an assault task we might measure targets destroyed and losses taken. Ultimately task effectiveness measures are key to successfully designing and evaluating human-robot teams. However, task effectiveness measures do not shed any insight on how to improve the human-robot interface or how that interface might be modified to increase the effectiveness. We believe that metrics must be based in a framework that guides design. We are looking for an engineering approach that leads us through a space of design alternatives to a human- robot interface that enhances the task effectiveness of the team. In this paper we will discuss six interrelated metrics that can guide the design of human-robot interaction. They are task effectiveness (TE), neglect tolerance(NT), robot attention demand(RAD), free time(FT), fan out (FO) and interaction effort (IE). These metrics are somewhat generic and are instantiated differently for different robot tasks. However, together they provide a framework for thinking about interaction design.
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 08/03/2010 for the course MECHANIC 65921 taught by Professor Jons during the Spring '10 term at Tampa.

Page1 / 8

Olsen - Metrics for Evaluating Human-Robot Interactions Dan...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online