Learning to Detect Natural Image Boundaries
Using Local Brightness, Color,
and Texture Cues
David R. Martin,
, Charless C. Fowlkes, and Jitendra Malik,
—The goal of this work is to accurately detect and localize boundaries in natural scenes using local image measurements.
We formulate features that respond to characteristic changes in brightness, color, and texture associated with natural boundaries. In
order to combine the information from these features in an optimal way, we train a classifier using human labeled images as ground
truth. The output of this classifier provides the posterior probability of a boundary at each image location and orientation. We present
precision-recall curves showing that the resulting detector significantly outperforms existing approaches. Our two main results are
1) that cue combination can be performed adequately with a simple linear model and 2) that a proper, explicit treatment of texture is
required to detect boundaries in natural images.
—Texture, supervised learning, cue combination, natural images, ground truth segmentation data set, boundary
detection, boundary localization.
the images and human-marked boundaries
shown in Fig. 1. How might we find these boundaries
We distinguish the problem of boundary detection from
what is classically referred to as edge detection. A
is a contour in the image plane that represents a change in
pixel ownership from one object or surface to another. In
is most often defined as an abrupt change
in some low-level image feature such as brightness or color.
Edge detection is thus one low-level technique that is
commonly applied toward the goal of boundary detection.
Another approach would be to recognize objects in the
scene and use that high-level information to infer the
In this paper, we focus on what information is available
in a local image patch like those shown in the first column
of Fig. 2. Though these patches lack global context, it is clear
to a human observer which contain boundaries and which
do not. Our goal is to use features extracted from such an
image patch to estimate the posterior probability of a
boundary passing through the center point. A boundary
model based on such local information is likely to be
integral to any perceptual organization algorithm that
operates on natural images, whether based on grouping
pixels into regions ,  or grouping edge fragments into
contours , . This paper is intentionally agnostic about
how a local boundary model might be used in a system for
performing a high-level visual task such as recognition.
The most common approach to local boundary detection
is to look for discontinuities in image brightness. For
example, the Canny detector  models boundaries as
brightness step edges. The brightness profiles in the second
column of Fig. 2 show that this is an inadequate model for
boundaries in natural images where texture is a ubiquitous
phenomenon. The Canny detector fires wildly inside