& Accuracy Uncertainty
Matt Rice
GGS 311 - GMU
Fall 2011
National
Map
Published by
Accuracy
USGS
Standards
http://egsc.usgs.gov/isb/pubs/factsheets/fs17199.html#US%
20National
Horizontal
accuracy
For maps on
publication
National
Map
Published by
Accuracy
USGS
Standards
http://egsc.usgs.gov/isb/pubs/factsheets/fs17199.html#US%
20National
Vertical
accuracy
Vertical
accuracy, as
National
Map
Published by
Accuracy
USGS
Standards
http://egsc.usgs.gov/isb/pubs/factsheets/fs17199.html#US%
20National
Conforming
Maps:
This map
complies with
Uncertainty & Accuracy
Longley et al. Chp. 6
Definitions
Conception, measurement and analysis
Vagueness, ambiguity, and other problems
Statistical models of uncertainty
Error propagation
Living with uncertainty
Introduction
Imperfect or uncertain reconciliation and
GIS
Tension between perfection/ideal and reality
It is impossible to make a perfect
representation of the world, so
uncertainty about it is inevitable
Sources of Uncertainty
Measurement error: different observers,
measuring instruments
Specification error: omitted variables
Ambiguity, vagueness and the quality of a
GIS representation
Uncertainty is a catch-all for incomplete
representations or quality measures
A conceptual view of uncertainty. The three filters, U1, U2, and U3, distort the
way in which the complexity of the real world is conceived, represented, and
analyzed in a cumulative way.
U1:
Conceptio
n
Spatial uncertainty &
formulating an approach to a
geographic problem:
Natural geographic units?
In many cases, there are
no natural units of
geographic analysis
Analysis involving more
than one variable makes
the search for a natural
unit more difficult
U1:
Conception:
GIS units/zones &
vagueness
Vagueness
Statistics & vagueness
Cartography & vagueness
visual communication of
vague boundaries or
labels
Cognition & vagueness
confidence intervals with
boundaries or labels
perception of
vagueness, need to
categorize data
A local assemblage of oak trees or part of an oak woodlandon Ragged
Boys Hill, in the New Forest, UK. The Ordnance Survey map of the area
suggests that there are trees beyond the perimeter of the woodland
area, while the tree symbology suggests that the woods are
characterized by varying proportions of deciduous and coniferous trees.
(Photo courtesy Jim Champion)
U1:
Conception:
GIS & ambiguity
Ambiguity
Many linguistic terms used to
convey geographic information
are inherently ambiguous
Geographic prepositions
(across, over, in) & translation
to other languages
Labels for terrain features:
mountain, hill, bluff, butte,
plateau, abundance, alp, cliff,
crag, dome, drift, elevation,
eminence, glob, heap, height,
hump, mass, mesa, mound,
mount, palisade, peak, pike,
pile, precipice, pyramid, range,
ridge, shock, sierra, stack, ton,
U1:
Conception:
GIS & ambiguity
Ambiguity
Direct vs. Indirect indicators
Direct: clear correspondence
with a mapped phenomenon
Indirect: perceived surrogate
link with the phenomenon of
interest
Ambiguity is introduced when
imperfect indicators of
phenomena are used instead of
the phenomena themselves.
Ambiguity: Classification
schemes, definitions, & crossborder data conflation
Fuzzy
Approache
s to
In fuzzy set theory, it is
possible to have partial
Uembership in a set:
m ncertaint
y
membership can vary,
e.g. from 0 to 1
this adds a third option
to classification: yes,
no, and maybe
Fuzzy approaches have
been applied to the
mapping of soils,
vegetation cover, and
A.X. Zhu, Soil Mapping Using GIS, Expert
Knowledge, and Fuzzy Logic
Scale &
Geographi
Identification of
c Units
homogeneous zones
and spheres of
influence lies at the
heart of traditional
regional geography as
well as contemporary
data analysis
The regional geography of Russia
Relationships typically
grow stronger when
based on larger
geographic units
(Source de Blij H.J. and Muller P.O. 2000 Geography: Realms, Regions
and Concepts (9th ed) New York: Wiley, p. 113)
U2:
Measurement/
representation
Representati
onal models
(raster/vector
) filter reality
differently
Uncertainty
can be
measured
Statistical measures of
uncertainty: nominal case
How to
measure the
accuracy of
nominal
attributes?
e.g., a
vegetation
cover map
Field checked
B
C
D
E
Total
A
80
4
0
15
7
106
B
2
17
0
9
2
30
C
12
5
9
4
8
38
D
7
8
0
65
0
80
E
3
2
1
6
38
50
Total
Recorded in database
A
104
36
10
99
55
304
Example a of confusion matrix. A grand total of 304 parcels have been checked. The
rows of the table correspond to the land use class of each parcel as recorded in the
database, and the columns correspond to the class as recorded in the field. The
numbers appearing on the principal diagonal of the table (from top left to bottom right)
reflect correct classification.
Confusion Matrix Statistics
Percent correctly classified
total of diagonal entries divided by the grand total,
times 100
209/304*100 = 68.8%
but chance (accidental correct classification) would
give a score of better than 0
Kappa statistic
normalized to range from 0 (chance) to 100
evaluates to 58.3%
Sampling for the Confusion Matrix
Examining every parcel may not be
practical
Rarer classes should be sampled more
often in order to assess accuracy
reliably
sampling is often stratified by class
An example of a vegetation cover map
Methods for estimating the accuracy of classification
Interval/Ratio Case
Errors distort measurements by small amounts
Accuracy refers to the amount of distortion from
the true value
Precision
refers to the variation among repeated measurements
and also to the amount of detail in the reporting of a
measurement
Higher precision,
lower accuracy
Higher accuracy,
lower precision
The term precision is often used to refer to the repeatability of
measurements. In both diagrams six measurements have been
taken of the same position, represented by the center of the circle.
On the left, successive measurements have similar values (they are
precise), but show a bias away from the correct value (they are
inaccurate). On the right, precision is lower but accuracy is higher.
Reporting Measurements
The amount of detail in a reported
measurement (e.g., output from a GIS)
should reflect its accuracy
14.4m implies an accuracy of 0.1m
14m implies an accuracy of 1m
Excess precision should be removed by
rounding
Measuring Accuracy
Root Mean Square Error is the square root
of the average squared error
the primary measure of accuracy in map
accuracy standards and GIS databases
e.g., elevations in a digital elevation model
might have an RMSE of 2m
the abundances of errors of different
magnitudes often closely follow a Gaussian or
normal distribution
The Gaussian or Normal distribution. The light blue area (between 61
standard deviation) encloses 68% of the area under the curve, so 68% of
observations will fall between these limits.
A Useful Rule of Thumb for Positional
Accuracy
Positional accuracy of features on a paper
map is roughly 0.5mm on the map
e.g., 0.5mm on a map at scale 1:24,000 gives a
positional accuracy of 12m
this is approximately the U.S. National Map
Accuracy Standard
and also allows for digitizing error, stretching of the
paper, and other common sources of positional
error
A useful rule of thumb is that positions measured from maps are
accurate to about 0.5 mm on the map. Multiplying this by the
scale of the map gives the corresponding distance on the
ground.
Map scale
1:1250
Ground distance corresponding to
0.5 mm map distance
62.5 cm
1:2500
1.25 m
1:5000
2.5 m
1:10,000
5m
1:24,000
12 m
1:50,000
25 m
1:100,000
50 m
1:250,000
125 m
1:1,000,000
500 m
1:10,000,000
5 km
U3:
Analysis.
Uncertainties in data
lead to uncertainties in
Error
the results of analysis
PAlmost every input to a
ropagatio
GIS is subject to error
n
and uncertainty
In principle, every
output should have
confidence limits or
some other
expression of
uncertainty
Three realizations of a model simulating the effects of error on a
digital elevation model
Overlay of two street networks in Goleta, California
Living with Uncertainty
It is easy to see the importance of
uncertainty in GIS
but much more difficult to deal with it
effectively
but we may have no option, especially in
disputes that are likely to involve litigation
Some Basic Principles
Uncertainty is inevitable in GIS
Data obtained from others should never be
taken as truth
efforts should be made to determine quality
Uncertainty information may be recorded in metadata
Effects on GIS outputs are often much greater
than expected
there is an automatic tendency to regard outputs from
a computer as the truth
More Basic Principles
Use as many sources of data as possible
Be honest and informative in reporting
results
and cross-check them for accuracy
add plenty of caveats and cautions
Keep a lab notebook, document your work