Unformatted text preview: Summary of Experimental
Uncertainty Assessment
Methodology
F. Stern, M. Muste, ML. Beninati, W.E. Eichinger 12/09/11 1 Table of Contents Introduction Test Design Philosophy Definitions Measurement Systems, DataReduction Equations,
Measurement
and Error Sources
and Uncertainty Propagation Equation Uncertainty Equations for Single and Multiple
Uncertainty
Tests
Tests Implementation & Recommendations Introduction Experiments are an essential and integral tool for
Experiments
engineering and science
engineering Experimentation: procedure for testing or determination of a
truth, principle, or effect
truth, True values are seldom known and experiments have errors
True
due to instruments, data acquisition, data reduction, and
environmental effects
environmental Therefore, determination of truth requires estimates for
Therefore,
experimental errors, i.e., uncertainties
experimental Uncertainty estimates are imperative for risk assessments in
Uncertainty
design both when using data directly or in calibrating and/or
validating simulation methods
validating Introduction Uncertainty analysis (UA): rigorous methodology for
Uncertainty
uncertainty assessment using statistical and engineering
concepts
concepts ASME (1998) and AIAA (1999) standards are the most
recent updates of UA methodologies, which are
internationally recognized Presentation purpose: to provide summary of EFD UA
Presentation
methodology accessible and suitable for student and
faculty use both in classroom and research laboratories Test design philosophy Purposes for experiments:
Purposes Type of tests: Science & technology
Research & development
Design, test, and product liability and acceptance
Instruction Small scale laboratory
Largescale TT, WT
Insitu experiments Examples of fluids engineering tests: Theoretical model formulation
Benchmark data for standardized testing and evaluation of facility biases
Simulation validation
Instrumentation calibration
Design optimization and analysis
Product liability and acceptance Test design philosophy Decisions on conducting experiments: governed
Decisions
by the ability of the expected test outcome to
achieve the test objectives within allowable
uncertainties Integration of UA into all test phases should be a
Integration
key part of entire experimental program
key Test description
Determination of error sources
Estimation of uncertainty
Documentation of the results Test design philosophy
D E F IN E P U R P O S E O F T E S T A N D
R E S U L T S U N C E R T A IN T Y R E Q U IR E M E N T S S E L E C T U N C E R T A IN T Y M E T H O D  D E S IG N T H E T E S T
D E S IR E D P A R A M E T E R S ( C D , C R , ....)
M O D E L C O N F IG U R A T IO N S (S )
T E S T T E C H N IQ U E ( S )
M E A S U R E M E N T S R E Q U IR E D
S P E C IF IC IN S T R U M E N T A T IO N
C O R R E C T IO N S T O B E A P P L IE D D E T E R M IN E E R R O R S O U R C E S
A F F E C T IN G R E S U L T S
YES
E S T IM A T E E F F E C T O F
T H E E R R O R S O N R E S U LTS IM P R O V E M E N T
P O S S IB L E ? NO U N C E R T A IN T Y
A C C E P TA BLE ? NO YES NO
NO TEST IM P L E M E N T T E S T
S TAR T TE S T R E S U LT S
A C C E P TA B LE ? NO YES YES
NO C O N T IN U E T E S T PURPOSE
A C H IE V E D ? M EASUREMENT
SYSTEM
PRO BLEM ? S O LV E P R O B LE M YES E S T IM A T E
A C TU A L D ATA
U N C E R T A IN T Y  DOCUMEN
REFERENC
P R E C IS IO N
B IA S L IM IT
TO TA L U N C T R E S U LT S
E C O N D IT IO N
L IM IT
E R T A IN T Y Definitions Accuracy: closeness of agreement between
measured and true value
measured Error: difference between measured and true value
difference Uncertainties (U): estimate of errors in
measurements of individual variables Xi (Uxi) or
or
results (Ur) obtained by combining Uxi
results
obtained Estimates of U made at 95% confidence level
Estimates
95% Definitions
ixed,
Bias Bias error β: ffixed,
systematic
systematic δ k+ Bias Bias limit B: estimate 1 εk δk ε k+ 1 X
X of β X t ru e X k k+ 1 ( a ) tw o r e a d in g s random
random Precision limit P:
estimate of ε β+ ε β
ε FREQUENCY OF OCCURRENCE Precision Precision error ε: Total Total error: δ = δ = to ta l e r ro r
β = b ia s e r r o r
ε = p r e c is io n e r r o r β ε X
X t ru e µ
M A G N IT U D E O F X (b ) in fin ite n u m b e r o f re a d in g s Measurement systems, data
reduction equations, & error sources Measurement systems for individual variables Xi:
Measurement
instrumentation, data acquisition and reduction procedures,
and operational environment (laboratory, largescale
facility, in situ) often including scale models
facility, Results expressed through datareduction equations
r = r(X1, X2, X3,…, Xj)
,…, Estimates of errors are meaningful only when considered
Estimates
in the context of the process leading to the value of the
quantity under consideration
quantity Identification and quantification of error sources require
Identification
considerations of: Steps used in the process to obtain the measurement of the quantity
The environment in which the steps were accomplished Measurement systems and data
reduction equations Block diagram showing elemental error sources, individual
Block
measurement systems, measurement of individual variables,
data reduction equations, and experimental results
data
E LE M EN TA L
ERROR SOURCES 1 2 J IN D IV ID U A L
M EASUREM ENT
SYSTEMS X
1
B ,P X
2
B ,P X
J
B ,P M EASUREM ENT
O F IN D IV ID U A L
V A R IA B L E S 1 1 2 2 J r = r ( X , X ,......, X )
1 2 r
B ,P
r J r J D A T A R E D U C T IO N
E Q U A T IO N E X P E R IM E N T A L
R E S U LT Error sources Estimation assumptions: 95% confidence level,
Estimation
largesample, statistical parent distribution
largesample,
T E S T E N V IR O N M E N T: M O D E L F ID E L IT Y A N D
TEST SETUP:
 A s b u ilt g e o m e tr y
 H y d ro d y n a m ic d e fo rm a tio n
 S u r f a c e f in is h
 M o d e l p o s itio n in g  C a l ib r a t i o n v e r s u s t e s t
 S p a t i a l/ t e m p o r a l v a r i a t i o n s
o f t h e f lo w
 S e n s o r i n s t a l l a t io n / l o c a t i o n
 W a ll i n t e r f e r e n c e
 F lu i d a n d f a c i l i t y c o n d i t i o n s C O N T R IB U T IO N S
T O E S T IM A T E D
U N C E R T A IN IT Y S IM U L A T IO N T E C H N IQ U E S :
 I n s tr u m e n ta tio n in te r fe r e n c e
 S c a le e ff e c ts D A T A A C Q U IS IT IO N A N D
R E D U C T IO N :
 S a m p l in g , f ilt e r in g , a n d s t a t is t i c s
 C u r v e fits
 C a li b r a t i o n s Uncertainty propagation equation Bias and precision errors in the measurement of Xi
Bias
propagate through the data reduction equation r = r(X1, X2,
X3,…, Xj) resulting in bias and precision errors in the
,…,
resulting
experimental result r A small error (δXi) in the measured variable leads to a
in
small error in the result (δr) that can be approximated using
small
that
Taylor series expansion of r(Xi) about rtrue(Xi) as
about
δ r = r ( X i ) − rtrue ( X i ) = δ X i dr
dX i The derivative is referred to as sensitivity coefficient. The
The
larger the derivative/slope, the more sensitive the value of
the result is to a small error in a measured variable
the Uncertainty propagation equation Overview given for derivation of equation describing the error
Overview
propagation with attention to assumptions and approximations used to
obtain final uncertainty equation applicable for single and multiple
tests Two variables, kth set of measurements (xk, yk)
Two
r = r ( x, y ) βx xk = xtrue + β xk + ε xk
x tr u e yk = ytrue + β yk + ε yk
rk − rtrue = βy k εx εy k µx yk r k = r ( x k , yk ) ∂r
( xk − xtrue ) + ∂r ( yk − ytrue ) + R2
∂x
∂y βr The total error in the kth determination of r
The
th
δ rk = rk − rtrue = θ x ( β xk + ε xk ) + θ y ( β yk + ε yk )
θ x = ∂r ∂x; θ y = ∂r ∂y µy xk r tru e (1 ) sensitivity coefficients k k k εrk
rk y tr u e Uncertainty propagation equation We would like to know the distribution of δr (called the parent
We
distribution) for a large number of determinations of the result r A measure of the parent distribution is its variance defined as
measure
σ 2
δr () 1 N
2
= lim ∑ δ rk N →∞ N k =1 (2 ) Substituting (1) into (2), taking the limit for N approaching infinity,
using definitions of variances similar to equation (2) for β ’s and ε ’s and
’s
their correlation, and assuming no correlated bias/precision errors
their
2
2
σ δ2r = θ x2σ β x + θ y2σ β y + 2θ xθ yσ β x β y + θ x2σ ε2x + θ y2σ ε2x + 2θ xθ yσ ε xε y (3 ) n σ’s iin equation (3) are not known; estimates for them must be made Uncertainty propagation equation
Defining
u c2 estimate for σ δ2r 2
2
bx2 , b y , bxy 2
S xw , S y , S xy estimates for the variances and covariances (correlated bias errors) of
the bias error distributions
estimates for the variances and covariances ( correlated precision
errors) of the precision error distributions equation (3) can be written as
22
22
uc2 = θ x2bx2 + θ y by + 2θ xθ y bxy + θ x2 S x2 + θ y S y + 2θ xθ y S xy Valid for any type of error
Valid
distribution
distribution To obtain uncertainty Ur at a specified confidence level (C%), a
%), coverage factor (K) must be used for uc:
coverage
must For normal distribution, U r = Kuc K is the t value from the Student t distribution.
is
value
distribution. For N ≥ 10, t = 2 for 95% confidence level
For Uncertainty propagation equation Generalization for J variables in a result r = r(X1, X2, X3,…, Xj)
Generalization
,…,
J J −1 U = ∑ θ B + 2∑
2
r i =1 2
i 2
i θi = J J ∑θ θ i =1 k =i +1 ik ∂r
∂X i J −1 Bik + ∑θ P + 2∑
i =1 22
ii J ∑θ θ i =1 k = i +1 P i k ik sensitivity coefficients Example:
CD =
U 2
CD D
= C D ( D, ρ , U , A )
2
1 2 ρU A
J J = ∑θ B + ∑θ i2 Pi 2
2
i i =1 2 2
i i =1 2 2 2 ∂C 2 ∂C 2 ∂C 2 ∂C 2
2
= D BD + PD + D Bρ + Pρ2 + D BU + PU2 + D BA + PA2 ∂ρ ∂D ∂U ∂A ( ) ( ) ( ) ( ) Uncertainty equations for single and
multiple tests
Measurements can be made in several ways: Single test (for complex or expensive experiments): one set
Single
of measurements (X1, X2, …, Xj) for r
of
…,
for According to the present methodology, a test is considered a single test if the entire
According
test is performed only once, even if the measurements of one or more variables are
made from many samples (e.g., LDV velocity measurements)
made Multiple tests (ideal situations): many sets of
Multiple
measurements (X1, X2, …, Xj) for r at a fixed test condition
measurements
…,
for
with the same measurement system
with Uncertainty equations for single and
multiple tests The total uncertainty of the result
The
U r2 = B 2 + P 2
r
r (4 ) Br : same estimation procedure for single and
multiple tests
multiple Pr : determined differently for single and multiple
tests
tests Uncertainty equations for single and
multiple tests: bias limits J J −1 B = ∑ θ B + 2∑
2
r Br : Sensitivity coefficients i =1 2
i 2
i J ∑θ θ i =1 k =i +1 ik Bik ∂r
∂X i θi = : estimate
data reduction, conceptual
estimate Berrors for Xof. calibration, data acquisition,may be several elemental
bias
Within each category, there
Within
i i. sources of bias. If for variable Xi there are J significant elemental bias
errors [estimated as (Bi)1, (Bi)2, … (Bi)J], the bias limit for Xi is calculated as
(B (B
(B ],
2
J B = ∑ ( Bi )
2
i k =1 k L Bik = ∑ ( Bi ) α ( Bk ) α stimate B : eestimate of correlated bias limits for X and X
ik α =1 i k Uncertainty equations for single test:
precision limits Precision limit of the result (end to end):
Pr = tS r t: coverage factor (t = 2 for N > 10)
Sr: the standard deviation for the N readings of the result. Sr must be
the
determined from N readings over an appropriate/sufficient time interval Precision limit of the result (individual variables):
Precision
J Pr = ∑ ( θ i Pi )2
i=1 Pi = ti Si the precision limits for Xi
the Often is the case that the time interval is inappropriate/insufficient and Pi’s
Often
’s
or Pr’s must be estimated based on previous readings or best available
’s
information
information Uncertainty equations for multiple tests:
precision limits
1
r=
M The average result: M ∑r k k =1 Precision limit of the result (end to end):
Precision
tS r
Pr =
M M ( rk − r ) 2 S r = ∑ M −1 k =1 t: coverage factor (t = 2 for N > 10)
S r : standard deviation for M readings of the result The total uncertainty for the average result:
The ( U = B + P = B + 2 Sr
2
r 2
r 2
r 2
r M ) 2 Alternatively Pr can be determined by RSS of the
precision limits of the individual variables
precision 1/ 2 Implementation Define purpose of the test Determine data reduction equation: r = r(X1, X2, …, Xj)
Determine
…, Construct the block diagram Construct datastream diagrams from sensor to result Identify, prioritize, and estimate bias limits at individual variable level Uncertainty sources smaller than 1/4 or 1/5 of the largest sources are neglected Estimate precision limits (endtoend procedure recommended) Computed precision limits are only applicable for the random error sources that were
Computed
“active” during the repeated measurements
“active”
Ideally M ≥ 10, however, often this is no the case and for M < 10, a coverage factor t
Ideally
= 2 is still permissible if the bias and precision limits have similar magnitude.
is
If unacceptably large P’s are involved, the elemental error sources contributions must
If
’s
be examined to see which need to be (or can be) improved
be Calculate total uncertainty using equation (4) For each r, report total uncertainty and bias and precision limits
For
r, Recommendations Recognize that uncertainty depends on entire testing process and that
Recognize
any changes in the process can significantly affect the uncertainty of
the test results
the
Integrate uncertainty assessment methodology into all phases of the
Integrate
testing process (design, planning, calibration, execution and posttest
analyses)
analyses)
Simplify analyses by using prior knowledge (e.g., data base),
Simplify
concentrate on dominant error sources and use endtoend calibrations
and/or bias and precision limit estimation
and/or
Document:
Document: test design, measurement systems, and data streams in block diagrams
equipment and procedures used
error sources considered
all estimates for bias and precision limits and the methods used in their estimation
all
(e.g., manufacturers specifications, comparisons against standards, experience, etc.)
(e.g.,
detailed uncertainty assessment methodology and actual data uncertainty estimates References AIAA, 1999, “Assessment of Wind Tunnel Data Uncertainty,” AIAA
AIAA,
S071A1999.
S071A1999.
ASME, 1998, “Test Uncertainty,” ASME PTC 19.11998.
ANSI/ASME, 1985, “Measurement Uncertainty: Part 1, Instrument
ANSI/ASME,
and Apparatus,” ANSI/ASME PTC 19.I1985.
and
Coleman, H.W. and Steele, W.G., 1999, Experimentation and
Coleman,
Uncertainty Analysis for Engineers, 2nd Edition, John Wiley & Sons,
Uncertainty
Inc., New York, NY.
Inc.,
Coleman, H.W. and Steele, W.G., 1995, “Engineering Application of
Coleman,
Experimental Uncertainty Analysis,” AIAA Journal, Vol. 33, No.10,
pp. 1888 – 1896.
pp.
ISO, 1993, “Guide to the Expression of Uncertainty in
Guide
Measurement,", 1st edition, ISBN 9267101889.
Measurement
ITTC, 1999, Proceedings 22nd International Towing Tank Conference,
ITTC,
Proceedings
“Resistance Committee Report,” Seoul Korea and Shanghai China.
“Resistance ...
View
Full
Document
This note was uploaded on 12/08/2011 for the course MECHANICS 57:020 taught by Professor Fredrickstern during the Fall '10 term at University of Iowa.
 Fall '10
 FredrickStern

Click to edit the document details