This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: is. ighaptem 0 Evaluation of Analytical Data 53 /________________._————————— ;4.1 INTRODUCTION The field of food analysis, or any type of analysis, :involves a Considerable amount of time learning prin
iciples, methods, and instrument operations and per
, fecﬁng Various techniques. Although these areas are extremely important, much of our effort would be for
tinaught if there were not some way for us to evaluate the
ram obtained from the various analytical assays. Sev
eral mathematical treatments are available that provide
an‘idea of how well a particular assay was performed
gibrhow well we can reproduce an experiment. Fortu
nately, the statistics are not too involved and apply to
' most analytical determinations. , The focus in this chapter is primarily on how to
evaluate replicate analyses of the same sample for accu
‘acy and precision. In addition, considerable attention
3. to the determination of best line fits for stan dard curve data. Keep in mind as you read and work
: through this chapter that there is a vast array of com
. _ telsoftware to perform most types of data evaluation
and calculations / plots. VA ; Proper sampling and sample size are not covered
_ in this chapter. Readers should refer to Chapter 5 (espe
~ section 5.3.4.5) for sampling in general and statis
cal approaches to determine the appropriate sample
. ,r'an‘d to Chapter 19, section 19.2.2 for mycotoxin
} isanipling. MEASURES OF CENTRAL TENDENCY ‘ 3T9 mercase accuracy and precision, as well as to eval
* hate these parameters, the analysis of a sample is
W 7 uallyiperformed (repeated) several times. At least assays are typically performed, though often the
umber can be much higher. Because we are not sure
hich value is closest to the true value, we determine
"theemean (or average) using all the values obtained
‘and report the results of the mean. The mean is des _ by the symbol 3': and calculated according to
{,theequation below. x1+x2+x3++Xn 2x1 it = = [1]
. . n 11
Where:
1‘ 3 5c 2 mean _ xi. 3C2, etc. 2 individually measured values (xi)
5 ' __ ,7 n = number of measurements ‘ _ For example, suppose we measured a sample
. 0f uncooked hamburger for percent moisture content four times and obtained the following results: 64.53%,
__64.45%, 65.10%, and 64.78%.  64.53 + 64.45 65.10 64.78
x = + = ’ Thus, the result would be reported as 64.72% mois
ture. When we report the mean value, we are indicating
that this is the best experimental estimate of the value.
We are not saying anything about how accurate or true
this value is. Some of the individual values may be
closer to the true value, but there is no way to make
that determination, so we report only the mean. Another determination that can be used is the
median, which is the midpoint or middle number
within a group of numbers. Basically, half of the experi
mental values will be less than the median and half will
be greater. The median is not used often, because the
mean is such a superior experimental estimator. 4.3 RELIABILITY OF ANALYSIS ___—.———————_ Returning to our previous example, recall that we
obtained a mean value for moisture. However, we did
not have any indication of how repeatable the tests were
or how close our results were to the true value. The next
several sections will deal with these questions and some
of the relatively simple ways to calculate the answers. 4.3.1 Accuracy and Precision One of the most confusing aspects of data analysis for
students is grasping the concepts of accuracy and preci
sion. These terms are commonly used interchangeably
in society, which only adds to this confusion. If we
consider the purpose of the analysis, then these terms
become much clearer. If we look at our experiments,
we know that the first data obtained are the individ
ual results and a mean value (it). The next questions
should be: “How close were our individual measure—
ments?” and “How close were they to the true value?”
Both questions involve accuracy and precision. Now,
let us turn our attention to these terms. Accuracy refers to how close a particular measure
is to the true or correct value. In the moisture analy—
sis for hamburger, recall that we obtained a mean of
64.72%. Let us say the true moisture value was actually
65.05%. By comparing these two numbers, you could
probably make a guess that your results were fairly
accurate because they were close to the correct value.
(The calculations of accuracy will be discussed later.) The problem in determining accuracy is that most
of the time we are not sure what the true value is.
For certain types of materials we can purchase known
samples from, for example, the National Institute
of Standards and Technology, and check our assays . against these samples. Only then can we have an indica tion of the accuracy of the testing procedures. Another
approach is to compare our results with those of other
labs to determine how well they agree, assuming the
other labs are accurate. 54 Partl 0 General Information a 41
m A term that is much easier to deal with and deter—
mine is precision. This parameter is a measure of
how reproducible or how close replicate measurements
become. If repetitive testing yields similar results, then
we would say the precision of that test was good. From a
true statistical view, the precision often is called error,
when we are actually looking at experimental varia
tion. 50, the concepts of precision, error, and variation
are closely related. The difference between precision and accuracy can
be illustrated best with Fig. 4—1. Imagine shooting a
riﬂe at a target that represents experimental values. The
bull’ s eye would be the true value, and where the bullets
hit would represent the individual experimental val
ues. As you can see in Fig. 4—1a, the values can be tightly
spaced (good precision) and close to the bull’s eye (good
accuracy), or, in some cases, there can be situations with
good precision but poor accuracy (Fig. 4—1b). The worst
situation, as illustrated in Fig. 41d, is when both the
accuracy and precision are poor. In this case, because
of errors or variation in the determination, interpre—
tation of the results becomes very difficult. Later, the
practical aspects of the various types of error will be
discussed. When evaluating data, several tests are used com—
monly to give some appreciation of how much the
experimental values would vary if we were to repeat
the test (indicators of precision). An easy way to look
at the variation or scattering is to report the range of
the experimental values. The range is simply the dif
ference between the largest and smallest observation.
This measurement is not too useful and thus is seldom
used in evaluating data. Probably the best and most commonly used sta
tistical evaluation of the precision of analytical data is
the standard deviation. The standard deviation mea
sures the spread of the experimental values and gives
a good indication of how close the values are to each
other. When evaluating the standard deviation, one has
to remember that we are never able to analyze the entire
food product. That would be difficult, if not impossible, and very time consuming. Thus, the calculations we use
are only estimates of the unknown true value. If we have many samples, then the standard devi
ation is designated by the Greek letter sigma (a). It is
calculated according to Equation [3], assuming all the
food product was evaluated (which would be an infinite
amount of assays). ._ 2
o = [3]
n
where: o = standard deviation xi = individual sample values it = true mean n = total population of samples Because we do not know the value for the true
mean, the equation becomes somewhat simplified so
that we can use it with real data. In this case, we now
call the a term the standard deviation of the sample and
designate it by SD or a. It is determined according to
the calculation in Equation [4], where 56 replaces the true
mean term it, and n represents the number of samples. ._ “ 2
SD = ,/ ———E(x‘n x) [4] If the number of replicate determinations is small
(about 30 or less), which is common with most assays,
the n is replaced by the n — 1 term, and Equation [5]
is used. Unless you know otherwise, Equation [5] is
always used in calculating the standard deviation of a group of assays.
. _ ' 2 .
SD = M [5]
V n — 1 Depending on which of the equations above is
used, the standard deviation may be reported as SDn
or on and SDn_1 or an_1. (Different brands of software Mm ‘ "wsu's‘vzmdstmmﬂ ":smamm‘ml ﬁhm,rm:‘“xii'1§;3’ Mir: b c d Comparison of accuracy and precision: (a) good accuracy and good precision, (b) good precision and poor accuracy,
(c) good accuracy and poor precision, and (d) poor accuracy and poor precision. _ ._.. AM“. A.“ ‘WA. m“th Chapter 4 0 Evaluation of Analytical Data 55
Determination of the Standard Deviation
4" l V of Percent Moisture in Uncooked
Hamburger
Deviation >
from 8
Observed the Mean 3
Measurement % Moisture (x, — R) (x, — {02 g
u.
1 64.53 —0.1 9 0.0361
2 64.45 ~O.27 0.0729
3 65.10 +0.38 0.1444
4 64.78 +0.06 0.0036
2x. = 258.86 2(xi 402 = 0.257 .aa 20 1a True Value +10 +20 +3.,
(mean)
)(_£X_i_:258.86_6472 ‘ ’ ‘
— n 4 4_2 A normal distribution curve for a population or , and scientiﬁc calculators sometimes use different labels , for the keys, so one must be careful.) Table 41 shows nan example of the determination of standard devia . ,. tion. The sample results would be reported to‘ average
‘ 64.72% moisture with a standard deviation of 0.293. ' ” Once we have a mean and standard deviation, we
finext must determine how to interpret these numbers.
I  'v One easy way to get a feel for the standard deviation is g to calculate what is called the coefﬁcient of variation I. _'(CV), also known as the relative standard deviation. ' i. calculation is shown below for our example of the
moisture determination of uncooked hamburger. Coefficient of Variation (CV) = 5—5?— x 100% [6]
a I, ,, 7 _ 0.293
{is CV _ 6472 x 100% = 0.453% [7] The CV tells us that our standard deviation is only
,  0.453% as large as the mean. For our example, that num ‘ _ her is small, which indicates a high level of precision or
improducibility of the replicates. As a rule, a CV below
,' ‘_ ~ 15:" 5% is considered acceptable, although it depends on the
‘ L a of analysis. Another way to evaluate the meaning of the stan
9 ,dard deviation is to examine its origin in statistical
_ ﬂ'theory. Many populations (in our case, sample values
3 ‘y _, _ ‘ 01' means) that exist in nature are said to have a normal 1 u distribution. If we were to measure an infinite num ﬁber of samples, we would get a distribution similar to I I that represented by Fig. 4—2. In a population with a nor ” ‘ _ mal distribution, 68% of those values would be within
‘ " ._ ,:l:'l standard deviation from the mean; 95% would be ‘ Within :l:3 standard deviations. In other words, there is
.a probability of less than 1% that a sample in a popu _ I ,lation would fall outside :l:3 standard deviations from
E  I iglhe mean value. ‘ :Within :l:2 standard deviations, and 99.7% would be ‘ a group of analyses. 4'2 Values for Z for Checking both
It!!! Upper and Lower Levels Degree of Certainty Z Value
(Confidence) 80% 1 .29
90% 1 .64
95% 1 .96
99% 2.58
99.9% 3.29 Another way of understanding the normal distri
bution curve is to realize that the probability of ﬁnding
the true mean is within certain confidence intervals as
defined by the standard deviation. For large numbers
of samples, we can determine the conﬁdence limit or
interval around the mean using the statistical param—
eter called the Z value. We do this calculation by ﬁrst
looking up the Z value from statistical tables once we
have decided the desired degree of certainty. Some Z
values are listed in Table 42. The confidence limit (or interval) for our moisture
data, assuming a 95% probability, is calculated accord
ing to Equation [8]. Since this calculation is not valid
for small numbers, assume we ran 25 samples instead
of four. Confidence Interval (CI) = 5c i Z value
standard deviation (SD) X ﬂ
[8]
CI( t95‘7) 64 72 :l: 1 96 x 02927
a o — . . m
= 64.72 :t 0.115% [9] Partl 0 General Information 4‘3 Values of "or Various Levels of m Probability'
Degrees of Levels of Certainty
Freedom
(n ~ 1) 95% 99% 99.9%
1 12.7 63.7 636
2 4.30 9.93 31.60
3 3.18 5.84 12.90
4 2.78 4.60 8.61
5 2.57 4.03 6.86
6 2.45 3.71 5.96
7 2.36 3.50 5.40
8 2.31 3.56 5.04
9 2.26 3.25 4.78
10 2.23 3.17 4.59 1 More extensive ttables can be found in statistics books. Because our example had only four values for the mois—
ture levels, the conﬁdence interval should be calculated
using statistical t tables. In this case, we have to look
up the t value from Table 4—3 based on the degrees of
freedom, which is the sample size minus one (71 — 1),
and the desired level of confidence. The calculation for our moisture example with four
samples (n) and 3 degrees of freedom (11 — 1) is given
below. standard deviation (SD) CI : 5c :l: t value X ——Jﬁ—— [10]
CI (at 95%) = 64.72 :i: 3.18 X 02927
J4
= 64.72 :I: 0.465% [11] To interpret this number, we can say that, with 95% con—
fidence, the true mean for our moisture will fall within
64.72 :t: 0.465% or between 65.185 and 64.255%. The expression SD / ¢n is often reported as the stan
dard error of the mean. It then is left to the reader to
calculate the confidence interval based on the desired
level of certainty. Other quick tests of precision used are the relative
deviation from the mean and the relative average devi
ation from the mean. The relative deviation from the
mean is useful when only two replicates have been per
formed. It is calculated according to Equation [12], with
values below 2% considered acceptable. xi—ic Relative deviation from the mean = x 100 [12] x The xi represents the individual sample value, and 5c is
the mean. If there are several experimental values, then the
relative average deviation from the mean becomes a useful indicator of precision. It is calculated similarly to
the relative deviation from the mean, except the average
deviation is used instead of the individual deviation. It
is calculated according to Equation [13]. Relative average deviation from the mean = x 1000
n/x parts per thousand ' [13] Using the moisture values discussed in Table 4—1,
the xi —5c terms for each determination are —0.19, —0.27,
+0.38, +0.06. Thus, the calculation becomes: 0.19 + 0.27 l 0.38 + 0.06
R l. . . = ———— 1000
e avg dev 11/6472 x
0.225
_ 64.72 X 1000
= 3.47 parts per thousand [14] Up to now, our discussions of calculations have
involved ways to evaluate precision. If the true value
is not known, we can calculate only precision. A low
degree of precision would make it difficult to predict a
realistic value for the sample. However, we occasionally may have a sample for
which we know the true value and can compare our
results with the known value. In this case, we can cal
culate the error for our test, compare it to the known
value, and determine the accuracy. One term that can
be calculated is the absolute error, which is simply the
difference between the experimental value and the true
value. Absolute error 2 Babs = x — T [15] where:
x = experimentally determined value T = true value The absolute error term can have either a positive or
negative value. If the experimentally determined value
is from several replicates, then the mean (0) would be
substituted for the x term. This is not a good test for
error, because the value is not related to the magnitude
of the true value. A more useful measurement of error
is relative error. E x — T
Relative error = Ere] = [25 = T T [161 The results are reported as a negative or positive value,
which represents a fraction of the true value. If desired, the relative error can be expressed as
percent relative error by multiplying by 100%. Then
the relationship becomes the following, where x can be w——— Chapter4 0 Evaluation of Analytical Data 57 either an individual determination or the mean (0) of
several determinations. ' x—T abs %E,el = x 100% = x 100% [17] Using the data for the percent moisture of uncooked
hamburger, suppose the true value of the sample is
65.05%. The percent relative error is calculated using
our mean value of 64.72% and Equation [17]. 5c — T 64.72 — 65.05 X 100% = 65.05 x 100% %Erel = = —0.507% [18] Note that we keep the negative value, which indicates
the direction of our error, that is, our results were
0.507% lower than the true value. 4.3.2 Sources of Errors As you may recall from our discussions of accuracy and
precision, error (variation) can be quite important in
analytical determinations. Although we strive to obtain
correct results, it is unreasonable to expect an analyt—
ical technique to be entirely free of error. The best we
can hope for is that the variation is small and, if possi
ble, at least consistent. As long as we know about the
‘ error, the analytical method often will be satisfactory.
There are several sources of error, which can be clas—
siﬁed as: systematic error (determinate), random error
 (indeterminate), and gross error or blunders. Again,
note that error and variation are used interchangeably
in this section and essentially have the same meaning
for these discussions. Systematic or determinate error produces results
that consistently deviate from the expected value in
one direction or the other. As illustrated in Fig. 41b,
the results are spaced closely together, but they are
consistently off the target. Identifying the source of
this serious type of error can be difficult and time ., consuming, because it often involves inaccurate instru
ments or measuring devices. For example, a pipette
that consistently delivers the wrong volume of reagent
Will produce a high degree of precision yet inaccurate
results. Sometimes impure Chemicals or the analyt
ical method itself are the cause. Generally, we can
overcome systematic errors by proper calibration of
instruments, running blank determinations, or using
a different analytical method. Random or indeterminate errors are always
\present in any analytical measurement. This type of error is due to our natural limitations in measuring a particular system. These errors fluctuate in a random ' fashion and are essentially unavoidable. For example, reading an analytical balance, judging the endpoint ‘
7 change in a titration, and using a pipette all contribute ‘ 4—_ to random error. Background instrument noise, which
is always present to some extent, is a factor in random
error. Both positive and negative errors are equally pos—
sible. Although this type of error is difficult to avoid,
fortunately it is usually small. Blunders are easy to eliminate, since they are ’so obvious. The experimental data are usually scat tered, and the results are not close to an expected
value. This type of error is a result of using the wrong
reagent or instrument or of sloppy technique. Some
people have called this type of error the “Monday
morning syndrome” error. Fortunately, blunders are
easily identified and corrected. ‘ 4.3.3 Specificity Specificity of a particular analytical method means that
it detects only the component of interest. Analytical
methods can be very specific for a certain food compo
nent or, in many cases, can analyze a broad spectrum of
components. Quite often, it is desirable for the method
to be somewhat broad in its detection. For example, the
determination of food lipid (fat) is actually the crude
analysis of any compound that is soluble in an organic
solvent. Some of these compounds are glycerides, phos—
pholipids, carotenes, and free fatty acids. Since we are
not concerned about each individual compound when
considering the crude fat content of food, it is desirable
that the method be broad in scope. On the other hand,
determining the lactose content of ice cream would
require a specific method. Because ice cream contains
other types of simple sugars, without a specific method
we would overestimate the amount of lactose present. There are no hard rules for what speciﬁcity is
required. Each situation is different and depends on
the desired results and type of assay used. However, it
is something to keep in mind as the various analytical
techniques are discussed. 4.3.4 Sensitivity and Detection Limit Although often used interchangeably, the terms sensi
tivity and detection limit should not be confused. They
have different meanings, yet are closely related. Sensi
tivity relates to the magnitude of change of a measuring
device (instrument) with changes in compound con—
centration. It is an indicator of how little change can
be made in the unknown material before we notice a
difference on a needle gauge or a digital readout. We
are all familiar with the process of tuning in a radio sta—
tion on our stereo and know how, at some point, once
the station is tuned in, we can move the dial without
disturbing the reception. This is sensitivity. In many
situations, we can adjust the sensitivity of an assay to
fit our needs, that is, whether we desire more or less
sensitivity. We even may desire a lower sensitivity so 58 Partl 0 General Information ‘\ that samples with widely varying concentration can be
analyzed at the same time. ' Detection limit, in contrast to sensitivity, is the low
est possible increment that we can detect with some
degree of conﬁdence (or statistical significance). With
every assay, there is a lower limit at which point we are
not sure if something is present or not. Obviously, the
best choice would be to concentrate the sample so we
are not working close to the detection limit. However,
this may not be possible, and we may need to know the
detection limit so we can work away from that limit. There are several ways to measure the detection
limit, depending on the apparatus that is used. If we
are using something like a spectrophotometer, gas chro
matograph, or high performance liquid chromatograph
(HPLC) the limit of detection often is reached when
the signal to noise ratio is 3 or greater. In other words,
when the sample gives a value that is three times the
magnitude of the noise detection, the instrument is at
the lowest limit possible. Noise is the random signal
ﬂuctuation that occurs with any instrument. A more general way to define the limit of detection
is to approach the problem from a statistical viewpoint,
in which the variation between samples is considered.
A common mathematical deﬁnition of detection limit
is given below. XLD = XBlk + 3 X SDBlk [19] where:
XLD = the minimum detectable concentration Xglk = the signal of a blank
SDglk = the standard deviation of the
blank readings In this equation, the variation of the blank values
(or noise, if we are talking about instruments) deter
mines the detection limit. High variability in the blank
values decreases the limit of detection. 4.4 CURVE FITTING: REGRESSION
ANALYSIS '————_— Curve ﬁtting is a generic term used to describe the rela
tionship and evaluation between two variables. Most
scientific fields use curve fitting procedures to evaluate
the relationship of two variables. Thus, curve ﬁtting or
curvilinear analysis of data is a vast area as evidenced
by the volumes of material describing these procedures.
In analytical determinations, we are usually concerned
with only a small segment of curvilinear analysis, the
standard curve or regression line. A standard curve or calibration curve is used to
determine unknown concentrations based on a method
that gives some type of measurable response that is
proportional to a known amount of standard. It typ
ically involves making a group of known standards in increasing concentration and then recording the partic
ular measured analytical parameter (e. g., absorbance,
area of a chromatography peak, etc.). What results
when we graph the paired x and y values is a scatterplot
of points that can be joined together to form a straight
line relating concentration to observed response. Once
we know how the observed values change with concen—
tration, it is fairly easy to estimate the concentration of
an unknown by interpolation from the standard curve. As you read through the next three sections, keep in
mind that not all correlations of observed values to stan
dard concentrations are linear (but most are). There are
many examples of nonlinear curves, such as antibody
binding, toxicity evaluations, and exponential growth
and decay. Fortunately, with the vast array of computer
software available today, it is relatively easy to analyze
any group of data. 4.4.1 Linear Regression So how do we set up a standard curve once the data
have been collected? First a decision must be made
regarding onto which axis to plot the paired sets of data.
Traditionally the concentration of the standards is rep
resented on the xaxis and the observed readings are
on the yaxis. However, this protocol is used for rea—
sons other than convention. The xaxis data are called
the independent variable and assumed to be essen
tially free of error, while the yaxis data (the dependent
variable) may have error associated with them. This
assumption may not be true because error could be
incorporated as the standards are made. With modern
day instruments the error can be very small. Although
arguments can be made for making the yaxis data con—
centration, for all practical purposes the end result is
essentially the same. Unless there are some unusual
data, the concentration should be associated with the xaxis
and the measured values with the yaxis. Figure 4—3 illustrates a typical standard curve used
in the determination of caffeine in various foods.
Caffeine is analyzed readily in foods by using HPLC
coupled with an ultraviolet detector set at 272 nm.
The area under the caffeine peak at 272 nm is directly
proportional to the concentration. When an unknown
sample (e.g., coffee) is run on the HPLC, a peak area is
obtained that can be related back to the sample using
the standard curve. The plot in Fig. 4—3 shows all the data points and
a straight line that appears to pass through most of
the points. The line almost passes through the origin,
which makes sense because zero concentration should
produce no signal at 272 nm. However, the line is not
perfectly straight (and never is) and does not quite pass
through the origin. To determine the caffeine concentration in a sample
that gave an area of say 4000, we could extrapolate to the Chapter4 0 Evaluation otAnaIytical Data 59 Peak Area 6 272 nm ‘0 60 Concentration (ppm) 80 A typical standard curve plot showing the data
points and the generated best ﬁt line. The data
used to plot the curve are presented on the graph. line and then draw a line down to the xaxis. Following
a line to the xaxis (concentration), we can estimate the
solution to be at about 42—43 ppm of caffeine. We can mathematically determine the best fit of the
line by using linear regression. Keep in mind the equa
tion for a straight line, which is: y = ax + b, where a =
slope and b = yintercept. To determine the slope and
yintercept, the regression equations shown below are
used. We determine a and b and, thus, for any value of
X y (measured), we can determine the concentration (x). 2(xi h 5C)(yi  3?)
mm—mz yintercept b = g — aft [21] slope a = [20] The xi and yi parameters are the individual values, and
5c and g are the means of the individual values. Low
Cost calculators and computer spreadsheet software can
, readily calculate regression equations so no attempt is
made to go through the mathematics in the formulas.
The formulas give what is known as the line of
regression of y on x which assumes that the error occurs
in the y direction. The regression line represents the
average relationship between all the data points and
thus is a balanced line. These equations also assume
that the straight line fit does not have to go through
the origin, which at first does not make much sense.
However, there are often background interferences so
that even at zero concentration a weak signal may be
‘ Observed. In most situations, calculating the origin as
going through zero will yield the same results. >
Using the data from Fig. 4—3, calculate the concen
tration of caffeine in the unknown and compare with
the graphing method. As you recall, the unknown had
an area at 272 nm of 4000. Linear regression analysis Y
x x
A4_4 Examples of standard curves showing the rela
tionship between the x and y variables when
IE3. there is (a) a high amount of correlation and
(b) a lower amount of correlation. Both lines
have the same equation. of the standard curve data gave the yintercept (b) as
84.66118 and the slope (a) as 90.07331. y = ax + b [22]
or b x=y_ mm a 4000 ~ 84.66118 = —— = 4 .4 ff ' x (conc) 90.07331 3 68 ppm ca e1n[e ]
24 The agreement is fairly close when comparing the
calculated value to that estimated from the graph.
Using high—quality graph paper with many lines could
give us a line very close to the calculated one. However,
as we will see in the next section, additional informa
tion can be obtained about the nature of the line when
using computer software or calculators. 4.4.2 Correlation Coefficient In observing any type of correlation, including linear
ones, questions always surface concerning how to draw
the line through the data points and how well the data
fit to the straight line. The first thing that should be done
with any group of data is to plot it to see if the points
fit a straight line. By just looking at the plotted data,
it is fairly easy to make a judgment on the linearity
of the line. We also can pick out regions on the line
where a linear relationship does not exist. The figures
below illustrate differences in standard curves; Fig. 4—4a
shows a good correlation of the data and Fig. 44b shows
a poor correlation. In both cases, We can draw a straight
line through the data points. Both curves yield the same
straight line but the precision is poorer for the latter.
There are other possibilities when working with
standard curves. Figure 4—5a shows a good correlation
between x and 3/ but in the negative direction, and
Fig. 4—5b illustrates data that have no correlation at all. 60 Partl 0 General Information X X 4_5 , Examples of standard curves showing the rela—
tionship between the x and y variables when
m there is (a) a high amount of negative correlation
and (b) no correlation between x and y values. The correlation coefﬁcient defines how well the
data fit to a straight line. For a standard curve, the
ideal situation would be that all data points lie per
fectly on a straight line. However, this is never the case,
because errors are introduced in making standards and
measuring the physical values (observations). The correlation coefficient and coefficient of deter
mination are defined below. Essentially all spread sheet and plotting software will calculate the values
automatically. correlation coefficient = r = v [2(xi — 3021mm — W] [25] For our example of the caffeine standard curve from
Fig. 4—3: r = 0.99943 (values are usually reported to at least
4 significant figures) For standard curves, we want the value of r as close
to +1.0000 or —1.000 as possible, because this value is a
perfect correlation (perfect straight line). Generally, in
analytical work, the r should be 0.9970 or better. (This
does not apply to biological studies.) The coefficient of determination (r2) is used quite
often because it gives a better perception of the straight
line even though it does not indicate the direction of
the correlation. The r2 for the example presented above
is 0.99886, which represents the proportiOn of the vari
ance of absorbance (y) that can be attributed to its linear
regression on concentration (x). This means that about
0.114% of the straight line variation (1.0000 — 0.99886 2
0.00114 X 100% = 0.114%) does not vary with changes
in x and y and, thus, is due to indeterminate variation.
A small amount of variation is expected normally. 4.4.3 Errors in Regression Lines While the correlation coefficient tells us something
about the error or variation in linear curve fits, it does __——————— not always give the complete picture. Also, neither lin
ear regression nor correlation coefficient will indicate
that a particular set of data have a linear relationship.
They only provide an estimate of the fit assuming the
line is a linear one. As indicated before, plotting the data
is critical when looking at how the data fit on the curve
(actually, a line). One parameter that is used often is the
y—residuals, which are simply the differences between
the observed values and the calculated or computed
values (from the regression line). Advanced computer
graphics software can actually plot the residuals for
each data point as a function of concentration. How
ever, plotting the residuals is usually not necessary
because data that do not ﬁt on the line are usually quite
obvious. If the residuals are large for the entire curve,
then the entire method needs to be evaluated carefully.
However, the presence of one point that is obviously off
the line while the rest of the points fit very well probably
indicates an improperly made standard. One way to reduce the amount of error is to include
more replicates of the data such as repeating the obser
vations with a new set of standards. The replicate
x and 3/ values can be entered into the calculator or
spreadsheet as separate points for the regression and
coefficient determinations. Another, probably more
desirable, option is to expand the concentrations at
which the readings are taken. Collecting observations at
more data points (concentrations) will produce a better
standard curve. However, increasing the data beyond
seven or eight points usually is not beneficial. Plotting conﬁdence intervals, or bands or limits,
on the standard curve along with the regression line is
another way to gain insight into the reliability of the
standard curve. Confidence bands define the statistical
uncertainty of the regression line at a chosen probabil—
ity (such as 95%) using the t—statistic and the calculated
standard deviation of the fit. In some aspects, the con
ﬁdence bands on the standard curve are similar to the
confidence interval discussed in section 4.3.1. However,
in this case we are looking at a line rather than a con
fidence interval around a mean. Figure 4—6 shows the
caffeine data from the standard curve presented before,
except some of the numbers have been modified to
enhance the confidence bands. The confidence bands
(dashed lines) consist of both an upper limit and a lower
limit that define the variation of the yaxis value. The
upper and lower bands are narrowest at the center of
the curve and get wider as the curve moves to the higher
or lower standard concentrations. Looking at Fig. 4—6 again, note that the confidence
bands show what amount of variation we expect in
a peak area at a particular concentration. At 60 ppm
concentration, by going up from the xaxis to the bands
and extrapolating to the y—axis, we see that with our
data the 95% confidence interval of the observed peak
area will be 4000 to 6000. In this case, the variation is P Chapter4 0 Evaluation of Analytical Data 61 ’______________________________———_————— Upper limit . E. I ,
sooo y s 30.386 (.x) . 244.29 \, I
r = 0.98183 1 E 5000 ’ ’ , ’ \
N y
.a ,
N  Lower limit
a z , I
a 7
2 “mo I 2 Peak
< I :
x : WM
3 I ; 5 :no
“ 2000 , ’ 5 25 2100 r g 50 4795 ' 75 6950 7565 o 20 40 so so too
Concentration (ppm) 4 _ 6 A standard curve graph showing the conﬁdence bands. The data used to plot the graph are pre
sented on the graph as are the equation of the
line and the correlation coefficient. Real
Outlier Measured Value Concentration 4_7 A standard curve plot showing possible devi
ations in the curve in the upper and lower
limits. { large and would not be acceptable as a standard curve,
‘ and is presented here only for illustration purposes. g Error bars also can be used to show the variation of I 'y at each data point. Several types of error or variation
' statistics can be used such as standard error, stan— dard deviation, or percentage of data (i.e., 5%). Any of these methods give a visual indication of experimental variation. Even with good standard curve data, problems
can arise if the standard curve is not used properly.
One common mistake is to extrapolate beyond the data
points used to construct the curve. Figure 4—7 illustrates
Some of the possible problems that might occur when
extrapolation is used. As shown in Fig. 47, the curve or
line may not be linear outside the area where the data were collected. This can occur in the region close to the
origin or especially at the higher concentration level.
Usually a standard curve will go through the ori
gin, but in some situations it may actually tail off as zero
concentration is approached. At the other end of the
curve, at higher concentrations, it is fairly common for
a plateau to be reached where the measured parameter
does not change much with an increase in concentra
tion. Care must be used at the upper limit of the curve
to ensure that data for unknowns are not collected out
side of the curve standards. Point Z on Fig. 47 should
be evaluated carefully to determine if the point is an
outlier or if the curve is actually tailing off. Collect
ing several sets of data at even higher concentrations
should clarify this. Regardless, the unknowns should
be measured only in the region of the curve that is linear. 4.5 REPORTING RESULTS In dealing with experimental results, we always are
confronted with reporting data in a way that indicates
the sensitivity and precision of the assay. Ideally, we do
not want to overstate or understate the sensitivity of the
assay, and thus strive to report a meaningful value, be
it a mean, standard deviation, or some other number.
The next three sections discuss how we can evaluate
experimental values so as to be precise when reporting
results. 4.5.1 Significant Figures The term signiﬁcant ﬁgure is used rather loosely to
describe some judgment of the number of reportable
digits in a result. Often, the judgment is not soundly
based, and meaningful digits are lost or meaningless
digits are retained. Exact rules are provided below to
help determine the number of signiﬁcant figures to
report. However, it is important to keep some ﬂexibility
when working with significant figures. Proper use of significant figures is meant to give an
indication of the sensitivity and reliability of the ana
lytical method. Thus, reported values should contain
only significant figures. A value is made up of signiﬁ
cant figures when it contains all digits known to be true
and one last digit that is in doubt. For example, a value
reported as 64.72 contains four significant ﬁgures, of
which three digits are certain (64.7) and the last digit is
uncertain. Thus, the 2 is somewhat uncertain and could
be either 1 or 3. As a rule, numbers that are presented
in a value represent the significant figures, regardless
of the position of any decimal points. This also is true ' for values containing zeros, provided they are bounded on either side by a number. For example, 64.72, 6.472,
0.6472, and 6.407 all contain four signiﬁcant figures.
Note that the zero to the left of the decimal point is — 62 Partl 0 General Information M used only to indicate that there are no numbers above
1. We could have reported the value as .6472, but using
the zero is better, since we know that a number was not
inadvertently left off our value. Special considerations are necessary for zeros that
may or may not be signiﬁcant. 1. Zeros after a decimal point are always signifi
cant figures. For example, 64.720 and 64.700 both
contain five significant figures. 2. Zeros before a decimal point with no other pre
ceding digits are not significant. As indicated
before, 0.6472 contains four significant figures. 3. Zeros after a decimal point are not significant if
there are no digits before the decimal point. For
example, 0.0072 has no digits before the decimal
point; thus, this value contains two significant
figures. In contrast, the value 1.0072 contains five
significant ﬁgures. 4. Final zeros in a number are not significant
unless indicated otherwise. Thus, the value 7000
contains only one significant figure. However,
adding a decimal point and another zero gives
the number 7000.0, which has five significant
figures. A good way to measure the significance of zeros,
if the above rules become confusing, is to convert the
number to the exponential form. If the zeros can be
omitted, then they are not significant. For example,
7000 expressed in exponential form is 7 x 103 and con
tains one significant ﬁgure. With 70000, the zeros are
retained and the number becomes 7.0000 x 103. If we
were to convert 0.007 to exponent form, the value is
7 x 10—3, and only one significant figure is indicated.
As a rule, determining significant figures in arithmetic
operations is dictated by the value having the least
number of significant figures. The easiest way to avoid
any confusion is to perform all the calculations and then
round off the final answer to the appropriate digits. For
example, 36.54 x 238 x 1.1 = 9566.172, and because
1.1 contains only two signiﬁcant figures, the answer
would be reported as 9600 (remember, the two zeros are
not significant). This method works fine for most cal
culations, except when adding or subtracting numbers
containing decimals. In those cases, the number of sig—
nificant figures in the final value is determined by the
numbers that follow the decimal point. Thus, adding
7.45 + 8.725 = 16.175; because 7.45 has only two num
bers after the decimal point, the sum is rounded to 16.18.
Likewise, 433.8 — 32.66 gives 401.14, which rounds off
to 401.1. A word of caution is warranted when using the
simple rule stated above, for there is a tendency to
underestimate the significant figures in the final answer.
For example, take the situation in which we determined
the caffeine in an unknown solution to be 43.5 ppm ________ (see Equation [24]). We had to dilute the sample 50fold
using a volumetric ﬂask in order to fit the unknown
within the range of our method. To calculate the caffeine
in the original sample, we multiply our result by 50 or
43.5 jig/m1 x 50 = 2175 jig/ml in the unknown. Based
on our rule above, we then would round the number
to one significant figure (because 50 contains one sig
nificant figure) and report the value as 2000. However,
doing this actually underestimates the sensitivity of our
procedure, because we ignore the accuracy of the volu
metric ﬂask used for the dilution. A ClassA volumetric
ﬂask has a tolerance of 0.05 ml; thus, a more reasonable
way to express the dilution factor would be 50.0 instead
of 50. We now have increased the signiﬁcant figures in
the answer by two, and the value becomes 21 80 mg / ml. As you can see, an awareness of significant figures
and how they are adopted requires close inspection.
The guidelines can be helpful but they do not always
work, unless each individual value or number is closely
inspected. 4.5.2 Rejecting Data Inevitably, during the course of working with experi
mental data we will come across a value that does not
match the others. Can you reject that value, and thus
not use it in calculating the final reported results? The answer is "sometimes," but only after careful
consideration. If you are routinely rejecting data to help
make your assay look better, then you are misrepresent
ing the results and the precision of the assay. If the bad
value resulted from an identifiable mistake in that par
ticular test, then it is probably safe to drop the value.
Again, caution is advised, because you may be reject
ing a value that is closer to the true value than some of
the other values. Consistently poor accuracy or precision indicates
that an improper technique or incorrect reagent was
used or that the test was not very good. It is best to make
changes in the procedure or change methods rather
than try to figure out ways to eliminate undesirable
values. There are several tests for rejecting an aberrant
value. One of these tests, the Qtest, is commonly used.
In this test, a Qvalue is calculated as shown below and
compared to values in a table. If the calculated value is
larger than the table value, then the questionable mea
surement can be rejected at the 90% conﬁdence level. 962—361 Qvalue = [26] where:
x1 = the questionable value x2 = the next closest value to x1 W=the total spread of all values; obtained
by subtracting the lowest value from the
highest value. w Chapter4 0 Evaluation of Analytical Data 63 4‘4 _ Q—Values for the Rejection
ot Results Number of O of Rejection
Observations (90% level) 3 0.94 4 0.76 5 0.64 6 0.56 7 0.51 8 0.47 9 0.44
‘l 0 0.41 Reprinted with permission from RB. Dean and
W..J. Dixon. 1951. Simplified statistics for small num
bers of observations. Analytical Chemistry 23: 686—
638. Copyright 1951, American Chemical Society. Table 4—4 provides the rejection Qvalues for a 90%
confidence level. The example below shows how the test is used for the moisture level of uncooked hamburger for which
four replicates were performed giving values of 64.53,
64.45, 64.78, and 55.31. The 55.31 value looks as if it is
too low compared to the other results. Can that value
be rejected? For our example, x1 = the questionable
value = 55.31, and xz is the closest neighbor to x1
(which is 64.45). The spread (W) is the high value minus
the low measurement, which is 64.78 — 55.31. 64.45 — 55.31 9.14
64.78 — 55.31 ‘ E1? 2 0'97 [27] Q—value =
From Table 4—4, we see that the calculated Qvalue
must be greater than 0.76 to reject the data. Thus, we
make the decision to reject the 55.31% moisture value
and do not use it in calculating the mean. 4.6 SUMMARY This chapter focused on the basic mathematical treat
ment that most likely will be used in evaluating a
group of data. For example, it should be almost second
nature to determine a mean, standard deviation, and
CV when evaluating replicate analyses of an individ
ual sample. In evaluating linear standard curves, best
line ﬁts always should be determined along with the
indicators of the degree of linearity (correlation coeffi
cient or coefficient of determination). Fortunately, most
computer spreadsheet and graphics software will read
ily perform the calculations for you. Guidelines are
available to enable one to report analytical results in
a way that tells something about the sensitivity and confidence of a particular test. These include the proper
use of significant figures, rules for rounding off num bers, and use of the Q—test to reject grossly aberrant
individual values. 4.7 STUDY QUESTIONS 1. Method A to quantitate a particular food component was
reported to be more specific and accurate than method
B, but method A had lower precision. Explain what this
means. 2. You are considering adopting a new analytical method
in your lab to measure moisture content of cereal prod
ucts. How would you determine the precision of the new
method and compare it to the old method? Include any
equations to be used for any needed calculations. 3. A sample known to contain 20 g/ L glucose is analyzed
by two methods. Ten determinations were made for each
method and the following results were obtained: Method A Method B Mean 2 19.6
Std. Dev. = 0.055 Mean = 20.2
Std. Dev. 2 0.134 a. Precision and accuracy: i. Which method is more precise? Why do you say
this? ii. Which method is more accurate? Why do you say
this? b. In the equation to determine the standard deviation,
n — 1 was used rather than just 11. Would the standard
deviation have been smaller or larger for each of those
values above if simply 11 had been used? c. You have determined that values obtained using
Method B should not be accepted if outside the range
of two standard deviations from the mean. What range
of values will be acceptable? d. Do the data above tell you anything about the speci
ficity of the method? Describe what “specificity” of the
method means, as you explain your answer. 4. Differentiate “standard deviation” from "coefficient of
variation,” “standard error of the mean,” and "confidence
interval.” 5. Differentiate the terms “absolute error” versus “relative
error.” Which is more useful? Why? 6. For each of the errors described below in performing
an analytical procedure, classify the error as random
error, systematic error, or blunder, and describe a way to
overcome the error.
at. Automatic pipettor consistently delivered 0.96 ml rather than 1.00 ml. b. Substrate was not added to one tube in an enzyme assay. 7. Differentiate the terms “sensitivity” and “detection limit.” 8. The correlation coefficient for standard curve A is reported
as 0.9970. The coefficient of determination for standard
curve B is reported as 0.9950. In which case do the data
better fit a straight line? — 64 pan] o General lnformatio 4.3 PRACTICEPROBLEMS 1. How many significant ﬁgures are in the following num— ’ bers: 0.0025, 4.50, 5.607? 2. What is the correct answer for the following calculation
expressed in the proper amount of significant ﬁgures? 2.43'x 0.01672 _
1.83215 ‘ 3. Given the following data on dry matter (88.62, 88.74, 89.20,
82.20), determine the mean, standard deviation, and CV.
Is the precision for this set of data acceptable? Can you
reject the value 82.20 since it seems to be different than
the others? What is the 95% confidence level you would
expect your values to fall within if the test were repeated?
If the true value for dry matter is 89.40, what is the percent
relative error? 4. Compare the two groups of standard curve data below for
sodium determination by atomic emission spectroscopy.
Draw the standard curves using graph paper or a com—
puter software program. Which group of data provides
a better standard curve? Note that the absorbance of the
emitted radiation at 589 nm increases proportionally to
sodium concentration. Calculate the amount of sodium in
a sample with a value of 0.555 for emission at 589 nm. Use
both standard curve groups and compare the results. Sodium Concentration Emission at
(pug/ml) 589 nm
Group A—Sodium Standard Curve
1.00 0.050 
3.00 0.140
5.00 0.242
10.0 0.521
20.0 0.998
Group B—Sodium Standard Curve
1.00 0.060
3.00 0.113
5.00 0.221
10.0 0.592
20.0 0.917 Answers 1. 2, 3, 4 2. 0.0222 3. mean : 87.19, SDn_1 : 3.34, CV = 3.83%; thus the prec
sion is acceptable. Qcalc = 0.92, therefore the value 82.2
can be rejected. CI = 87.19 :t 5.31. %E,el = —2.47%. 4. Group A is the better standard curve (group A, r2 : 0.9991 group B, r2 = 0.9708). Sodium in the sample using grou
A standard curve : 11.1 ug/ ml; with group B standar
curve = 11.5 ug/ml. ‘ 4.9 RESOURCE MATERIALS 1. Garﬁeld, F.M., Klestra, E., and Hirsch, ]. 2000. Qua
ity Assurance Principles for Analytical Laboratories. AOA
International, Gaithersburg, MD. This book covers most
quality assurance issues but has a good chapter (chapter
on the basics of statistical applications to data. 2. Meier, RC, and Richard EZ. 2000. Statistical Methods
Analytical Chemistry, 2nd ed. John Wiley & Sons, New Yor
This is another excellent text for beginner and advancr
analytical chemists alike. It contains a fair amount of deta
sufficient for most analytical statistics, yet works throug
the material starting at a basic introductory level. T]
authors also discuss the Q—test used for rejecting da:
A big plus of this book is the realistic approach tak;
throughout. 3. Skoog, D.A., West, D.M., Holler, ].F., Crouch, S.
2000. Analytical Chemistry: An Introduction, 7th e
Brooks/ Cole, Pacific Grove, CA. Section 1, chapters ,
7 do an excellent job of covering most of the stat:
tics needed by an analytical chemist in an easy—torea
style. 4. Willard, H.H., Merritt, L.L., In, Dean, ].A., and Sett
F.A., Ir. 1988. Instrumental Methods of Analysis, 7th e
Wadsworth Publishing, Belmont, CA. This gives a rig
ous treatment of instrumentation and has a very use!
chapter (chapter 2, Measurements, Signals and Data) r
types of error generated by instruments. ...
View
Full
Document
 Spring '10
 Pegg
 Normal Distribution, Regression Analysis, Standard Deviation, Evaluation of Analytical Data

Click to edit the document details