of Review Probability Theory
CWR 6536 Stochastic Subsurface Hydrololgy
Random Variable (r.v.)
A variable (x) which takes on values at random, and may be thought of as a function of the outcomes of some random experiment. The r.v. maps sample space of experiment onto the real line The probability with which different values are taken by the r.v. is defined by the cumulative distribution function, F(x), or the probability density function, f(x).
Examples
Discontinuous r.v. - die tossing experiment
Examples
Categorical r.v. An observation, s(), that can take on any of a finite number of mutually exclusive, exhaustive states (sk) , e.g. soil type, land use, landscape position An indicator random variable can be defined
i ( , sk ) = 1 if s( ) = sk = 0 otherwise
The frequency of occurrence of a state f (sk) can be determined as the arithmetic average of n indicator data (i(,sk) )where:
1 n f ( sk ) = i ( , s k ) n =1
The joint frequency of two states sk and vk is
1 n f (s , v ) = i , s k i , vk k k n = 1
(
)(
)
Frequency Table for Categorical Data
Soil type Sand Silt Loam Clay Frequency 20% 33% 25% 22% Land Use Forest Pasture Meadow Frequency 14% 21% 65%
The probability distribution of categorical data is completely described by a frequency table
Probability Density Function (pdf)
The function f(x) is a pdf for the continuous random variable x, defined over the set of real numbers R, if
1) 2) 3) f ( x) 0 for all x R f ( x)dx = 1 - b P (a < x < b) = f ( x)dx, a
i.e. f ( x ) x = P x < x < x + x 0 0 0
[
]
Cumulative Distribution Function (cdf)
The cdf of a continuous r.v. x with a pdf f(x) is given by: x
F ( x0 ) = P( x x0 ) = f ( x)dx
-
0
Therefore
Properties of the cdf:
F ( x1 ) F ( x 2) for x1 x 2 F ( - ) =0, F ( = , ) 1
dF ( x) f ( x) = dx
0 F ( x1 ) 1
F ( x + =F ( x ) ) b P ( a <x <b) = f ( x ) dx =F(b) - F(a) a
Examples: Continuous r.v.
uniform distribution, exponential distribution, gaussian distribution log-normal distribution
Moments of a Random Variable
The pdf (and cdf) summarize all knowledge of the r.v., however we almost never really know this much about actual natural phenomena. Moments of a r.v. provide a more aggregated description of its behavior is which often easier to estimate from field data than pdf or cdf
The First Moment
The expected value (or first moment, or population mean, or ensemble mean) of a r.v. is defined as the sum of all the values a r.v. may take, each weighted by the probability with which the value is taken
x = E ( x) = xf ( x) dx
-
This quantity is a single valued, deterministic summary of the r.v.
Properties of the Expectation Operator
If the pdf, f(x) is even (i.e. f(x)=f(-x)), then the expected value is equal to zero The expectation is a linear operator The expected value of a function of the r.v.
Higher Order Moments
x = E ( x ) = x f ( x)dx
- n n n
n=1 E[x]=mean (measure of central tendency) n=2 E[x2]= mean square n=3 E[x3]= mean cube
E ( x - x) = ( x - x) f ( x)dx
n=1 n=2 n=3 n=4
1 = E[( x - x)1] = 0 2 = E[( x - x) 2 ] = 2 3 = E[( x - x)3 ] 4 = E[( x - x) 4 ]
[
Central Moments
n
]
n
-
variance skewness kurtosis
It can be shown that the full (infinite) set of moments completely exhausts the statistical information concerning the r.v., and thus the pdf can be constructed from the full set of moments
Joint Probability Distributions
The joint cdf for two random variables, x and y, is
Fxy ( x0 , y0 ) = P( x x0 , y y0 ) =
x0 - -
f
y0
xy
( x, y )dxdy
where
Fxy (- , y ) = Fxy ( x,- ) = 0, Fxy (, ) = 1,
Marginal cdfs and pdfs
cdfs
pdfs
conditional pdfs
Moments of two random variables
E [ xy ] =
- -
xyf xy ( x, y )dxdy
Cov ( x, y ) = E ( x - x)( y - y ) = E[ x / y ] =
[
]
- -
( x - x)( y - y ) f xy ( x, y )dxdy
E [ g ( x, y ) ] =
- -
xf x / y ( x, y )dx g ( x, y ) f xy ( x, y )dxdy
- -
Statistical Independence vs Uncorrelation
Two random variables are statistically independent if Two random variables are uncorrelated if
Two random variables are orthogonal if
Statistical Independence vs Uncorrelation
If two r.v. are independent then they are uncorrelated (but not vice versa) If x and y are independent random variables then g(x) and h(y) are also independent random variables (this is not generally true if x and y are merely uncorrelated) Correlation measures linear relatedness only
An exception is jointly normal r.v.s where uncorrelation is equivalent to independence