Monday, Jan 10th
1. Basic concepts:
Uncertainty / Randomness: The lack of certainty, a state of
having limited knowledge where it is impossible to exactly describe current state or future outcome, more than one possible
outcome (Doug Hubbard). From the po
4001 4002 450
= 0.1 + 0 + 0.1 = 0.2.
P (X = Y ) = pX,Y (400, 400) + pX,Y (450, 450) +
P (X = a) = P (a X a) =
It follows that
P (a X b) =
FX (t) = P (X t) =
Since the density function fX is dened as the derivative of the cumulative
distribution function, we can obtain the cumulative
This distribution is commonly used to model waiting times between
occurrences of rare events, lifetimes of electrical or mechanical devices.
Exponential density A random variable X has exponential density if
ex if x 0
fX ( x) =
Central Limit Theorem (CLT)
Main Idea: Sums and averages of random variables from arbitrary
distributions have approximate normal distributions for suciently large
Suppose X1, X2, . . . , Xn are iid random variables with
Review: X =number of repetitions of the experiment until we have the rst
success in a Bernoulli experiment.
1. pX (k ) = P (X = k ) = (1 p)k1
2. E [X ] = p , Var[X ] =
3. c.d.f. is: FX (t) = P (X t) = 1 (1 p)
Gamma Example (Baron 4.7)
Compilation of a computer program consists of 3 blocks that are processed
sequentially, one after the other. Each block takes Exponential time with
mean of 5 minutes, independently of other blocks.
(a) Compute the expectation and
( Vi V ) 2
i.e. E (V ) = V
Vi = Xi, i = 1, 2, . . . , 100 and Xi is the average of the ith sample. The
larger N is, the more accurate this estimate as the standard error is reduced.
each sample, then average them ov
Outcome ( ), Sample space (), Event (E )
Example: Consider a playo series (maximum 7 games) between
the L.A. Lakers and Boston Celtics. In terms of winning-lose
(LA-BOS) scores, the sample space consists of outcomes
= cfw_(4, 0)
Generating from a Normal distribution (Box-Mller Method): We need two
sequences of standard uniform variables. Let U1 and U2 be two independent
standard uniform variables.
Z1 := [2 ln U1]
Z2 := [2 ln U1]
Review: What is a Random variable?
Denition: A stochastic process is a set of random variables indexed by
some indices, particularly time t, and is usually denoted by X (t).
1. Stochastic process is a mathematical model
Birth and Death Processes
Review: What is a stochastic process? What is a Poisson Process?
Motivation: Birth and Death process (B + D) is a generalization of Poisson
process, and it provides for modeling of queues, i.e. we assume that arrivals
stay some t
Balance equations: In the context of physical-chemistry, it is called the
Flows: The Flow-In = Flow-Out Principle provides us with the means to
derive equations between the steady state probabilities.
1 p1 = 0
The M/M/1 Queue: Example
Printer Queue (continued) A certain printer in the Stat Lab gets jobs with
a rate of 3 per hour. On average, the printer needs 15 min to nish a job.
Let X (t) be the number of jobs in the printer and its queue at time t. We
Queuing Queueing system
enter the system
some population of
according to some
Depending upon the specic application there are many varieties of queuing
size & nature
Question: What is Statistics?
Statistics: is the science and art of studying data. It involves collecting,
classifying, summarizing, organizing, analyzing, and interpreting numerical
More: In general, there are two diere
Other descriptive statistics
Review: Descriptive statistics, inferential statistics, sample/population
mean, sample/population variance, sample/population median, range
Population quantile: A p-quantile of a population is a number x that solves
The M/M/c queue
Again, X (t) the number of individuals in the queueing system can be
modeled as a birth & death process.
The transition state diagram for the X (t) is:
Clearly, the critical thing here in terms of whether or n
Review: What is estimator? What is estimates? What are the properties
we used to compare estimators?
Example: The sample mean x is consistent for . That means that if the
sample size is getting large, then X is getting very closed to in
Examples for CI of proportion
Example 1: Suppose we want to estimate the fraction of records in the
2000 IRS data base that have a taxable income over 35K.
Question: We want to get a 98% condence interval and wish to estimate
the quantity to within 0.01.
Example for MLE:
Review: What is MLE? How to nd it (5 steps)?
may be multiple: Rp with p > 1
Example: Let X1, . . . , Xn be i.i.d N (, 2), both and 2 are unknown.
x1, , xn are the data/sample value of X1, , Xn
What is the pdf of normal random variable?
Concept: Uncertainty/randomness, probability, mathematical model
(random experiment, outcome , sample space , event E).
Set theory: set, operations (belongs, subset, empty set, , , A, A \
B ), disjoint, exhaustive, De Morgan's l
Goodness of t (Contd)
Review: sample correlation as a measure of goodness of t
Second measure of goodness of t: Coecient of determination R2, it is
based on a comparison of variation accounted for by the line versus raw
variation of y .
Ideas: The quantit
Counting: Two principles, Permutation (ordered sample) with replacement, without replacement
Example: A survey question lists seven pizza toppings and ask you
to rank your favorite 3. How many possible an
Example 1: A box has 5 computer chips. Two are defective. Two chips are
selected from the box, one at a time.
1. Compute the probability that the second chip is defective.
Again common sense tells us that P ( a chip is defective) =
Testing Hypothesis (Contd)
Recall: In the previous lecture, 0.021 is called the p-value for testing the
null hypothesis p = 0.5 against an alternative hypothesis p = 0.5.
Formal procedure: Let
H0 : p = 0.5 v.s. H1 : p = 0.5
then the test statistic is T =
Special discrete pmfs
Intuitive idea: In many theoretical and practical problems, several probability
mass functions occur often enough to be worth exploring.
Common feature: The sample space is always nite or countably many.
1. Bernoulli distrib
Two Basic Counting Principles
Summation Principle: If a complex action can be performed using one of
k alternative methods, m1, . . . , mk , and the methods can be performed
in n1, . . . , nk ways, respectively, then the complex action can be peformed
Statistics of R.V.s
Expectation The expected value of a function h(X ) is dened as
E [h(X )] :=
i h( x i )
pX ( x i ) .
The most important version of this is the case h(x) = x:
E [X ] =
pX (xi) =:
E [X ] is usually denoted by the symbol .
Intuitive idea: If the value of a numerical variable depends on the outcome
of an experiment, we call the variable a random variable.
Random Variable A function X : R is called a random variable.
Standard notation: Denote random variables