Monday, Jan 10th
1. Basic concepts:
Uncertainty / Randomness: The lack of certainty, a state of
having limited knowledge where it is impossible to exactly describe current state or future outcome, more than one possible
outcome (Doug Hubbard). From the po
Compound p.m.f.
1st processor
4001 4002 450
x
x
x
x
x
x
x
x
x
x
x
x
5002
x
x
x
x

1st proc.
mHz
400
450
500
2nd
400
0.1
0.1
0.2
processor
450 500
0.1 0.2
0.0 0.1
0.1 0.1
Solution:
= 0.1 + 0 + 0.1 = 0.2.
3
P (X = Y ) = pX,Y (400, 400) + pX,Y (450, 450) +
f (x)dx
= 1.
P (X = a) = P (a X a) =
It follows that
P (a X b) =
t
f (x)dx
b
f (x)dx
a
FX (t) = P (X t) =
a
f (x)dx
a
= 0.
3
Since the density function fX is dened as the derivative of the cumulative
distribution function, we can obtain the cumulative
Exponential Distribution
This distribution is commonly used to model waiting times between
occurrences of rare events, lifetimes of electrical or mechanical devices.
Exponential density A random variable X has exponential density if
ex if x 0
fX ( x) =
0
Central Limit Theorem (CLT)
Main Idea: Sums and averages of random variables from arbitrary
distributions have approximate normal distributions for suciently large
sample sizes.
Suppose X1, X2, . . . , Xn are iid random variables with
E[Xi] =
Dene
Sample
Geometric pmf
Review: X =number of repetitions of the experiment until we have the rst
success in a Bernoulli experiment.
1. pX (k ) = P (X = k ) = (1 p)k1
1
2. E [X ] = p , Var[X ] =
k1 failures
1 p
p2
p
success!
3. c.d.f. is: FX (t) = P (X t) = 1 (1 p)
Gamma Example (Baron 4.7)
Compilation of a computer program consists of 3 blocks that are processed
sequentially, one after the other. Each block takes Exponential time with
mean of 5 minutes, independently of other blocks.
(a) Compute the expectation and
i=1
n
Vi ,
i=1
n
( Vi V ) 2
i.e. E (V ) = V
1
1
N
i=1
N
Vi where
3
Vi = Xi, i = 1, 2, . . . , 100 and Xi is the average of the ith sample. The
larger N is, the more accurate this estimate as the standard error is reduced.
each sample, then average them ov
Wednesday, 12nd
1. Review
Outcome ( ), Sample space (), Event (E )
Example: Consider a playo series (maximum 7 games) between
the L.A. Lakers and Boston Celtics. In terms of winninglose
(LABOS) scores, the sample space consists of outcomes
= cfw_(4, 0)
Using Transformations
Generating from a Normal distribution (BoxMller Method): We need two
u
sequences of standard uniform variables. Let U1 and U2 be two independent
standard uniform variables.
Dene
Z1 := [2 ln U1]
Z2 := [2 ln U1]
1 /2
cos(2U2)
1 /2
s
Stochastic Processes
Review: What is a Random variable?
Denition: A stochastic process is a set of random variables indexed by
some indices, particularly time t, and is usually denoted by X (t).
Some remarks:
1. Stochastic process is a mathematical model
Birth and Death Processes
Review: What is a stochastic process? What is a Poisson Process?
Motivation: Birth and Death process (B + D) is a generalization of Poisson
process, and it provides for modeling of queues, i.e. we assume that arrivals
stay some t
Balance equations:
Balance equations: In the context of physicalchemistry, it is called the
master equation.
Flows: The FlowIn = FlowOut Principle provides us with the means to
derive equations between the steady state probabilities.
State 0:
1 p1 = 0
The M/M/1 Queue: Example
Printer Queue (continued) A certain printer in the Stat Lab gets jobs with
a rate of 3 per hour. On average, the printer needs 15 min to nish a job.
Let X (t) be the number of jobs in the printer and its queue at time t. We
know a
Queuing Queueing system
systems
server 1
enter the system
some population of
individuals
server 2
according to some
random mechanism
exit the
system
server c
Depending upon the specic application there are many varieties of queuing
systems:
size & nature
Statistical inference
Question: What is Statistics?
Statistics: is the science and art of studying data. It involves collecting,
classifying, summarizing, organizing, analyzing, and interpreting numerical
information.
More: In general, there are two diere
Other descriptive statistics
Review: Descriptive statistics, inferential statistics, sample/population
mean, sample/population variance, sample/population median, range
Population quantile: A pquantile of a population is a number x that solves
equations
The M/M/c queue
Again, X (t) the number of individuals in the queueing system can be
modeled as a birth & death process.
The transition state diagram for the X (t) is:
0
2
1
2
K
c1
3
(c1)
1
c
c
c
Clearly, the critical thing here in terms of whether or n
Estimator (Contd)
Review: What is estimator? What is estimates? What are the properties
we used to compare estimators?
Example: The sample mean x is consistent for . That means that if the
sample size is getting large, then X is getting very closed to in
Examples for CI of proportion
Example 1: Suppose we want to estimate the fraction of records in the
2000 IRS data base that have a taxable income over 35K.
Question: We want to get a 98% condence interval and wish to estimate
the quantity to within 0.01.
Example for MLE:
Review: What is MLE? How to nd it (5 steps)?
may be multiple: Rp with p > 1
Example: Let X1, . . . , Xn be i.i.d N (, 2), both and 2 are unknown.
x1, , xn are the data/sample value of X1, , Xn
What is the pdf of normal random variable?
Wednesday, 19th
1. Review:
Concept: Uncertainty/randomness, probability, mathematical model
(random experiment, outcome , sample space , event E).
Set theory: set, operations (belongs, subset, empty set, , , A, A \
B ), disjoint, exhaustive, De Morgan's l
Goodness of t (Contd)
Review: sample correlation as a measure of goodness of t
Second measure of goodness of t: Coecient of determination R2, it is
based on a comparison of variation accounted for by the line versus raw
variation of y .
Ideas: The quantit
Friday, 21st
1. Review:
Probability:
Independence:
Counting: Two principles, Permutation (ordered sample) with replacement, without replacement
Example: A survey question lists seven pizza toppings and ask you
to rank your favorite 3. How many possible an
Conditional Probability
Example 1: A box has 5 computer chips. Two are defective. Two chips are
selected from the box, one at a time.
1. Compute the probability that the second chip is defective.
Again common sense tells us that P ( a chip is defective) =
Testing Hypothesis (Contd)
Recall: In the previous lecture, 0.021 is called the pvalue for testing the
null hypothesis p = 0.5 against an alternative hypothesis p = 0.5.
Formal procedure: Let
H0 : p = 0.5 v.s. H1 : p = 0.5
then the test statistic is T =
Special discrete pmfs
Intuitive idea: In many theoretical and practical problems, several probability
mass functions occur often enough to be worth exploring.
Common feature: The sample space is always nite or countably many.
Example:
1. Bernoulli distrib
Two Basic Counting Principles
Summation Principle: If a complex action can be performed using one of
k alternative methods, m1, . . . , mk , and the methods can be performed
in n1, . . . , nk ways, respectively, then the complex action can be peformed
in
Statistics of R.V.s
Expectation The expected value of a function h(X ) is dened as
E [h(X )] :=
i h( x i )
pX ( x i ) .
The most important version of this is the case h(x) = x:
E [X ] =
i xi
pX (xi) =:
E [X ] is usually denoted by the symbol .
The expe
Random Variables
Intuitive idea: If the value of a numerical variable depends on the outcome
of an experiment, we call the variable a random variable.
Random Variable A function X : R is called a random variable.
Standard notation: Denote random variables