Math 4740
Problem Set 1
Spring 2014
Group work policy
Working in groups is strongly encouraged! You must list the group members at the top of your problem
set and write the solutions entirely in your own words. Examples:
Figuring out the solution togethe
Math 4740
Toms Optional Problems
Spring 2014
1. Suppose marksmen A and B have a competition to see who can hit a target rst, with A shooting
rst and alternating turns afterwards. A hits the target with probability pA and B hits the target
with probability
Math 4740
Problem Set 1 Solutions
Spring 2014
1. (a) Many ways of doing this problem. Here are a couple of solutions:
Method 1 By the Law of Total Expectation,
EN = E (N |First ip is tails)P (First ip is tails) + E (N |First ip is heads)P (First ip is hea
Math 4740
Exam 2, March 30, 2009
R. Durrett
1. (21 points) Let Y1 , Y2 , . . . be independent with P (Yi = 2) = p and P (Yi =
1/2) = 1 p. Let Xn = X0 Y1 Yn and think of Xn as the price of a
stock at time n. (a) What choice of p makes Xn a martingale? We w
Chapter 4
FINITE-STATE MARKOV
CHAINS
4.1
Introduction
The counting processes cfw_N (t); t > 0 described in Section 2.1.1 have the property that N (t)
changes at discrete instants of time, but is dened for all real t > 0. The Markov chains
to be discussed
Homework Solution 8 for APPM4/5560 Markov Processes
8.11 Two queues in series. Consider a two station queueing network in which arrivals only occur at the
rst server and do so at rate 2. If a customer nds server 1 free he enters the system; otherwise he g
Homework Solution 4 for APPM4/5560 Markov Processes
9.23Reecting random walk on the line. Consider the points 1, 2, 3, 4 to be marked on a straight line.
2
Let Xn be a Markov chain that moves to the right with probability 3 and to the left with probabilit
i
Essentials of Stochastic Processes
Rick Durrett
70
10 Sep
60
10 Jun
10 May
50
at expiry
40
30
20
10
0
500
520
540
560
580
600
620
640
660
680
Almost Final Version of the 2nd Edition, December, 2011
Copyright 2011, All rights reserved.
700
ii
Preface
Bet
Math 4740: Stochastic Processes
Problem Set 4
due Friday, February 22 at the beginning of class
In this problem set we are going to model English text as a Markov chain on 27 states
S = cfw_a, b, c, . . . , z, space (we ignore numbers and punctuation). Pe
Deeper Inside PageRank
Amy N. Langville and Carl D. Meyer October 20, 2004
Abstract This paper serves as a companion or extension to the "Inside PageRank" paper by Bianchini et al. [19]. It is a comprehensive survey of all issues associated with PageRank,
\documentclass[12pt,letter]cfw_article
% basic article document class
% use percent signs to make comments to yourself - they will not show up.
\usepackagecfw_amsmath
\usepackagecfw_amssymb
\usepackagecfw_bbm
% packages that allow mathematical formatting
Math 4740
Problem Set 2
Spring 2014
Group work policy
Working in groups is strongly encouraged! You must list the group members at the top of your problem
set and write the solutions entirely in your own words. Please consult the writing guidelines:
http:
ONE-DIMENSIONAL RANDOM WALKS
1. SIMPLE RANDOM WALK Denition 1. A random walk on the integers with step distribution F and initial state x is a sequence S n of random variables whose increments are independent, identically distributed random variables i wi
1
Introduction to Markov Chain Monte Carlo
Charles J. Geyer
1.1
History
Despite a few notable uses of simulation of random processes in the pre-computer era
(Hammersley and Handscomb, 1964, Section 1.2; Stigler, 2002, Chapter 7), practical
widespread use
Math 4740: Stochastic Processes
Practice problems for the March 1 prelim
Durrett 2nd edition has a recap of all the chapter topics starting on page 59. The
prelim will only cover nite Markov chains, which makes some of the theorems a
little simpler. For e
5
Poisson Processes
5.1
Exponential Distribution
Denition. A random variable T is said to have an exponential distribution with rate
> 0, if
P (T t) = 1 et for all t 0.
Let us denote it as T Exp(). Its density function will be
fT (t) =
et , if
0, if
t0
x
Math 4740: Stochastic Processes
Problem Set 1
Due Friday, February 1 by the beginning of class
Explain your answers. Please see the writing guidelines and the group work policy at
http:/www.math.cornell.edu/~levine/4740/guidelines.html
1. Let X, Y, Z be i
ISyE 6650
Probabilistic Models
Fall 2007
Homework 5 Solution 1. (Ross 5.3) Due to the memoryless property of the exponential distribution, the conditional distribution of X, given that X > 1, is the same as the unconditional distribution of 1 + X. Hence,
Math 4740: Stochastic Processes
Problem Set 6
due Friday, March 15 at the beginning of class
0 Please take the mid-semester survey at
https:/cornell.qualtrics.com/SE/?SID=SV_2tce3zdT6UQWRo1
If youve already taken it, thank you for your input!
.
1 Durrett
1.
Markov chains
What is a Markov chain? How to simulate one. The Markov property. Section 3. How matrix multiplication gets into the picture. Section 4. Statement of the Basic Limit Theorem about convergence to stationarity. A motivating example shows ho
Math 4740: Stochastic Processes
Problem Set 5
due Friday, March 8 at the beginning of class
P1 Consider a Markov chain with innite state space cfw_0, 1, 2, . . . and transition
matrix p(k, k +1) = qk and p(k, 0) = 1qk . Give an example of a sequence q0 ,
1
Discrete-time Markov Chains
Let cfw_Xk be a discrete-time stochastic process which takes on values in a countable set S , called the state space. cfw_Xk is called a Discrete-time Markov chain (or simply a Markov chain, when the discrete nature of the
Math 4740
Problem Set 2 Solutions
Spring 2014
1. Solution 1 We have
P (N
k) =
k1
P (N = ) =
k=1 =k
Solution 2 Consider the matrix
P (N = 1)
P (N = 2)
M = P (N = 3)
P (N = 4)
.
.
.
P (N = ) =
=1 k=1
0
P (N = 2)
P (N = 3)
P (N = 4)
.
.
.
0
0
P (N = 3)
P (N
Math 4740: Problem Set 4
March 6, 2014
Use L to denote the number of letters investigated by the author, use W to denote the number of
words (counted with multiplicity), use C to denote the number of characters, that is, letters and space,
then C = L + W
Math 4740
Sample Problem Set 2
Spring 2014
1. At a taxi stand, let taxis arrive at unit intervals of time (one at a time). If no one is waiting at
the stand, the taxi leaves immediately. Let Xt be the number of passengers who arrive at the stand
at time t
Math 4740: Stochastic Processes
Spring 2014
Practice problems for the March 7 prelim
The prelim will be Friday March 7 in class: 9:05-9:55 in 251 Malott.
No books, notes or electronic devices.
Here are some problems to help you study. You do not have to h
Math 4740
Spring 2013
Prelim
3/1/13
Time Limit: 50 Minutes
Name:
This exam contains 5 pages (including this cover page) and 3 problems. Check to see if any pages are
missing.
You may not use books, notes, calculator, phone, tablet, laptop or any other dev
Emails arrive in Emma X. Ponentials inbox according to a Poisson process at a rate
of 5 emails per hour. Each email independently is
spam with probability 50%
interesting but requires no reply with probability 40%
interesting and requires a reply with pro
A knight starts in one of the 4 central squares of an 8 8 chessboard. Once per second
it makes a legal move chosen uniformly at random. What is the expected number of
moves it takes to return to its starting square?
(A legal move for a knight means jumpin
Flip a fair coin until it comes up heads twice in a row or tails twice in a row.
We can model this process as a Markov chain with states , H, T, HH, T T . For
example, state H means that the most recent ip was heads but we have not yet
achieved two heads