# Register now to access 7 million high quality study materials (What's Course Hero?) Course Hero is the premier provider of high quality online educational resources. With millions of study documents, online tutors, digital flashcards and free courseware, Course Hero is helping students learn more efficiently and effectively. Whether you're interested in exploring new subjects or mastering key topics for your next exam, Course Hero has the tools you need to achieve your goals.

39 Pages

### rec7

Course: CPS 111, Fall 2009
School: Duke
Rating:

Word Count: 908

#### Document Preview

Markov Tutorial: Chains Steve Gu Feb 28, 2008 Outline Markov chain Applications Weather forecasting Enrollment assessment Sequence generation Rank the web page Life cycle analysis Summary History The origin of Markov chains is due to Markov, a Russian mathematician who first published in the Imperial Academy of Sciences in St. Petersburg in 1907, a paper studying the statistical behavior of the letters...

Register Now

#### Unformatted Document Excerpt

Coursehero >> North Carolina >> Duke >> CPS 111

Course Hero has millions of student submitted documents similar to the one
below including study guides, practice problems, reference materials, practice exams, textbook help and tutor support.

Course Hero has millions of student submitted documents similar to the one below including study guides, practice problems, reference materials, practice exams, textbook help and tutor support.
Markov Tutorial: Chains Steve Gu Feb 28, 2008 Outline Markov chain Applications Weather forecasting Enrollment assessment Sequence generation Rank the web page Life cycle analysis Summary History The origin of Markov chains is due to Markov, a Russian mathematician who first published in the Imperial Academy of Sciences in St. Petersburg in 1907, a paper studying the statistical behavior of the letters in Onegin, a well known poem of Pushkin. A Markov Chain P01 P00 "0" "1" P 11 P10 Transition Probability Table P11 P P 21 P31 P12 P22 P32 P13 P23 P33 Pij 0, i = 1,..., n; j = 1,..., n and Pij j =1 n 1 P11 = 0.7 P12 = 0.2 P13 = 0.1 P21 = 0. P22 = 0.6 P23 = 0.4 P31 = 0.3 P32 = 0.5 P33 = 0.2 Example 1: Weather Forecasting[1] Weather Forecasting Weather forecasting example: Suppose tomorrow's weather depends on today's weather only. We call it an Order-1 Markov Chain, as the transition function depends on the current state only. Given today is sunny, what is the probability that the coming days are sunny, rainy, cloudy, cloudy, sunny ? Obviously, the answer is : (0.5)(0.4)(0.3)(0.5) (0.2) = 0.0054 0.1 0.5 0.4 sunny 0.3 rainy 0.3 0.2 0.4 0.3 cloudy 0.5 Weather Forecasting Weather forecasting example: Given today is sunny, what is the probability that it will be rainy 4 days later? We only knows the start state, the final state and the input length = 4 There are a number of possible combinations of states in between. 0.1 0.5 0.4 sunny 0.3 rainy 0.3 0.2 0.4 0.3 cloudy 0.5 Weather Forecasting Weather forecasting example: Chapman-Kolmogorov Equation: Transition Matrix: s s 0.5 r 0.3 P r ( nm) ij ( ( Pikn ) Pkj m ) k 0 c 0.4 0.1 0.4 0.3 c 0.2 0.3 0.5 0.1 0.5 0.4 sunny 0.3 rainy 0.3 0.2 0.4 0.3 cloudy 0.5 Weather Forecasting Weather forecasting example: Two days: (00 x 01) + (01 x 11) + (02 x 21) 01 0.5 0.4 0.1 0.5 0.4 0.1 0.39 0.39 0.22 0.3 0.4 0.3 0.3 0.4 0.3 0.33 0.37 0.30 0.2 0.3 0.5 0.2 0.3 0.5 0.29 0.35 0.36 Four days: 0.5 0.4 0.1 0.5 0.4 0.1 0.3446 0.3734 0.2820 0.3 0.4 0.3 0.3 0.4 0.3 0.3378 0.3706 0.2916 0.2 0.3 0.5 0.2 0.3 0.5 0.3330 0.3686 0.2984 2 2 0.1 0.5 0.4 sunny 0.3 rainy 0.3 0.2 0.4 0.3 cloudy 0.5 Weather Forecasting Weather forecasting example: What is the probability that today is cloudy? There are infinite number of days before today. It is equivalent to ask the probability after infinite number of days. We do not care the "start state" as it brings little effect when there are infinite number of states. We call it the "Limiting probability" when the machine becomes steady. 0.1 0.5 0.4 sunny 0.3 rainy 0.3 0.2 0.4 0.3 cloudy 0.5 Weather Forecasting Weather forecasting example: Since the start state is "don't care", instead of forming a 2-D matrix, the limiting probability is express a a single row matrix : 0 , 1 , 2 Since the machine is steady, the limiting probability does not change even it goes one more step. 0.1 0.5 0.4 sunny 0.3 rainy 0.3 0.2 0.4 0.3 cloudy 0.5 Weather Forecasting Weather forecasting example: So the limiting probability can be computed by: 0 , 1 , 2 0.5 0.4 0.1 0.3 0.4 0.3 0.2 0.3 0.5 0 , 1 , 2 18 We have 0 , 1 , 2 ( 21 , 23 , 18 ) probability that today is cloudy = 62 62 62 62 0.1 0.5 0.4 sunny 0.3 rainy 0.3 0.2 0.4 0.3 cloudy 0.5 Example 2: Enrollment Assessment [1] Undergraduate Enrollment Model Stop Out Freshmen Sophomore Junior Senior Graduate State Transition Probabilities Fr Fr So .2 0 0 So .65 .25 0 Jr 0 .6 .3 Sr 0 0 S/O .14 .02 .13 Gr .01 .03 TP = Jr .55 .12 Sr 0 0 0.1 0 0.4 .4 .05 .55 0 S/O 0.1 0.1 0.3 Gr 0 0 0 0 0 1 Enrollment Assessment Stop Out Freshmen Sophomore Junior Senior Graduate TP = Fr So Jr Fr .2 0 0 0 0.1 0 So .65 .25 0 0 0.1 0 Jr 0 .6 .3 0 0.4 0 Sr 0 0 .55 .4 0.1 0 S/O .14 .13 .12 .05 0.3 0 Gr .01 .02 .03 .55 0 1 Given: Transition probability table & Initial enrollment estimation, we can estimate the number of students at each time point Sr S/O Gr Example 3: Sequence Generation[3] Sequence Generation Markov Chains as Models of Sequence Generation s s1s2 s3s4 ttacggt 0th-order N N P0 s pt 1 pt pa s3pc ps psi s ps2 p g p i 1st-order 1th-order i 1 i 1 N N P s pt 1 pts2 | spaps3 |ps2 | ps1 s p|si |1 si 1 s p | t 1 | t c a p 1 psi 1 2 2nd-order i 2 i 2 N N P22s ptt1s2pa s3 |s1sc|tas4p|s2 s3 p1s2s2 psis|i 2i s21i1 P s ps p | tt p2 p g | ac p s 1 p si | s i s i 3 3 i A Fifth Order Markov Chain Example 4: Rank the web page How to rank the importance of web pages? PageRank PageRank http://en.wikipedia.org/wiki/Image:PageRanks-Example.svg PageRank: Markov Chain For N pages, say p1,...,pN Write the Equation to compute PageRank as: where l(i,j) is define to be: PageRank: Markov Chain Written in Matrix Form: PR(p1,n + 1) PR(p1,n) l(1,N) l(1,1) l(1,2) PR(p 2 ,n + 1) PR(p 2 ,n) l(2,1) l(2,2) l(2,N) PR(pN-1,n) PR(pN-1,n + 1) l(N,1) l(N,N - 1) l(N,N) PR(p ,n) PR(p ,n + 1) N N Example 5: Life Cycle Analysis[4] How to model life cycles of Whales? http://www.specieslist.com/images/external/Humpback_Whale_underwater.jpg Life cycle analysis In real application, we need to specify or learn the transition probability table calf immature mature mom Post-mom dead Application: The North Atlantic right whale (Eubalaena glacialis) June 2006 Hal Caswell -- Markov Anniversary Meeting 30 Endangered, by any standard N < 300 individuals Minimal recovery since 1935 feeding Ship strikes Entanglement with fishing gear calving June 2006 Hal Caswell -- Markov Anniversary Meeting 31 Mortality and serious injury due ...

Find millions of documents on Course Hero - Study Guides, Lecture Notes, Reference Materials, Practice Exams and more. Course Hero has millions of course specific materials providing students with the best way to expand their education.

Below is a small sample set of documents:

Duke - CPS - 111
Recitation 3Steve Gu Jan 31 2008Outline Part I: Review of LDSDDS Linear, Deterministic, Stationary, Discrete, Dynamic System Example: Googles PageRank Part II: From Deterministic to Stochastic Randomness Some histogramsPart IReview of L
Duke - CPS - 111
CPS111- Homework 1Due on January 15, 2008 Questions may continue on the back. Please write clearly. What I cannot read, I will not grade. Typed homework is preferable. A good compromise is to type the words and write the math by hand.The questions
Duke - CPS - 111
Tutorial 5: Discrete Probability ISteve Gu Feb 15, 2008Reference:http:/www.cs.duke.edu/courses/fall07/ cps102/lecture11.pptLanguage of ProbabilityThe formal language of probability is a very important tool in describing and analyzing probabilit
Duke - CPS - 111
Introduction to MATLABJanuary 18, 2008 Steve GuReference:Eta Kappa Nu, UCLA Iota Gamma Chapter, Introduction to MATLAB,Part I: Basics MATLAB Environment Getting Help Variables Vectors, Matrices, and Linear Algebra Flow Control / LoopsDi
Duke - CPS - 111
CPS111- Homework 6Due on March 4, 2008 Questions may continue on the back. Please write clearly. What I cannot read, I will not grade. Typed homework is preferable. A good compromise is to type the words and write the math by hand. The rst three pro
SUNY Buffalo - EE - 494
Presenter, Menq J Pan University of New York at Buffalo ElectricalTable of Content What is L.I.D.A.R.? Motivation Varies Types/Techniques Other Applications Summary ReferenceUniversity of New York at Buffalo ElectricalWhat is it? LIDAR a
SUNY Buffalo - EE - 494
a( 6 10 R 6 &amp; &amp; v#yx 'wBvB 41 ( \$ h 6 \$ #384\$ 7 I u#' t%s)fB10 R r \$ R U h Bq4p\$ &quot; '3f8p\$ 7 di \$ R a( \$ b`AVh g '3f e dc R a( b`8YX W V 'G!T@S &quot; PQ U I \$ PH4G&amp; FD4 E 2\$ #DCBA 9 # #384\$ 7 ( 6 ( @
SUNY Buffalo - EE - 494
LASERS IN MEDICINECorrective Laser Eye SurgeryJeffrey Griffith 3/24/99SUMMARYOverview on How the Eye Works Refractive Eye Surgery PRK and LASIK Excimer Laser Glaucoma Lasers Used to Treat Glaucoma ConclusionHOW THE EYE WORKS
SUNY Buffalo - EE - 494
Duke - AB - 216
STATISTICS ON MANIFOLDSFRECHET MEANS AND THEIR ESTIMATIONProject Submitted by: Abhishek Bhattacharya Advisor: Dr.Rabi Bhattacharya1Overview Frechet mean on Metric Spaces Extrinsic Mean Examples on some common Manifolds Intrinsic Mean Exam
SUNY Buffalo - PHY - 410
PHY 410-505 Computational Physics IChapter 1: A First Numerical ProblemLecture 2Wednesday August 27, 20081LECTURE OUTLINELECTURE OUTLINELecture Outline Programming Concepts Useful References . . . . Comments . . . . . . . Keywords . . .
SUNY Buffalo - CSE - 562
Outline of the 2nd phase of the projectQuery optimization rules1. Cartesian Products converted to Theta-Joins, as many as possible 2. Selections pushed down, as many as possible and as far as possible 3. Theta-Joins pushed under Cartesian Products
SUNY Buffalo - BMS - 503
REPORTSchor oligonucleotide (Operon, Alameda, CA) at 37C for 2 hours, followed by heat inactivation at 70C for 15 min, then digestion with Eco RI restriction endonuclease, and purication with QIAquick PCR kit (Qiagen, Valencia, CA). The ligated DNA
Duke - STA - 216
Ordered Categorical Data Analysis (continued) Often, an ordinal outcome yi {1, . . . , C} indexes when an event occurs. For example, yi may be age at puberty or number of weeks of gestation Such data are are often referred to as discrete event t
Duke - STA - 103
Duke - STA - 215
Duke - STA - 216
Midterm Review STA216: Generalized Linear ModelsPrimary Topics Covered So Far: 1. GLM Basics (a) Definition of exponential family, f (yi ; i , ) = exp yi i - b(i ) + c(yi , ) , a()where i , are location and scale parameters, respectively. Mean an
Duke - STA - 101
7.0 Lesson Plan Discuss Quizzes/Answer Questions Regression Residuals1 Perils of Overthink Variability: SD = , SD 2 = 2 ; Variation in data decomposes into several terms (bias, variance, tracking) Population vs. Sample2Conditional Popul
Duke - STA - 101
11.0 Contingency Tables Discuss Quizzes/Answer Questions More Exercises Contingency Tables111.1 More ExercisesRecall the simple version of Bayes Rule:. Let A1 , . . . , An be mutually exclusive and suppose that P [A1 or A2 or . . . or An ] =
Duke - STA - 205
Sta 205 : Homework 2Due : January 28, 2009I. - algebras and Probability Assignments. (A) Let {A, B, C} F be three events in a probability space (, F , P). Recall that a partition is a nite or countable collection of disjoint events j F with j =
Duke - STA - 114
Midterm Examination # 1Mth 136 = Sta 114 Thursday, February 26, 2009 2:50 4:05 pm Version b This is a closed book exam put your books on the oor. You may use a calculator and one page of your own notes. Do not share calculators or notes. Show y
Duke - STA - 205
Final ExaminationSTA 205: Probability and Measure Theory Due by Thursday, 2007 May 3, 7:00 pmThis is an open-book take-home examination. You may work on it during any consecutive 48-hour period you like; please record your starting and ending time
Duke - STA - 122
STA122 Lab Session # 4: Flow Control in RCourse Instructor: Prof. Merlise Clyde Teaching Assistant: Debdeep Pati (dp55@stat.duke.edu) February 3, 200911.1Grouping, loops and conditional executionGrouped expressionsR is an expression language
Duke - STA - 205
Sta 205 : Home Work #10Due : April 08, 2009 I. Convergence Of Series, Strong Law (A) Let Sn := n 1/i. Explain carefully why Sn does or does not converge to a i=1 n i nite limit. Now let Sn := i=1 (1 )/i. Does the sequence of real numbers Sn converge
Duke - STA - 114
STA114/MTH136: Statistics Homework #3 SolutionsJanuary 25, 200916.3, p 3443. Let X denote the number of defective items out of the randomly selected 100 items. Note that X | Bin(100, ) where denotes the proportion of defective items in the l
Duke - STA - 205
Markov, Chebychev and Hoeding InequalitiesRobert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USAFor each constant c &gt; 0, any non-negative integrable random variable Y satises the inequalities 0 Y c 1{Y c} P[Y c] E
Duke - STA - 205
Sta 205 : Home Work #7Due : March 16, 2009 I. True or False? Answer whether each of the following statements is true or false. If your answer is true, answer why it is true. If it is false, show why- perhaps by giving a simple counter example. (A) I
Duke - STA - 205
STA 205Week 2 Construction &amp; Extension of MeasuresFor any finite set = {1 , ., n }, the &quot;power set&quot; P() has |P| = 2n elements; it can also be identified with the set of all possible functions a : {0, 1} by the relation A = { : a() = 1}. Set th
Duke - STA - 205
Sta 205 : Home Work #9Due : April 1, 2009 I. Convergence Concepts: a.s. and i.p. (A) Let {Xn } be a monotonically increasing sequence of RVs such that Xn X in probability (i.p). Show that Xn X almost surely (a.s). (B) Let {Xn } be any sequence of
Duke - STA - 205
Sta 205 : Home Work #6Due : 2009 Mar 04 1. Fubini and Tonelli. (a) Let X be a positive random variable (i.e, X 0 a.s) on (, F , P). Show thatE(X) =0P(X &gt; t) dt(note X need not have an absolutely-continuous distribution). Also verify that fo
Duke - STA - 205
Sta 205 : Home Work #4Due : February 11, 2009 1. Expectation. (a) Consider the triangle with vertices (1, 0), (1, 0), (0, 1) and suppose (X1 , X2 ) is a random vector uniformly distributed with in this triangle. Compute E(X1 + X2 ). (b) Let (0, 1],
Duke - STA - 205
Sta 205 : Homework #11Due : April 15, 2009 I. Convergence In Distribution (A) For events {An } and A in some probability space (, F , P), dene Bernoulli random variables by Xn 1An and X 1A . As n , i. Under what conditions on {An } and A will Xn
Duke - STA - 205
Introduction to MartingalesRobert L. Wolpert Institute of Statistics and Decision Sciences Duke University, Durham, NC, USAWe've already encountered and used martingales in this course to help study the hitting-times of Markov processes. Informall
Duke - STA - 244
STA2441/29/2009Homework 4Due 2/24/2009For the normal linear model Y N(X, In /) (with X n p of rank p), consider the conjugate Normal-Gamma prior with | N( 0 , ()1 ) Gamma(0 /2, SS0 /2) (2) (3) (1)where is a symmetric positive denite
Duke - STA - 244
STA2441/22/2009Homework 2Due 1/29/2009Work the following problems from Christensen 1. Exercise A.2 2. Exercise A.3 3. Exercise A.4 4. Exercise A.5 5. Exercise A.6 6. Exercise A.7 7. Exercise A.8 8. Exercise A.9 9. Exercise A.10 10. Exercise B.
Duke - STA - 114
STA114/MTH136: Statistics Homework #2 SolutionsJanuary 25, 200916.2, p 3341. Let Xi denote the lifetime of the ith bulb, i = 1, 2, , 6. We have X1 , X2 , X6 conditionally i.i.d given with the common conditional distribution of Xi given
Duke - STA - 114
STA114/MTH136: Statistics Homework #4 SolutionsFebruary 16, 200916.5, p 362fn (x|) = n e The log likelihood is given by l() = nlog i=1n i=17. The likelihood function is given byxinxiDifferentiating l() w.r.t. , we getl n = nXn
Duke - STA - 205
Sta 205 : Homework 1Due : January 23, 2008I. Fields and - fields. (A) For a three-point outcome set = {a, b, c} and C := {a} , enumerate the class of all -fields F on that contain C, i.e., satisfy C F . Also find (C). (B) For each integer n
Duke - STA - 205
Sta 205 : Home Work #9Due : April 2, 2008 I. Convergence Concepts: a.s. and i.p. (A) Let {Xn } be a monotonically increasing sequence of RVs such that Xn X in probability (i.p). Show that Xn X almost surely (a.s). (B) Let {Xn } be any sequence of
Duke - STA - 205
Sta 205 : Home Work #10Due : April 09, 2008 I. Convergence Of Series, Strong Law (A) Let Sn := n 1/i. Explain carefully why Sn does or does not converge to a i=1 n i nite limit. Now let Sn := i=1 (1 )/i. Does the sequence of real numbers Sn converge
Duke - STA - 293
Duke - STA - 205
Range: 34.17% 92.5%, 14 Students Median = 77.5, Quartiles = [65, 83.25] Mean = 77.36, Std Dev = 20.98Students012340204060 Stat 205 Midterm, Spring 200880100120
Duke - STA - 205
3.0Range: 64.17% 99.17%, 14 Students Median = 104.5, Quartiles = [100.25, 113.75] Mean = 103.43, Std Dev = 13.13Students0.00.51.01.52.02.50204060 STA 205 Final Exam, Spring 200880100120
Duke - STA - 205
Range: 74.03% 99.68%, 14 Students Median = 91.19, Quartiles = [86.6, 94.94] Mean = 89.7, Std Dev = 7.64 B B B+ A AStudents01234607080 STA 205 Course Averages: Spring 200890100
Duke - STA - 205
Sta 205 : Homework #5Due : February 20, 2008 1. Indepedence. (a) Let {Bi }, be independent events. For N N show thatN NPi=1Bi=1i=1[1 P(Bi )](b) If {An , n N} is a sequence of events such that P(An Am ) = P(An )P(Am ) n, m N, n = m,
Duke - STA - 205
Probability and MeasureRobert L. Wolpert Institute of Statistics and Decision Sciences Duke University, Durham, NC, USAConvergence of Random Variables11.1Convergence ConceptsConvergence of Real NumbersA sequence of real numbers an converge
Duke - STA - 293
Spring 2008STA 293: Stoch Proc for Nonpar BayesR. WolpertBROWNIAN MOTION AND RELATED PROCESSES Karlin &amp; Taylor, A First Course in Stochastic Processes, ch 7, 15 Brownian Motion: Denitions Brownian motion can be dened and constructed in many way
Duke - STA - 293
Spring 2008STA 293: Stochastic Processes &amp; Bayesian NonparametricsR. L. WolpertPARAMETRIC AND NONPARAMETRIC STATISTICS Parametric Models There is little to learn from observing some random quantities X1 , , Xn all drawn independently from a
Duke - CPS - 196
ConsensusP1 P1 d1Consensus algorithmConsensus in Distributed SystemsP2 Jeff Chase Duke University v2v1Unreliable multicastP3 Step 1 Propose. v3P2 d2 Step 2 Decide.P3 d3Generalizes to N nodes/processes.Fischer-Lynch-Patterson (1985)
Duke - CPS - 111
CPS111- Homework 2Due on January 27, 2009 Questions may continue on the back. Please write clearly. What I cannot read, I will not grade. Typed homework is preferable. A good compromise is to type the words and write the math by hand. This homework
Duke - CPS - 111
Computational ModelingCPS 111 Lecture Notes Carlo Tomasi Duke University Spring 2009Warning. These notes are being written as the course progresses. As errors are discovered, old notes will be corrected and the new version will be posted online.
Duke - CPS - 111
CPS111- Homework 1Due on January 15, 2009 Questions may continue on the back. Please write clearly. What I cannot read, I will not grade. Typed homework is preferable. A good compromise is to type the words and write the math by hand.The questions
Duke - CPS - 104
CPS 104 Midterm ExamOctober 20, 1998 2:15pm to 3:15pm Open book examAnswer all questions, state all your assumptions, clearly mark your nal answer. Be sure you have all seven (7) pages of the exam. Write your name on each page of the exam. Read all
Duke - CPS - 102
CPS102- Homework 5Due on December 8, 2005 Questions may continue on the back. Please write clearly. What I cannot read, I will not grade. Typed homework is preferable. A good compromise is to type the words and write the math by hand. The Duke Commu
Duke - CPS - 196
Automated mechanism designVincent Conitzer conitzer@cs.duke.eduGeneral vs. specific mechanisms Mechanisms such as Clarke (VCG) mechanism are very general. . but will instantiate to something specific in any specific setting This is what we car
Duke - CPS - 296
Efficient Algorithms for Geometric OptimizationPANKAJ K. AGARWALDuke UniversityAND MICHA SHARIRTel Aviv UniversityWe review the recent progress in the design of efficient algorithms for various problems in geometric optimization. We present se
Duke - CPS - 296
Mechanism Design for Online Real-Time SchedulingRyan PorterComputer Science Department Stanford University Stanford, CA 94305rwporter@stanford.eduABSTRACTFor the problem of online real-time scheduling of jobs on a single processor, previous wo
Duke - CPS - 296
Mechanisms for Partial Information Elicitation: The Truth, but Not the Whole TruthAviv Zohar and Jeffrey S. RosenscheinSchool of Engineering and Computer Science The Hebrew University of Jerusalem Jerusalem, Israel {avivz, jeff}@cs.huji.ac.il Abstr
Duke - CPS - 296
The Neural Basis of Economic Decision-Making in the Ultimatum Game Alan G. Sanfey, et al. Science 300, 1755 (2003); DOI: 10.1126/science.1082976 The following resources related to this article are available online at www.sciencemag.org (this informat
Duke - CPS - 271
Chapter GGame TheoryG.1 Two-Person Zero Sum Games; Reduction by Dominance G.2 Strictly Determined Games G.3 Mixing Strategies G.4 Solving Games with the Simplex Method Youre the ExpertHarvesting ForestsForest Lumber Inc. has a large plantation o
Duke - CPS - 001
Adversaries Consider the process of reasoning when an adversary is trying to defeat our efforts In game playing situations one searches down the tree of alternative moves while accounting for the opponents actions Problem is more difficult because