Unformatted text preview: CSC5420 CSC5420
Computer System Performance Evaluation John C.S. Lui
http://www.cse.cuhk.edu.hk/~cslui/csc5420
1 Copyright John C.S. Lui CSC5420 Introduction
Material from lectures (ref. books on web page) Grading homework for students' benefit (will include use of softwaretools on web page) 10% homework 40% projects 50% final exam 2 Copyright John C.S. Lui CSC5420 Course Material
Review of Probability, R.V., Transforms Intro. to Stoch. process (m.c.'s) Baby queuery theory m/m/1... Intermediate queuery theory m/g/1... Markovian model is a special structure Appr. Tech. Stoch. Couple Matrix geometric structure Sample Path Analysis Transient Analysis Reversibility Queuery Networks  product form Simulation Measurements
Copyright John C.S. Lui Real World Model Solution 3 CSC5420 Project: List to choose from, FCFS MS Comp: Final 4 Copyright John C.S. Lui CSC5420 Combinatorics
Permutations kpermutation of a set of n elements Combinations kcombination of a set of n elements kpermutation / k! k! is the number of possible ways to permute that combination 5 Copyright John C.S. Lui CSC5420 Combinatorics (Cont...)
Binomial Coefficients: Binomial Expansion: 6 Copyright John C.S. Lui CSC5420 Probability
Sample Space (S), collection of objects, where each object is a sample point. A family of events, ={A,B,C,...} where an event is a set of sample points. A probability measure P is an assignment (mapping) of events defined on S into real numbers (which has properties or axioms). P[A] = Probability of event A 7 Copyright John C.S. Lui CSC5420 Probability
S = sample space = set whose elements are elementary events (possible outcome of an experiment) The elementary events are points in a sample space (1 or more dimensions), and they are mutually exclusive Ex: flipping a coin, a elementary events (sample points) H, T Event: subset of sample points Ex: toss dice
event 1 2 4 3 6 5 sample point 8 Copyright John C.S. Lui CSC5420 Probability (Cont...)
Aximoms of Probability: A probability distribution Pr{} on a sample space S is a mapping from events of S to real numbers s.t. the following proability axioms hold: 1) Pr{A} 0 for any event A (where Pr{A} probability of event A) 2) Pr{S} = 1 3) Pr{AB} = Pr{A} + Pr{B} for events A and B that are mutually exclusive 9 Copyright John C.S. Lui c) 7 6 &5 98 a) d)
' &$ % b) 0)( Copyright John C.S. Lui
C DB 2 31 4 EF A H IG P W Q ! C DB EF C DB EF R SP R SP C B H E C B H E " # " ! @ Things that follow: Probability (Cont...) P T UP C DB EF H IV P CSC5420 10 CSC5420 Discrete Probability Distribution
Probability distribution is discrete if it is defined over a finite or countably infinite sample space if x's are mutually exclusive events in A If S is finite and event elementary event in S has probability then we have the uniform distribution on S (or we pick an element of S at random) 11 Copyright John C.S. Lui CSC5420 Discrete Probability Distribution (Cont...)
Ex: flipping a fair coin, Pr{H} = Pr{T} = 0.5 flip coin n times A = {exactly k heads and exactly nk tails} 1% 342 6 " #! $% %& ( ' )0 ($ $ 5 12 Copyright John C.S. Lui CSC5420 Continuous Uniform Probability Distribution
Defined over closed interval [a,b] of reals where a < b (all subsets here, not events) want each point in [a,b] to be equally likely but, infinite number of points, if give each one finite probability, will not be able to satisfy axioms 2 and 3 associate probability with some of the subsets for any closed interval [c,d], a c d b, continuous uniform probability distribution: ! # $% '% 4 ! $% % # ( 0 1) 23 " 4 & & 4 " " " & 5 ' 13 Copyright John C.S. Lui CSC5420 Conditional Probabilities and Independence Def'n: normalizing (sum to 1) Ex:
1 2 4 6 ! constrained sample space, so we scale up
0 ( 1 2 3 5 " # $% & ' ' ) 3 0 1 0 14 Copyright John C.S. Lui CSC5420 Conditional Probabilities and Independence (Cont...)
A and B are statistically independent if and only if: " ! ' # ! 10 ) 4 % &$ $ 10 ) 10 ) 4 23 5 3 4 30 ) 4 5 If A1, A2, ..., An are statistically independent ( Also, if A and B are statistically independent, then 15 Copyright John C.S. Lui CSC5420 Theorem of Total Probability more useful form where {Ai} is a set of mutually exclusive exhaustive events
if occurs, occurs with exactly 1 mutually exclusive exhaustive event (Ai) ! ' using conditional probability:
) ' # ' # $) ' " & " & " & " & ( % % ( % " % ' #$ #$ #$ 16 Copyright John C.S. Lui CSC5420 Theorem of Total Probability (Cont...)
Ex: reliability R1 R2 R3 R3 R3 R4 R5 R5 where {Ri} is the reliability of component i Importance of Theorem of Total Probability is to break a complex problem into many simpler problems 17 Copyright John C.S. Lui CSC5420 Bayes' Theorem
Look at problem from another perspective Assume we know event B has occurred, but we want to find which mutually exclusive event has occurred " $ " $ " # # ! ! % " $ " # ! more general form above
Copyright John C.S. Lui # $ # $ 18 CSC5420 Bayes' Theorem (Cont...)
More general forms: Ai, 1 i n, are mutually exclusive, exhaustive events Theorem of total probability " # &# " ! $ (&'% ) & Bayes' Theorem 19 Copyright John C.S. Lui CSC5420 Example
Ex: gambling, DH honest dealer, DC cheating dealer L you lose play honest dealer lose with prob = 1/2 play cheating dealer lose with prob = p (of p > 1/2 against you, of p < 1/2 for you) if p=1, prob. that cheating dealer if lost 1 game = 2/3 20 Copyright John C.S. Lui CSC5420 Random Variables (R.V.)
We have the probability system (S, , P) R.V. is a variable whose value depends upon the outcome of the random experiment The outcome of a random experiment is w S We associate a real number X(w) with W Thus our r.v. X(W) is nothing more than a function defined on the sample space S i.e., a function from a finite or countably infinite sample space S to real numbers 21 Copyright John C.S. Lui CSC5420 Example
Ex: playing a game of black jack in Las Vegas
Associate a probability with it Sample Space: all possible pairs of scores obtained by player & dealer (single game) W
3/8 D
1/4 L
3/8 5 (lose $5) 0 ($0) 5 (win $5) real line Notation: R.V. X on a sample S X:SR X(w)  our winnings on a single game of black jack
Copyright John C.S. Lui X(w) = +5 w W 0 wD 5 w L
22 CSC5420 Discrete Random Variables
Discrete Random Variables: (X) a function from a finite (or countably infinite) sample sapce S to real numbers interested in functions of events each outcome is a combination of events, so we can assign probability to it P[Xx]: probability distribution function
1 x 23 Copyright John C.S. Lui u j i h d kl xy w v }x { z ~ p q 4 z ~ ih F 4 )C } m u o t i g % S G 9 7 s z rq n m e gf h i ht t E vs % " 0 Y r v s s ih p a q R ' & () @ A6B B DF EC w v V z ~ a z fg % " % H0 P QI 1 23 4 6 5 75 9 78 " 0 p m v { } n o d ec i s () uy ' & () z m ' & % v xew Y ab` 1 xy kl $ t us " #! Copyright John C.S. Lui
w v j i T W V X#U Discrete Random Variable (Cont...) CSC5420 24 ! # $" `Y x { a lki j { nm t x o F G 8 W gf D HG a g d C pq s k nm h I e u h p n q d x e u Y s r p nm ot e E U S T @ U S T v V x E U x yw v @ S T t t us C G 7 R ot t p Y e ) 1 v c r G C PQI D &6 i G 5 4 C 2 3' # 1 $% h ` a h p n r kq w z 0 Y gf A E ) ~ k{ } edb c D C '( h &% A [email protected] v z yw x d f ge X 9 Copyright John C.S. Lui
W 8 Expectation, Variance, and Standard Deviation
FX(x) 1 x CSC5420 25 CSC5420 Probability Distribution Function (PDF) or Cumulative Distribution Function 6 5 8 7 9 CB A ed Wf g @ c b h d i f 8D ! ) 0( 231 ) 4& 0( "# & '%$ E Properties: 1) 2) 3) QI P H G F T S VWU h pq 4) 5) e vu c b y X `Y Ex: PDF for the Las Vegas Game
1
3/8 5/8
1/4 3/8
26 a 5
Copyright John C.S. Lui y 0 5 R xw h re r ts CSC5420 Probability Density Function (pdf) or Probability Mass Function (pmf) Ex: pdf for the blackjack game
3/8 1/4 3/8 5 0 5 Different ways to view pdf: 1) 2) 3) % &$ ( )' 0 6 75 9 8 [email protected] 9 43 3 1 2 B T V WU Y X a` Y R S G I H PQ C 4) D F E ! " # " 27 Copyright John C.S. Lui CSC5420 Special Discrete Distribution
The Bernoulli pmf pmf
q p PDF 1 q 0 1 0 !
1 " 28 Copyright John C.S. Lui CSC5420 Geometric Distribution
Bernoulli trial: experiment with only 2 possible outcomes success with probability p failure with probability q = 1p Bernoulli trials, a sequence of independent trials each with probability p r.v. X = number of trials needed to obtain success X { 1, 2, ... } ! " & ! S T U6 4 geometric distribution
6 F D EC 5 786 R EQ 10 )0 43 2 9 D 4 H IG P R 4 W XG P @ V e fd AB Y a b` c g ` g on average, 1/p trials before obtain success
29 Copyright John C.S. Lui $ % ' ( ! # CSC5420 Binomial Distribution r.v. X = number of successes in n trials, X { 0, 1, ... } ! " $# % binomial distribution
5 786 ) & 3 412 12 0 0 A 2 B [email protected] '( 30 Copyright John C.S. Lui CSC5420 Multiple R.V.
Can, of course, define many r.v. on same sample space Let X & Y be 2 r.v. on some probability system (S, , P) Natural extension of PDF: joint PDF Joint probability density function:
21) 0 ) $%#! " $%#! " &' ( ") & &' H I9 H Marginal density function (for one of the variables):
7 48 53 6 9 A D CB A 4E 53 FG86 7 @ (given by integrating over all possible values of the 2nd variable)
31 Copyright John C.S. Lui CSC5420 Multiple R.V. (Cont...)
Notion of independence between r.v.'s X & Y are independent iff: (same for more than 2 variables) ) Can also define one random variable in terms of another, i.e., %$ $0 34 54 16 6 '(& ! # ' (& # 21 7) " " Of course, can be a function of many r.v. (could be complex computation) 32 Copyright John C.S. Lui CSC5420 Example
Ex: Let Y = X1+X2 (i.e., sum of 2 r.v.) where X1 and X2 are independent % # &)( ' ! " 21 30 1 65" 4 9# 7 684 654 48
x1 $ &% &% x1+x2y x2 y ) 9 33 Copyright John C.S. Lui CSC5420 Example (Cont...) due to independence: ! " # ! # 2 ) 76 8(& C6 8(& A [email protected] A [email protected] 54 D2 [email protected] Q # ! # 0 ) W US E V 9 ' (& 0 1) 2 3 4 convolution of density functions of X1 and X2 F GE I PH Q I PH Q R TUSGE I PH (same for any n sum of independent r.v.) $ # % 34 Copyright John C.S. Lui CSC5420 Expectation
The expectation of a real r.v. X(w) is denoted by E[X] ! ' # %$ 2 4 5 "! 7 [email protected] A ) 1 7 BC 01 3 3 65 7 & # ( I P Stieltjes Integral W XV G Y T US T US H R P R For X, a nonnegative r.v.
e g t Xs b r Uq u ev cd ci t f h p t w x aI W V `Y W D EF G` EQ W 35 Copyright John C.S. Lui CSC5420 Fundementation Theorem of Expectation
Let y=g(x)
could be complex to compute fY(y) & ! " $% # Fundementation Theorem of Expectation:
( "# )% ! '& ( "# 0% ( 8 98
(generalize to any number of variables)
36 Expectation of sum of 2 r.v.
8 98 8 A 8 98 E 9 8 98 8 F A "@ 8 A E A "@ PD A FG CB4 A H "@ ED 4 B FG PD 98 B 8 H A I"@ A P B E B G PD B @ A 4 PD BP B 98 8 32 1 4 7 56 7 32 9 8 6 True whether or not X & Y are independent
Copyright John C.S. Lui 7 41 56 1 2 FG HI"@ A B PD A BP 7 B E m t o pn q o r s uvt d Q h f Q ia R e j r t w kh g li h i u g f ca e H I P ` B t  } q o w~ { fe h 4g u c da b FG 4 X `Y ED C u v x yw ! W B 9 Copyright John C.S. Lui Interested in power of r.v.'s x yw z If X & Y are independent, then Product of R.V. p q h sr h S HT R VU "# @4 $ !1 A2 & % % 1 "2 & % # % ' $ ) 0( 1 @ 4 '42 3 !1 5 0( 7 2# 31 62 A '6 3 1 "2 $ # ! 78#
binomial theorem "#
CSC5420 37 follows from the fundemental theorem of expectation l n om p s tr f e d g i g h j k y v y c qr irg h a V Y X V Y a b ) 0 $ w y t u t ih p A S nq ih [email protected] r s S T ) HG I x y HG [email protected] QR $ % $ '& ( 1 20 V WU X V Y ` $ % '& ( # d e g8f B EFC D !" c A Copyright John C.S. Lui '& [email protected]( 3 4 53 64 % !" 7 8# take expectation of both sides Product of R.V. (Cont...) coefficient of variation sums of expectation, expectation of sums, expectation of constant CSC5420 38 CSC5420 Transforms
Transforms, characteristic function, generating function... Laplace, z, Fourior, ... When introduce into solution method, simplify calculations Appear naturally, why? Linear Timeinvariant Systems systems transformations, mapping, inputoutput relationship between 2 functions f System g black box assume f=f(t), f(t) g(t)
39 Copyright John C.S. Lui CSC5420 Linear and Timeinvariant
Linear if when f1(t) g1(t) and f2(t) g2(t) then also af1(t)+bf2(t) ag1(t)+bg2(t) Timeinvariant if when f(t) g(t) then also f(t+) g(t+) If both holds, we have a linear timeinvariant system
we focus on these 40 Copyright John C.S. Lui CSC5420 Transforms
Decompose function of time into sums (integrals) of complex exponentials complex exponentials form building blocks of transforms Question: which functions of time can pass through linear timeinvariant systems without change? i.e., f(t) g(t) = Hf(t), where H is some scalar multiple if can find these, then can find eigenfunctions or characteristic functions, or invariants of our system where s is a complex variable form the set of eigenfunctions for all linear timeinvariant systems 41 Copyright John C.S. Lui CSC5420 Characteristic Functions
Derivation: linearity $# " ! % & '( 0 1) 32 654 DE 7 CB A timeinvariant where and hence e s are constant 32 uv
independent of t but can be function of s [email protected] IH R SQP T WVU X unique solution gf d ec w i q rp y uv x w st h T WVU ba` YR X 0G 8 654 F 7 42 Copyright John C.S. Lui CSC5420 Characteristic Functions (Cont...)
Overall output found by summing (integrating) these individual components of the output decompose input into sums of exponentials, computing response to each as above, and then reconstituting the output from sums of exponentials is referred to as transform method 43 Copyright John C.S. Lui q Ip s Tr Y a b` # ( ) 10 2 '& @ A9 5 CB ) 3 ED 7 @ s CB (8 10 v 3 4 8 5 F H IG 3 60 2 QP Q G SU TR VS WX Q H R QP R 2 60 2 4 % ! " 3 % "$ c d tu e v w g h x e y g d i f f Copyright John C.S. Lui eigenfunctions: b Focus on discrete time first also a complex variable
44 Transforms
CSC5420 CSC5420 Transforms (Cont...) ! % &$ ( 01) 0 B7 0 @ A9 8 # ' 32 56 7 2 7 D 8 9 " H expresses how much we get out of unit input system (or transfer) function Kronecker delta function or unit function:
P QI R F GE H H T S H R S S Unit response (when apply un to system), hn
V GU W V X f Y a b` dc e dc a B26 CB 4 45 Copyright John C.S. Lui CSC5420 Transforms (Cont...) linearity multiple by unity on both sides # $" V T R USQ I PH ic d b c d H W X ! Consider set of inputs {fn }
( 0) 1 2 2 4 0) ' 3 1 (i) 78 69 B 8 69 linearity @ 6 b e fc d 6 @ a hUb Y` a b ` g` b c c d sum over all integer value of m only 1 nonzero term, when m=n, and it equals 1 c ` b D FGE 5 A 5 DC 46 Copyright John C.S. Lui % & CSC5420 Transforms (Cont...) plus change of variables $ &'% ( # 0 0& ) 1 0 related system function H(z) to unit response both H(z) and unit response describe how the system operates, so they are related itself a transform, a zTransform so transforms arise naturally ! " " 47 Copyright John C.S. Lui CSC5420 zTransform
Let fn be a function which takes on nonzero values only for nonnegative integers, n=0, 1, 2, ... (fn=0 for n < 0) Compress sequence into a single function such that can expand later Place a tag on each term n i.e., tag each fn with z (n unique each tage is unique) Define ztransform (or generating function, or geometric transform) ! The ztransform will exist as long as terms don't grow any faster than geometrically, i.e., as long as a exists, s.t. 48 Copyright John C.S. Lui CSC5420 zTransform (Cont...) Given a sequence fn, its ztransform is unique If sum over all fn is finite, then F(z) is analytic on z 1. Then: Notation: has a unique derivative at that point function is analytic at that point 49 Copyright John C.S. Lui CSC5420 Examples of zTransforms
Ex 1: Recall the unit function A C DB 7 86 Q DT P FU GA Q RP A F S QP A W XP 9 U HGA F 5 [email protected] E P W aHF
50 I V I I Q Q ) Ex 2: Shift the unit function to the right 1 20 3 4 $" %#! Copyright John C.S. Lui & '( & Y `Q W Copyright John C.S. Lui V u It v fe g t i Ih pqe Y b e u
region of analycity Y `W X bca d P F R Q R S RUU T U D EC F H IG D Ex 4: geometric series Ex 3: unit step function y x u h fe g Y ba e x r s i Y iX w 1 20 3 4 76 5 38 @ 9 @ 4 @BB A B d Examples of zTransforms (Cont...) h &
51 ! !" $# !% ' " () %" CSC5420 Copyright John C.S. Lui @ A9 DCB @ )' 0( 12 3 54 6 a b` edc E HIG C F GP Q @R G a i pg h rsq i pt h q f Properties of zTransforms 5 67 4 8 F " #! G R A9 5 @ 3 67 3 $ PG @R 5 4 % 7 S TG S & % VW % " & P X HIG C F Q " % YW GP PX % G HU I
52 CSC5420 Convolution property We have 2 function, fn and gn with fn F(z) and gn G(z) UP 9 U F CSC5420 Other Properties of zTransforms linearity scale change in the transform and time domains #$ ! " % '(& 0) 2 1 3 5640 5 58 70 F GE HI P R TS Q RXUV W 88 Y `E a D @ BA 1 9C 9 tsq r u wv w xy advance or delay by unit of time results in divide or multiply in transform domain Rb RU W V T b F GE Hb cU d eV P R hgIf p i If i fb iI E 53 Copyright John C.S. Lui CSC5420 Inverse Transforms
Given F(z), find sequence fn Power series method, e.g., not as useful if want many terms ratio of numerators and denominators Inspection method express F(z) in terms which have recognizable transform pairs partialfraction expansion 54 Copyright John C.S. Lui also
y B C A E D F B # x h k lj e m on q d p s "r ) 55 ` a c db g h ef f w ' ( ' i uIs t v r4p q Copyright John C.S. Lui ut p e Wd f gd E D F H IB A P QG ) P P e.g., each term is either 1) a simple pole 2) a multiple pole
G 3 42 E D 5 i 0 1 ( ' !" # $ t u v wd o x 1o u o y d kz % & % # Inverse Transforms (Cont...) in addition, sum of transforms is transform of sums
A RG S T H A R U WG V X Y A D F 6 8 97 [email protected] 2 factoring could be the hard part CSC5420 Q # # ( ) 4 A &@ 7 h h W ` S d D 86 q a ts v u X r B C 5 786 9 % ' E GHF G @ X X q a q a w xyx x x x P P t 56 SU TR V XY # $ % & % ' % & " ! GP I 9 % ' % & " " ! ` W ba ` d i qf U i ph f U W fgU e 3 c S 0 1) 2 W Copyright John C.S. Lui use known transform Inverse Transforms (Cont...)
CSC5420 CSC5420 Example # ! % ! " & # % ! $ B C H E D % (' F G 3 8 3 @ I )! $ $ need numerator power < denominator power 2 poles: 1) 2) two poles here
1 342 A 75 A9 1 2 0 1 342 6 75 69 1 1 57 Copyright John C.S. Lui ungroup previous equation )((( ' ' ' ' ' ! 0 " 1 1 58 Copyright John C.S. Lui 1 1 (((3 4' ' ' ' ' 0 " ! % $ ! 2 )(' ' ' 0 " % % & $ ! # " & Example (Cont...) '('((4 ' ' ' ! 0 " CSC5420 d F $ ) 0( A D E % Q r u s U S` V 8H U X W y c S` V HU W ! "
59 ' by inspection f ge H 6G IP p i tu v 0q V 8H U w v yf P q q h s Q` abS B C9 sf x X YW @A 9 ST QR 75 864 2 31 h Copyright John C.S. Lui need to account for & " " # # # Example (Cont...)
CSC5420 CSC5420 Laplace Transform
Consider function of continuous time f(t), f(t) = 0, for t < 0 As before, want to transform from a function of t to a function of a complex variable s, and also want to be able to "untransform", so want "tag" each value f(t) use as our tag where G IPH F 3 ! " &$ %# A9 &@ 8 since assume f(t) = 0 for t<0 DE ' ( 0) 1 5 6 !) 71 2 4 any accumulation at origin (e.g., impluse function) will be included Exists as long as f(t) grown no faster than exponential, i.e., there is some real number s.t.
ed X X V YWU T a` i hd p r sq u &t g g !f v b c w R SQ
60 Copyright John C.S. Lui BC 6 CSC5420 Laplace Transform (Cont...) Laplace transform for a given f(t) is unique _ If integral of f(t) is finite, then Re(s)>0 represents region of analyticity for F*() (z = 1 corresponds to s = 0) Use notation: ! " Inverse by inspection 61 Copyright John C.S. Lui Ex 2: @ ( A CB u t R v w y x S T F $ d % U ) (& ' e t d d 10 r q ` Y a ` 2 X W W 9 U VI Q # E D ! " s H P I Q Ex 1: Copyright John C.S. Lui G 4 53 b a 3 60 1 2 7 3 8 9 c d b e f gd P W 2 ` Y a X Examples of Laplace Transforms b e i c f d p h unit step function in continuous time (to get f(t) defined above) CSC5420 62 P CSC5420 Inspection Method
Assume F*(s) is a rational function of s, i.e., where N(s) and D(s) are polynomials in s degree of N(s) < degree of D(s) factor D(s) #" ! 63 Copyright John C.S. Lui ( 21) 0 0 " ! P H 9 4 R Q TS H 9 E V H 9 U B % ' B B W c abXYXY`YX X X Td % % ' & " % & G G " ! # # % & 8 8 CB ED F6 ' HF ID 6 E % # $ Copyright John C.S. Lui 46 53 [email protected] A 7 Inspection Method (Cont...) from table CSC5420 64 ' 9 64 9 5 2 3 1 0 7 @ 2A ( ) 10 CD2B EF A 5 3 G @ P 7 @ 2 0 EF 5 @ C QSQRQRQX2 Q 3 I T I @ 2 0 I 8 7 @ 2 3 7 @ 2 0 5 I P 0 A2 3 G 0H I H 7 @ 2 1 0 7 @ 7 9 4 9 92 1 0 7 9A 4 8 2 8 2 1 0 7 @ 7 9 64 5 A2 7 5 64 5 1 0 ! " # !$ % & " % 18 & "$ # 3 U V 9 C SRQ Q Q2 EF 7 @ ( ) 10 7 G 10 2 0 7 8 2 9 EF 2 C QRQXQSQRWQ 8 2 7 18 1 0 1 0 3 5 3 G 1 0 7 8 1 YP 1 0 7 8 92 7 1 0 I 7 1 U 3 9 64 5 64 ( ) 10 Copyright John C.S. Lui Example CSC5420 65 using table 2 675 [email protected] B 9% " " 8 9 C 66 2 3 4 ) " ) " # # " 0 # # 1( ' # # ' # ) & ' ( ' # # % $ Copyright John C.S. Lui " 8 " # 6 8 9 6 % ) 0 765 C GEFD 6 H ! " Example (Cont...) 8 9 CSC5420 already took this derirative ( CSC5420 Difference Equations Nth order difference equation: (standard method) 8 a b Q X R 7 where ai are the known constants and gi are the unknown functions to be found, and en is a given function of n plus we are given N bounding equations as usual, solution has homogeneous and particular part: # " ! homogeneous solution must satisfy homogeneous equation:
1 1 % &$ ( )' 0 % % &$ 2 ()' 0 % 1 5 6$ U Y `G general form of solution is:
9 [email protected] B E where A and are to be determined
H H P TSR P W V U H IG Q HG Q U V X X P RS
67 C D Copyright John C.S. Lui S F C 0 ( )' 4 3 2 4 4 CSC5420 Difference Equations (Cont...) Nth order polynomial has N solutions: (assume for now that i are distinct) 's are determined from initial conditions by cancelling common terms, get characteristic equation:
! " ! " # $ % & ' ! $ find roots of
3 12 0 5 4 , if all i are distinct, then
A 86 D 6 4 4 @ B [email protected] 7 9 A 9 @ D 9 B B 4 with 1 as a multiple root of order k,
G Q RP Q RP T P G XV W G H EF S GHF S GHF S GHF U "S I I U U 7 86 # $ gn is determined by approp.given from the form en (p) G Q RP G T I Y ( 68 Copyright John C.S. Lui ) guess using init. cond. s p 7 9 @8 B CA e g 9 D yw ~ u z y z ~ hr u l " ! # ~ h jki m nl p o q u dge f u W Q RP W T XVUS `Y a u i e f I 6 5 h 0c f ' ) 0( 1 32 E GHF 4 0( 1 $ %& u v y y x yw t q sr d 0c u t v u t xw z } y v u t w { b homogen. sol. Copyright John C.S. Lui Example CSC5420 69 def. )( 0 ' 1 GH 2 54 6 7 8A 4B W W I YRU uss PH ` Ra Y bcX ` d f ge iph TX 54 6 F 4 C 4 8C ( D E4 ( 3 @ 3 5 98 " #! %&$ QP PI xr w SR U VT S d e f gh s xv i y g v qr tus Copyright John C.S. Lui " Use zTranform CSC5420 70 (same) " % ' (& # ) 0 1# P 1E I FE G H fu dc gh h k l i j 2 i s f b s Q ` FE G C X Y D ) # A f dx f ge yh RE G I RE V WG f i j g f ge iph I r 6q G v wt FE G H $# % & 9 @ # D RE S I RE T US m $# % ' 3# % ) 7 8# " Q D 3# 4 ) 5 6# " $# % " '2 e n o p rsq dc FE G H E I D $# t r sq vw qu ~} 6 (e d 6} d { z f i pt s 6q v 6 r xy r i 6q 6( f ge iph g b C Copyright John C.S. Lui a B ! Example  Use zTranform CSC5420 71 I E E CSC5420 Example  Use zTranform (Cont...) part. frac. exp. 0" ( 0" 0" ( ' ' & 2 ' ( $ 786 7 ( 7 3 9 9 9 1 " #! 1 $% ) 3 4 5 72 Copyright John C.S. Lui CSC5420 ConstantCoeff. Linear Differential Equations Nth order eq.: H ai's are const., e(t) is a given func. also given N init. cond. (usually first N derivatives, usually at t=0). find f(t) (h) (p) have f (t) and f (t) form subst. has N solutions which must solve char. eq: " #! % ( )' $ 3 & 0 1 3 9 B 9 9 6 )5 8 6 )5 C 8 6 )5 8 5 @ D @ @ 3 42 7 3 42 7 2 7 @ E F2 7 8 9 D B D P A T PU B R V R R I Q I Q I Q R W FI V P S )P T V if all i are distinct, then
e d f#c ab u u g p qi ts w qi xs i v y v r r v h y T X where Ai's are determined using init. cond.
Copyright John C.S. Lui y r s ` u Y G 73 CSC5420 ConstantCoeff. Linear Differential Equations (Cont...) when have multiple root 1 of order k, then have contribute in above form to homogeneous sol. (p) make a guess to find the particular sol. f (t)
complete sol. f(t) = f (t) + f (t) (h) (p) 74 Copyright John C.S. Lui homogen. sol. v 4 31 2 b 0 5 7 86 f [email protected] gh R pi f e p 3 r sj ut & $ ) '( y & xv Q7 y PB D 2 6 # $ %# nq m t u u p a G H6 @ D 6 IE w xv n om n h r 8q s Ci W B @ Y `X E 4 FD "! V B 4 CA 3 l k lj ig h f ige { ~ } d d ec z yz R TUS Copyright John C.S. Lui
xw C ! Example CSC5420 75 CSC5420 Example (Cont...)
complete sol. using init. cond. final sol. % 1 0 8 @ 8 8 @ C 3 5 4 7 6 FE C # " % ! $ " & ' 8 )$ # #( 5 5 G 2 9 A B D 9 G B 5 H I 76 Copyright John C.S. Lui CSC5420 Example  Using Laplace Transform Method
(using prev. example) 0 $# ! % ! ! & & " " ' " ' ( ) 2 31 D E Q Q 4 675 6 75 @ A9 5 S 75 6 75 9 R P 9 8 BC F G4 IH P C H H 4C H 9 TH 4 P C U 6 2 S S @ b ca e fd X p qg b r h i Y ` g b s t F part. frac. exp. inverting y cx f v w u y cx y y y gf f d e h (In queueing theory, sometimes need both, i.e., have differentialdifference equations then use both z and Laplace tranforms).
77 Copyright John C.S. Lui d gf V H W e 0 d e I G I 6H 5 64 7 8 X t u x yqr t p Ct d ) d & e b 6 Y X v wu i Y a ` 9 5 B C9 D 9 A @ ! # $" & '% ( 5 3 1 P V UQTQTQSQRQ r s W E 21 p g hf e p qi d FE f f hgf 6 TTSR Copyright John C.S. Lui Discrete time c Deriving Moments Via Transforms (Expectation) CSC5420 (not quite variance, but can get it from this and the exp.) 78 CSC5420 Deriving Moments Via Transforms (Cont...)
Continuous time
# % &$ ' ) 0 &$ ' 2 # % &$ '8 ( ! " 4 5 1 B DEC [email protected] F I C ST V F 9 Q R U U @ G D @ H g t rshqhqhphih c e fd ba P v e a sqqpi f u w y 67 3 % U HW: Compute first 2 moments of geometric and exponential distributions using transforms 79
Copyright John C.S. Lui X X `YX W x CSC5420 Relationship of Transforms to Expectations 3 1 $# % % ! ( 2 0 1 " 0 ( 1 4 $# % " ' ()# ! & 80 Copyright John C.S. Lui CSC5420 Inequalities and Limit Theorems
May not be possible to determine distributions, but might be able to derive and use: (a) moments (b) inequalities and limits 81 Copyright John C.S. Lui CSC5420 Markov Inequality
Simple Markov Inequality: If only know the expectation, provides a bound on probability distribution For a r.v. X with mean , the Markov Inequality is: assume X is a nonnegtive r.v. Proof: area under the curve ! " FX(x) = 1  FX(x) 1 FX(t) t x area of rectangle
% &$ )(' 1 20 ( # P 3 4 8 area under the curve
65 7 [email protected] CB EFD C [email protected] 9 G 9 CB D H I C G H 82 Copyright John C.S. Lui CSC5420 Example
Consider a system with MTTF of 100 hours Let X be a r.v. denoting the lifetime of a system By Markov Inequality:
! ! Define reliability of a system as Then
() " $ %# & ' $ ) To ensure system reliability more than 0.9, the system mission time t < 111. 83 Copyright John C.S. Lui CSC5420 Example (Cont...)
HW: How tight is the Markov bound for the exponential distribution with parameter Review Distribution Exponential distribution: f ( )' 4 53 E F # @ 6 # $" ! % & 7 0 1 2 8 1A B R $Q U I V S b 5a R $Q I PH S X YW ` T D )C d 9 7 G FX(x) 1 R U T 1e x fX(x) e
x x Copyright John C.S. Lui c e x 1 84 CSC5420 Chebycher's Inequality Assume we know and of r.v. X, then for all t > 0,
2 Proof: 2 let Y = (X  ) Y is a nonnegative r.v. applying Markov Inequality: ( ! " & #! $" %" & ' %" ) 89 A [email protected] [email protected] D FG G 1 6 5 8A E C H %" 0 " but
1 3 42 6 75 85 Copyright John C.S. Lui CSC5420 Weak Law of Large Numbers
Let be independant identically distributed r.v.'s with Define arithmetic mean to be " ! %# &$ " 0 1) 2 31) (' We would expect that for sufficient large
@ DB ECA F B G S H I QP 8 Let ; consider r.v.
ir S 7R U Y T d e f hbg p V WU X i i qr q c a b` 9 2 67) 4 4 4 52 x y y uw vt uw t s gi hf j lbk m d e d n Copyright John C.S. Lui 86 CSC5420 Weak Law of Large Numbers (Cont...)
Applying Chebychev's Inequality to # $ %'%&% % ) 1 0( 2 3% %'%&% 4 5 6 2 7 @98 A 1 2 ! ! 7 B , we get " the distribution of arithmetic mean becomes increasingly concentrated around the mean as n grows! can be thought of as the error in approximating by the arithmetic mean G E HFDC PI Q R S'&S S S U G T V S SW S '&S X Y ` V a b
or
Copyright John C.S. Lui for any small , as n grows, the error will be less than with probability 1 87 f e HFdc hg i p q'&q q q s f r t q qu q '&q v w x t y weak law of large numbers Copyright John C.S. Lui
ED I G HF $ " %#! '& a P b` a c Q R ST V WU X I P 9 Q Y B @ A 3 4 5 67 8 ( ( ( 10)( ( ( ( 10)( 2 $ Strong Law of Large Numbers CSC5420 88 C CSC5420 Central Limit Theorem where Xi are i.i.d. with mean X and variance X
2 PDF of Zn tends to the standard normal, i.e., x is real " ' % )0 1 # &%$ A
where (Gaussian or normal distribution) [email protected] D E BC H9 G H PQI F
89 ( 4 52 3 ! 6 7 Copyright John C.S. Lui 8 ...
View
Full Document
 Spring '10
 Lui
 Normal Distribution, Probability theory, probability density function, John C.S. Lui

Click to edit the document details