02_intro - CSC5420 CSC5420 Computer System Performance...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: CSC5420 CSC5420 Computer System Performance Evaluation John C.S. Lui http://www.cse.cuhk.edu.hk/~cslui/csc5420 1 Copyright John C.S. Lui CSC5420 Introduction Material from lectures (ref. books on web page) Grading homework for students' benefit (will include use of software-tools on web page) 10% homework 40% projects 50% final exam 2 Copyright John C.S. Lui CSC5420 Course Material Review of Probability, R.V., Transforms Intro. to Stoch. process (m.c.'s) Baby queuery theory m/m/1... Intermediate queuery theory m/g/1... Markovian model is a special structure Appr. Tech. Stoch. Couple Matrix geometric structure Sample Path Analysis Transient Analysis Reversibility Queuery Networks - product form Simulation Measurements Copyright John C.S. Lui Real World Model Solution 3 CSC5420 Project: List to choose from, FCFS MS Comp: Final 4 Copyright John C.S. Lui CSC5420 Combinatorics Permutations k-permutation of a set of n elements Combinations k-combination of a set of n elements k-permutation / k! k! is the number of possible ways to permute that combination 5 Copyright John C.S. Lui CSC5420 Combinatorics (Cont...) Binomial Coefficients: Binomial Expansion: 6 Copyright John C.S. Lui CSC5420 Probability Sample Space (S), collection of objects, where each object is a sample point. A family of events, ={A,B,C,...} where an event is a set of sample points. A probability measure P is an assignment (mapping) of events defined on S into real numbers (which has properties or axioms). P[A] = Probability of event A 7 Copyright John C.S. Lui CSC5420 Probability S = sample space = set whose elements are elementary events (possible outcome of an experiment) The elementary events are points in a sample space (1 or more dimensions), and they are mutually exclusive Ex: flipping a coin, a elementary events (sample points) H, T Event: subset of sample points Ex: toss dice event 1 2 4 3 6 5 sample point 8 Copyright John C.S. Lui CSC5420 Probability (Cont...) Aximoms of Probability: A probability distribution Pr{} on a sample space S is a mapping from events of S to real numbers s.t. the following proability axioms hold: 1) Pr{A} 0 for any event A (where Pr{A} probability of event A) 2) Pr{S} = 1 3) Pr{AB} = Pr{A} + Pr{B} for events A and B that are mutually exclusive 9 Copyright John C.S. Lui c) 7 6 &5 98 a) d) ' &$ % b) 0)( Copyright John C.S. Lui C DB 2 31 4 EF A H IG P W Q ! C DB EF C DB EF R SP R SP C B H E C B H E " # " ! @ Things that follow: Probability (Cont...) P T UP C DB EF H IV P CSC5420 10 CSC5420 Discrete Probability Distribution Probability distribution is discrete if it is defined over a finite or countably infinite sample space if x's are mutually exclusive events in A If S is finite and event elementary event in S has probability then we have the uniform distribution on S (or we pick an element of S at random) 11 Copyright John C.S. Lui CSC5420 Discrete Probability Distribution (Cont...) Ex: flipping a fair coin, Pr{H} = Pr{T} = 0.5 flip coin n times A = {exactly k heads and exactly n-k tails} 1% 342 6 " #! $% %& ( ' )0 ($ $ 5 12 Copyright John C.S. Lui CSC5420 Continuous Uniform Probability Distribution Defined over closed interval [a,b] of reals where a < b (all subsets here, not events) want each point in [a,b] to be equally likely but, infinite number of points, if give each one finite probability, will not be able to satisfy axioms 2 and 3 associate probability with some of the subsets for any closed interval [c,d], a c d b, continuous uniform probability distribution: ! # $% '% 4 ! $% % # ( 0 1) 23 " 4 & & 4 " " " & 5 ' 13 Copyright John C.S. Lui CSC5420 Conditional Probabilities and Independence Def'n: normalizing (sum to 1) Ex: 1 2 4 6 ! constrained sample space, so we scale up 0 ( 1 2 3 5 " # $% & ' ' ) 3 0 1 0 14 Copyright John C.S. Lui CSC5420 Conditional Probabilities and Independence (Cont...) A and B are statistically independent if and only if: " ! ' # ! 10 ) 4 % &$ $ 10 ) 10 ) 4 23 5 3 4 30 ) 4 5 If A1, A2, ..., An are statistically independent ( Also, if A and B are statistically independent, then 15 Copyright John C.S. Lui CSC5420 Theorem of Total Probability more useful form where {Ai} is a set of mutually exclusive exhaustive events if occurs, occurs with exactly 1 mutually exclusive exhaustive event (Ai) ! ' using conditional probability: ) ' # ' # $) ' " & " & " & " & ( % % ( % " % ' #$ #$ #$ 16 Copyright John C.S. Lui CSC5420 Theorem of Total Probability (Cont...) Ex: reliability R1 R2 R3 R3 R3 R4 R5 R5 where {Ri} is the reliability of component i Importance of Theorem of Total Probability is to break a complex problem into many simpler problems 17 Copyright John C.S. Lui CSC5420 Bayes' Theorem Look at problem from another perspective Assume we know event B has occurred, but we want to find which mutually exclusive event has occurred " $ " $ " # # ! ! % " $ " # ! more general form above Copyright John C.S. Lui # $ # $ 18 CSC5420 Bayes' Theorem (Cont...) More general forms: Ai, 1 i n, are mutually exclusive, exhaustive events Theorem of total probability " # &# " ! $ (&'% ) & Bayes' Theorem 19 Copyright John C.S. Lui CSC5420 Example Ex: gambling, DH honest dealer, DC cheating dealer L you lose play honest dealer lose with prob = 1/2 play cheating dealer lose with prob = p (of p > 1/2 against you, of p < 1/2 for you) if p=1, prob. that cheating dealer if lost 1 game = 2/3 20 Copyright John C.S. Lui CSC5420 Random Variables (R.V.) We have the probability system (S, , P) R.V. is a variable whose value depends upon the outcome of the random experiment The outcome of a random experiment is w S We associate a real number X(w) with W Thus our r.v. X(W) is nothing more than a function defined on the sample space S i.e., a function from a finite or countably infinite sample space S to real numbers 21 Copyright John C.S. Lui CSC5420 Example Ex: playing a game of black jack in Las Vegas Associate a probability with it Sample Space: all possible pairs of scores obtained by player & dealer (single game) W 3/8 D 1/4 L 3/8 -5 (lose $5) 0 ($0) 5 (win $5) real line Notation: R.V. X on a sample S X:SR X(w) - our winnings on a single game of black jack Copyright John C.S. Lui X(w) = +5 w W 0 wD -5 w L 22 CSC5420 Discrete Random Variables Discrete Random Variables: (X) a function from a finite (or countably infinite) sample sapce S to real numbers interested in functions of events each outcome is a combination of events, so we can assign probability to it P[Xx]: probability distribution function 1 x 23 Copyright John C.S. Lui u j i h d kl xy w v }x { z ~ p q 4 z ~ ih F 4 )C } m u o t i g % S G 9 7 s z rq n m e gf h i ht t E vs % " 0 Y r v s s ih p a q R ' & () @ A6B B DF EC w v V z ~ a z fg % " % H0 P QI 1 23 4 6 5 75 9 78 " 0 p m v { |} n o d ec i s () uy ' & () z m ' & % v xew Y ab` 1 xy kl $ t us " #! Copyright John C.S. Lui w v j i T W V X#U Discrete Random Variable (Cont...) CSC5420 24 ! # $" `Y |x { a lki j { nm t x o F G 8 W gf D HG a g d C pq s k nm h I e u h p n q d x e u Y s r p nm ot e E U S T @ U S T v V x E U x yw v @ S T t t us C G 7 R ot t p Y e ) 1 v c r G C PQI D &6 i G 5 4 C 2 3' # 1 $% h ` a h p n r kq w z 0 Y gf A E ) ~ k{ } edb c D C '( h &% A B@ v z yw x d f ge X 9 Copyright John C.S. Lui W 8 Expectation, Variance, and Standard Deviation FX(x) 1 x CSC5420 25 CSC5420 Probability Distribution Function (PDF) or Cumulative Distribution Function 6 5 8 7 9 CB A ed Wf g @ c b h d i f 8D ! ) 0( 231 ) 4& 0( "# & '%$ E Properties: 1) 2) 3) QI P H G F T S VWU h pq 4) 5) e vu c b y X `Y Ex: PDF for the Las Vegas Game 1 3/8 5/8 1/4 3/8 26 a -5 Copyright John C.S. Lui y 0 5 R xw h re r ts CSC5420 Probability Density Function (pdf) or Probability Mass Function (pmf) Ex: pdf for the blackjack game 3/8 1/4 3/8 -5 0 5 Different ways to view pdf: 1) 2) 3) % &$ ( )' 0 6 75 9 8 A@ 9 43 3 1 2 B T V WU Y X a` Y R S G I H PQ C 4) D F E ! " # " 27 Copyright John C.S. Lui CSC5420 Special Discrete Distribution The Bernoulli pmf pmf q p PDF 1 q 0 1 0 ! 1 " 28 Copyright John C.S. Lui CSC5420 Geometric Distribution Bernoulli trial: experiment with only 2 possible outcomes success with probability p failure with probability q = 1-p Bernoulli trials, a sequence of independent trials each with probability p r.v. X = number of trials needed to obtain success X { 1, 2, ... } ! " & ! S T U6 4 geometric distribution 6 F D EC 5 786 R EQ 10 )0 43 2 9 D 4 H IG P R 4 W XG P @ V e fd AB Y a b` c g ` g on average, 1/p trials before obtain success 29 Copyright John C.S. Lui $ % ' ( ! # CSC5420 Binomial Distribution r.v. X = number of successes in n trials, X { 0, 1, ... } ! " $# % binomial distribution 5 786 ) & 3 412 12 0 0 A 2 B 9@ '( 30 Copyright John C.S. Lui CSC5420 Multiple R.V. Can, of course, define many r.v. on same sample space Let X & Y be 2 r.v. on some probability system (S, , P) Natural extension of PDF: joint PDF Joint probability density function: 21) 0 ) $%#! " $%#! " &' ( ") & &' H I9 H Marginal density function (for one of the variables): 7 48 53 6 9 A D CB A 4E 53 FG86 7 @ (given by integrating over all possible values of the 2nd variable) 31 Copyright John C.S. Lui CSC5420 Multiple R.V. (Cont...) Notion of independence between r.v.'s X & Y are independent iff: (same for more than 2 variables) ) Can also define one random variable in terms of another, i.e., %$ $0 34 54 16 6 '(& ! # ' (& # 21 7) " " Of course, can be a function of many r.v. (could be complex computation) 32 Copyright John C.S. Lui CSC5420 Example Ex: Let Y = X1+X2 (i.e., sum of 2 r.v.) where X1 and X2 are independent % # &)( ' ! " 21 30 1 65" 4 9# 7 684 654 48 x1 $ &% &% x1+x2y x2 y ) 9 33 Copyright John C.S. Lui CSC5420 Example (Cont...) due to independence: ! " # ! # 2 ) 76 8(& C6 8(& A B@ A B@ 54 D2 A@ Q # ! # 0 ) W US E V 9 ' (& 0 1) 2 3 4 convolution of density functions of X1 and X2 F GE I PH Q I PH Q R TUSGE I PH (same for any n sum of independent r.v.) $ # % 34 Copyright John C.S. Lui CSC5420 Expectation The expectation of a real r.v. X(w) is denoted by E[X] ! ' # %$ 2 4 5 "! 7 9@8 A ) 1 7 BC 01 3 3 65 7 & # ( I P Stieltjes Integral W XV G Y T US T US H R P R For X, a nonnegative r.v. e g t Xs b r Uq u ev cd ci t f h p t w x aI W V `Y W D EF G` EQ W 35 Copyright John C.S. Lui CSC5420 Fundementation Theorem of Expectation Let y=g(x) could be complex to compute fY(y) & ! " $% # Fundementation Theorem of Expectation: ( "# )% ! '& ( "# 0% ( 8 98 (generalize to any number of variables) 36 Expectation of sum of 2 r.v. 8 98 8 A 8 98 E 9 8 98 8 F A "@ 8 A E A "@ PD A FG CB4 A H "@ ED 4 B FG PD 98 B 8 H A I"@ A P B E B G PD B @ A 4 PD BP B 98 8 32 1 4 7 56 7 32 9 8 6 True whether or not X & Y are independent Copyright John C.S. Lui 7 41 56 1 2 FG HI"@ A B PD A BP 7 B E m t o pn q o r s uvt d Q h f Q ia R e j r t w kh g li h i u g f ca e H I P ` B t | } q o w~ { fe h 4g u c da b FG 4 X `Y ED C u v x yw ! W B 9 Copyright John C.S. Lui Interested in power of r.v.'s x yw z If X & Y are independent, then Product of R.V. p q h sr h S HT R VU "# @4 $ !1 A2 & % % 1 "2 & % # % ' $ ) 0( 1 @ 4 '42 3 !1 5 0( 7 2# 31 62 A '6 3 1 "2 $ # ! 78# binomial theorem "# CSC5420 37 follows from the fundemental theorem of expectation l n om p s tr f e d g i g h j k y v y c qr irg h a V Y X V Y a b ) 0 $ w y t u t ih p A S nq ih q@p r s S T ) HG I x y HG P@I QR $ % $ '& ( 1 20 V WU X V Y ` $ % '& ( # d e g8f B EFC D !" c A Copyright John C.S. Lui '& 9@( 3 4 53 64 % !" 7 8# take expectation of both sides Product of R.V. (Cont...) coefficient of variation sums of expectation, expectation of sums, expectation of constant CSC5420 38 CSC5420 Transforms Transforms, characteristic function, generating function... Laplace, z, Fourior, ... When introduce into solution method, simplify calculations Appear naturally, why? Linear Time-invariant Systems systems transformations, mapping, input-output relationship between 2 functions f System g black box assume f=f(t), f(t) g(t) 39 Copyright John C.S. Lui CSC5420 Linear and Time-invariant Linear if when f1(t) g1(t) and f2(t) g2(t) then also af1(t)+bf2(t) ag1(t)+bg2(t) Time-invariant if when f(t) g(t) then also f(t+) g(t+) If both holds, we have a linear time-invariant system we focus on these 40 Copyright John C.S. Lui CSC5420 Transforms Decompose function of time into sums (integrals) of complex exponentials complex exponentials form building blocks of transforms Question: which functions of time can pass through linear time-invariant systems without change? i.e., f(t) g(t) = Hf(t), where H is some scalar multiple if can find these, then can find eigenfunctions or characteristic functions, or invariants of our system where s is a complex variable form the set of eigenfunctions for all linear time-invariant systems 41 Copyright John C.S. Lui CSC5420 Characteristic Functions Derivation: linearity $# " ! % & '( 0 1) 32 654 DE 7 CB A timeinvariant where and hence e s are constant 32 uv independent of t but can be function of s 9@ IH R SQP T WVU X unique solution gf d ec w i q rp y uv x w st h T WVU ba` YR X 0G 8 654 F 7 42 Copyright John C.S. Lui CSC5420 Characteristic Functions (Cont...) Overall output found by summing (integrating) these individual components of the output decompose input into sums of exponentials, computing response to each as above, and then reconstituting the output from sums of exponentials is referred to as transform method 43 Copyright John C.S. Lui q Ip s Tr Y a b` # ( ) 10 2 '& @ A9 5 CB ) 3 ED 7 @ s CB (8 10 v 3 4 8 5 F H IG 3 60 2 QP Q G SU TR VS WX Q H R QP R 2 60 2 4 % ! " 3 % "$ c d tu e v w g h x e y g d i f f Copyright John C.S. Lui eigenfunctions: b Focus on discrete time first also a complex variable 44 Transforms CSC5420 CSC5420 Transforms (Cont...) ! % &$ ( 01) 0 B7 0 @ A9 8 # ' 32 56 7 2 7 D 8 9 " H expresses how much we get out of unit input system (or transfer) function Kronecker delta function or unit function: P QI R F GE H H T S H R S S Unit response (when apply un to system), hn V GU W V X f Y a b` dc e dc a B26 CB 4 45 Copyright John C.S. Lui CSC5420 Transforms (Cont...) linearity multiple by unity on both sides # $" V T R USQ I PH ic d b c d H W X ! Consider set of inputs {fn } ( 0) 1 2 2 4 0) ' 3 1 (i) 78 69 B 8 69 linearity @ 6 b e fc d 6 @ a hUb Y` a b ` g` b c c d sum over all integer value of m only 1 non-zero term, when m=-n, and it equals 1 c ` b D FGE 5 A 5 DC 46 Copyright John C.S. Lui % & CSC5420 Transforms (Cont...) plus change of variables $ &'% ( # 0 0& ) 1 0 related system function H(z) to unit response both H(z) and unit response describe how the system operates, so they are related itself a transform, a z-Transform so transforms arise naturally ! " " 47 Copyright John C.S. Lui CSC5420 z-Transform Let fn be a function which takes on nonzero values only for non-negative integers, n=0, 1, 2, ... (fn=0 for n < 0) Compress sequence into a single function such that can expand later Place a tag on each term n i.e., tag each fn with z (n unique each tage is unique) Define z-transform (or generating function, or geometric transform) ! The z-transform will exist as long as terms don't grow any faster than geometrically, i.e., as long as a exists, s.t. 48 Copyright John C.S. Lui CSC5420 z-Transform (Cont...) Given a sequence fn, its z-transform is unique If sum over all fn is finite, then F(z) is analytic on |z| 1. Then: Notation: has a unique derivative at that point function is analytic at that point 49 Copyright John C.S. Lui CSC5420 Examples of z-Transforms Ex 1: Recall the unit function A C DB 7 86 Q DT P FU GA Q RP A F S QP A W XP 9 U HGA F 5 9@ E P W aHF 50 I V I I Q Q ) Ex 2: Shift the unit function to the right 1 20 3 4 $" %#! Copyright John C.S. Lui & '( & Y `Q W Copyright John C.S. Lui V u It v fe g t i Ih pqe Y b e u region of analycity Y `W X bca d P F R Q R S RUU T U D EC F H IG D Ex 4: geometric series Ex 3: unit step function y x u h fe g Y ba e x r s i Y iX w 1 20 3 4 76 5 38 @ 9 @ 4 @BB A B d Examples of z-Transforms (Cont...) h & 51 ! !" $# !% ' " () %" CSC5420 Copyright John C.S. Lui @ A9 DCB @ )' 0( 12 3 54 6 a b` edc E HIG C F GP Q @R G a i pg h rsq i pt h q f Properties of z-Transforms 5 67 4 8 F " #! G R A9 5 @ 3 67 3 $ PG @R 5 4 % 7 S TG S & % VW % " & P X HIG C F Q " % YW GP PX % G HU I 52 CSC5420 Convolution property We have 2 function, fn and gn with fn F(z) and gn G(z) UP 9 U F CSC5420 Other Properties of z-Transforms linearity scale change in the transform and time domains #$ ! " % '(& 0) 2 1 3 5640 5 58 70 F GE HI P R TS Q RXUV W 88 Y `E a D @ BA 1 9C 9 tsq r u wv w xy advance or delay by unit of time results in divide or multiply in transform domain Rb RU W V T b F GE Hb cU d eV P R hgIf p i If i fb iI E 53 Copyright John C.S. Lui CSC5420 Inverse Transforms Given F(z), find sequence fn Power series method, e.g., not as useful if want many terms ratio of numerators and denominators Inspection method express F(z) in terms which have recognizable transform pairs partial-fraction expansion 54 Copyright John C.S. Lui also y B C A E D F B # x h k lj e m on q d p s "r ) 55 ` a c db g h ef f w ' ( ' i uIs t v r4p q Copyright John C.S. Lui ut p e Wd f gd E D F H IB A P QG ) P P e.g., each term is either 1) a simple pole 2) a multiple pole G 3 42 E D 5 i 0 1 ( ' !" # $ t u v wd o x 1o u o y d kz % & % # Inverse Transforms (Cont...) in addition, sum of transforms is transform of sums A RG S T H A R U WG V X Y A D F 6 8 97 3@ 2 factoring could be the hard part CSC5420 Q # # ( ) 4 A &@ 7 h h W ` S d D 86 q a ts v u X r B C 5 786 9 % ' E GHF G @ X X q a q a w xyx x x x P P t 56 SU TR V XY # $ % & % ' % & " ! GP I 9 % ' % & " " ! ` W ba ` d i qf U i ph f U W fgU e 3 c S 0 1) 2 W Copyright John C.S. Lui use known transform Inverse Transforms (Cont...) CSC5420 CSC5420 Example # ! % ! " & # % ! $ B C H E D % (' F G 3 8 3 @ I )! $ $ need numerator power < denominator power 2 poles: 1) 2) two poles here 1 342 A 75 A9 1 2 0 1 342 6 75 69 1 1 57 Copyright John C.S. Lui ungroup previous equation )((( ' ' ' ' ' ! 0 " 1 1 58 Copyright John C.S. Lui 1 1 (((3 4' ' ' ' ' 0 " ! % $ ! 2 )(' ' ' 0 " % % & $ ! # " & Example (Cont...) '('((4 ' ' ' ! 0 " CSC5420 d F $ ) 0( A D E % Q r u s U S` V 8H U X W y c S` V HU W ! " 59 ' by inspection f ge H 6G IP p i tu v 0q V 8H U w v yf P q q h s Q` abS B C9 sf x X YW @A 9 ST QR 75 864 2 31 h Copyright John C.S. Lui need to account for & " " # # # Example (Cont...) CSC5420 CSC5420 Laplace Transform Consider function of continuous time f(t), f(t) = 0, for t < 0 As before, want to transform from a function of t to a function of a complex variable s, and also want to be able to "untransform", so want "tag" each value f(t) use as our tag where G IPH F 3 ! " &$ %# A9 &@ 8 since assume f(t) = 0 for t<0 DE ' ( 0) 1 5 6 !) 71 2 4 any accumulation at origin (e.g., impluse function) will be included Exists as long as f(t) grown no faster than exponential, i.e., there is some real number s.t. ed X X V YWU T a` i hd p r sq u &t g g !f v b c w R SQ 60 Copyright John C.S. Lui BC 6 CSC5420 Laplace Transform (Cont...) Laplace transform for a given f(t) is unique _ If integral of f(t) is finite, then Re(s)>0 represents region of analyticity for F*() (z = 1 corresponds to s = 0) Use notation: ! " Inverse by inspection 61 Copyright John C.S. Lui Ex 2: @ ( A CB u t R v w y x S T F $ d % U ) (& ' e t d d 10 r q ` Y a ` 2 X W W 9 U VI Q # E D ! " s H P I Q Ex 1: Copyright John C.S. Lui G 4 53 b a 3 60 1 2 7 3 8 9 c d b e f gd P W 2 ` Y a X Examples of Laplace Transforms b e i c f d p h unit step function in continuous time (to get f(t) defined above) CSC5420 62 P CSC5420 Inspection Method Assume F*(s) is a rational function of s, i.e., where N(s) and D(s) are polynomials in s degree of N(s) < degree of D(s) factor D(s) #" ! 63 Copyright John C.S. Lui ( 21) 0 0 " ! P H 9 4 R Q TS H 9 E V H 9 U B % ' B B W c abXYXY`YX X X Td % % ' & " % & G G " ! # # % & 8 8 CB ED F6 ' HF ID 6 E % # $ Copyright John C.S. Lui 46 53 9@ A 7 Inspection Method (Cont...) from table CSC5420 64 ' 9 64 9 5 2 3 1 0 7 @ 2A ( ) 10 CD2B EF A 5 3 G @ P 7 @ 2 0 EF 5 @ C QSQRQRQX2 Q 3 I T I @ 2 0 I 8 7 @ 2 3 7 @ 2 0 5 I P 0 A2 3 G 0H I H 7 @ 2 1 0 7 @ 7 9 4 9 92 1 0 7 9A 4 8 2 8 2 1 0 7 @ 7 9 64 5 A2 7 5 64 5 1 0 ! " # !$ % & " % 18 & "$ # 3 U V 9 C SRQ Q Q2 EF 7 @ ( ) 10 7 G 10 2 0 7 8 2 9 EF 2 C QRQXQSQRWQ 8 2 7 18 1 0 1 0 3 5 3 G 1 0 7 8 1 YP 1 0 7 8 92 7 1 0 I 7 1 U 3 9 64 5 64 ( ) 10 Copyright John C.S. Lui Example CSC5420 65 using table 2 675 A@ B 9% " " 8 9 C 66 2 3 4 ) " ) " # # " 0 # # 1( ' # # ' # ) & ' ( ' # # % $ Copyright John C.S. Lui " 8 " # 6 8 9 6 % ) 0 765 C GEFD 6 H ! " Example (Cont...) 8 9 CSC5420 already took this derirative ( CSC5420 Difference Equations Nth order difference equation: (standard method) 8 a b Q X R 7 where ai are the known constants and gi are the unknown functions to be found, and en is a given function of n plus we are given N bounding equations as usual, solution has homogeneous and particular part: # " ! homogeneous solution must satisfy homogeneous equation: 1 1 % &$ ( )' 0 % % &$ 2 ()' 0 % 1 5 6$ U Y `G general form of solution is: 9 A@ B E where A and are to be determined H H P TSR P W V U H IG Q HG Q U V X X P RS 67 C D Copyright John C.S. Lui S F C 0 ( )' 4 3 2 4 4 CSC5420 Difference Equations (Cont...) Nth order polynomial has N solutions: (assume for now that i are distinct) 's are determined from initial conditions by cancelling common terms, get characteristic equation: ! " ! " # $ % & ' ! $ find roots of 3 12 0 5 4 , if all i are distinct, then A 86 D 6 4 4 @ B C@ 7 9 A 9 @ D 9 B B 4 with 1 as a multiple root of order k, G Q RP Q RP T P G XV W G H EF S GHF S GHF S GHF U "S I I U U 7 86 # $ gn is determined by approp.-given from the form en (p) G Q RP G T I Y ( 68 Copyright John C.S. Lui ) guess using init. cond. s p 7 9 @8 B CA e g 9 D yw ~ u z y| z ~ hr u l " ! # ~ h jki m nl p o q u dge f u W Q RP W T XVUS `Y a u i e f I 6 5 h 0c f ' ) 0( 1 32 E GHF 4 0( 1 $ %& u v y y x yw t q sr d 0c u t v u t xw z } y| v u t w { b homogen. sol. Copyright John C.S. Lui Example CSC5420 69 def. )( 0 ' 1 GH 2 54 6 7 8A 4B W W I YRU uss PH ` Ra Y bcX ` d f ge iph TX 54 6 F 4 C 4 8C ( D E4 ( 3 @ 3 5 98 " #! %&$ QP PI xr w SR U VT S d e f gh s xv i y g v qr tus Copyright John C.S. Lui " Use z-Tranform CSC5420 70 (same) " % ' (& # ) 0 1# P 1E I FE G H fu dc gh h k l i j 2 i s f b s Q ` FE G C X Y D ) # A f dx f ge yh RE G I RE V WG f i j g f ge iph I r 6q G v wt FE G H $# % & 9 @ # D RE S I RE T US m $# % ' 3# % ) 7 8# " Q D 3# 4 ) 5 6# " $# % " '2 e n o p rsq dc FE G H E I D $# t r sq vw qu ~} 6| (e d 6} d { z f i pt s 6q v 6 r xy r i 6q 6( f ge iph g b C Copyright John C.S. Lui a B ! Example - Use z-Tranform CSC5420 71 I E E CSC5420 Example - Use z-Tranform (Cont...) part. frac. exp. 0" ( 0" 0" ( ' ' & 2 ' ( $ 786 7 ( 7 3 9 9 9 1 " #! 1 $% ) 3 4 5 72 Copyright John C.S. Lui CSC5420 Constant-Coeff. Linear Differential Equations Nth order eq.: H ai's are const., e(t) is a given func. also given N init. cond. (usually first N derivatives, usually at t=0). find f(t) (h) (p) have f (t) and f (t) form subst. has N solutions which must solve char. eq: " #! % ( )' $ 3 & 0 1 3 9 B 9 9 6 )5 8 6 )5 C 8 6 )5 8 5 @ D @ @ 3 42 7 3 42 7 2 7 @ E F2 7 8 9 D B D P A T PU B R V R R I Q I Q I Q R W FI V P S )P T V if all i are distinct, then e d f#c ab u u g p qi ts w qi xs i v y v r r v h y T X where Ai's are determined using init. cond. Copyright John C.S. Lui y r s ` u Y G 73 CSC5420 Constant-Coeff. Linear Differential Equations (Cont...) when have multiple root 1 of order k, then have contribute in above form to homogeneous sol. (p) make a guess to find the particular sol. f (t) complete sol. f(t) = f (t) + f (t) (h) (p) 74 Copyright John C.S. Lui homogen. sol. v 4 31 2 b 0 5 7 86 f 9@ gh R pi f e p 3 r sj ut & $ ) '( y & xv Q7 y PB D 2 6 # $ %# nq m t u u p a G H6 @ D 6 IE w xv n om n h r 8q s Ci W B @ Y `X E 4 FD "! V B 4 CA 3 l k lj ig h f ige { ~| } d d ec z yz R TUS Copyright John C.S. Lui xw C ! Example CSC5420 75 CSC5420 Example (Cont...) complete sol. using init. cond. final sol. % 1 0 8 @ 8 8 @ C 3 5 4 7 6 FE C # " % ! $ " & ' 8 )$ # #( 5 5 G 2 9 A B D 9 G B 5 H I 76 Copyright John C.S. Lui CSC5420 Example - Using Laplace Transform Method (using prev. example) 0 $# ! % ! ! & & " " ' " ' ( ) 2 31 D E Q Q 4 675 6 75 @ A9 5 S 75 6 75 9 R P 9 8 BC F G4 IH P C H H 4C H 9 TH 4 P C U 6 2 S S @ b ca e fd X p qg b r h i Y ` g b s t F part. frac. exp. inverting y cx f v w u y cx y y y gf f d e h (In queueing theory, sometimes need both, i.e., have differential-difference equations then use both z and Laplace tranforms). 77 Copyright John C.S. Lui d gf V H W e 0 d e I G I 6H 5 64 7 8 X t u x yqr t p Ct d ) d & e b 6 Y X v wu i Y a ` 9 5 B C9 D 9 A @ ! # $" & '% ( 5 3 1 P V UQTQTQSQRQ r s W E 21 p g hf e p qi d FE f f hgf 6 TTSR Copyright John C.S. Lui Discrete time c Deriving Moments Via Transforms (Expectation) CSC5420 (not quite variance, but can get it from this and the exp.) 78 CSC5420 Deriving Moments Via Transforms (Cont...) Continuous time # % &$ ' ) 0 &$ ' 2 # % &$ '8 ( ! " 4 5 1 B DEC A@ F I C ST V F 9 Q R U U @ G D @ H g t rshqhqhphih c e fd ba P v e a sqqpi f u w y 67 3 % U HW: Compute first 2 moments of geometric and exponential distributions using transforms 79 Copyright John C.S. Lui X X `YX W x CSC5420 Relationship of Transforms to Expectations 3 1 $# % % ! ( 2 0 1 " 0 ( 1 4 $# % " ' ()# ! & 80 Copyright John C.S. Lui CSC5420 Inequalities and Limit Theorems May not be possible to determine distributions, but might be able to derive and use: (a) moments (b) inequalities and limits 81 Copyright John C.S. Lui CSC5420 Markov Inequality Simple Markov Inequality: If only know the expectation, provides a bound on probability distribution For a r.v. X with mean , the Markov Inequality is: assume X is a non-negtive r.v. Proof: area under the curve ! " FX(x) = 1 - FX(x) 1 FX(t) t x area of rectangle % &$ )(' 1 20 ( # P 3 4 8 area under the curve 65 7 A@ CB EFD C A@ 9 G 9 CB D H I C G H 82 Copyright John C.S. Lui CSC5420 Example Consider a system with MTTF of 100 hours Let X be a r.v. denoting the lifetime of a system By Markov Inequality: ! ! Define reliability of a system as Then () " $ %# & ' $ ) To ensure system reliability more than 0.9, the system mission time t < 111. 83 Copyright John C.S. Lui CSC5420 Example (Cont...) HW: How tight is the Markov bound for the exponential distribution with parameter Review Distribution Exponential distribution: f ( )' 4 53 E F # @ 6 # $" ! % & 7 0 1 2 8 1A B R $Q U I V S b 5a R $Q I PH S X YW ` T D )C d 9 7 G FX(x) 1 R U T 1-e -x fX(x) e -x x Copyright John C.S. Lui c e x 1 84 CSC5420 Chebycher's Inequality Assume we know and of r.v. X, then for all t > 0, 2 Proof: 2 let Y = (X - ) Y is a non-negative r.v. applying Markov Inequality: ( ! " & #! $" %" & ' %" ) 89 A B@ C@ D FG G 1 6 5 8A E C H %" 0 " but 1 3 42 6 75 85 Copyright John C.S. Lui CSC5420 Weak Law of Large Numbers Let be independant identically distributed r.v.'s with Define arithmetic mean to be " ! %# &$ " 0 1) 2 31) (' We would expect that for sufficient large @ DB ECA F B G S H I QP 8 Let ; consider r.v. ir S 7R U Y T d e f hbg p V WU X i i qr q c a b` 9 2 67) 4 4 4 52 x y y uw vt uw t s gi hf j lbk m d e d n Copyright John C.S. Lui 86 CSC5420 Weak Law of Large Numbers (Cont...) Applying Chebychev's Inequality to # $ %'%&% % ) 1 0( 2 3% %'%&% 4 5 6 2 7 @98 A 1 2 ! ! 7 B , we get " the distribution of arithmetic mean becomes increasingly concentrated around the mean as n grows! can be thought of as the error in approximating by the arithmetic mean G E HFDC PI Q R S'&S S S U G T V S SW S '&S X Y ` V a b or Copyright John C.S. Lui for any small , as n grows, the error will be less than with probability 1 87 f e HFdc hg i p q'&q q q s f r t q qu q '&q v w x t y weak law of large numbers Copyright John C.S. Lui ED I G HF $ " %#! '& a P b` a c Q R ST V WU X I P 9 Q Y B @ A 3 4 5 67 8 ( ( ( 10)( ( ( ( 10)( 2 $ Strong Law of Large Numbers CSC5420 88 C CSC5420 Central Limit Theorem where Xi are i.i.d. with mean X and variance X 2 PDF of Zn tends to the standard normal, i.e., x is real " ' % )0 1 # &%$ A where (Gaussian or normal distribution) 9@ D E BC H9 G H PQI F 89 ( 4 52 3 ! 6 7 Copyright John C.S. Lui 8 ...
View Full Document

Ask a homework question - tutors are online