Register now to access 7 million high quality study materials (What's Course Hero?) Course Hero is the premier provider of high quality online educational resources. With millions of study documents, online tutors, digital flashcards and free courseware, Course Hero is helping students learn more efficiently and effectively. Whether you're interested in exploring new subjects or mastering key topics for your next exam, Course Hero has the tools you need to achieve your goals.

8 Pages

lecture-28

Course: STAT 36-754, Spring 2006
School: Michigan
Rating:

Word Count: 2480

Document Preview

28 Shannon Chapter Entropy and Kullback-Leibler Divergence Section 28.1 introduces Shannon entropy and its most basic properties, including the way it measures how close a random variable is to being uniformly distributed. Section 28.2 describes relative entropy, or Kullback-Leibler divergence, which measures the discrepancy between two probability distributions, and from which Shannon entropy can be...

Register Now

Unformatted Document Excerpt

Coursehero >> Michigan >> Michigan >> STAT 36-754

Course Hero has millions of student submitted documents similar to the one
below including study guides, practice problems, reference materials, practice exams, textbook help and tutor support.

Course Hero has millions of student submitted documents similar to the one below including study guides, practice problems, reference materials, practice exams, textbook help and tutor support.
Find millions of documents on Course Hero - Study Guides, Lecture Notes, Reference Materials, Practice Exams and more. Course Hero has millions of course specific materials providing students with the best way to expand their education.

Below is a small sample set of documents:

Michigan - STAT - 36-754
Chapter 29Entropy Rates andAsymptotic EquipartitionSection 29.1 introduces the entropy rate the asymptotic entropy per time-step of a stochastic process and shows that it iswell-dened; and similarly for information, divergence, etc. rates.Section 29.
Michigan - STAT - 36-754
Chapter 30General Theory of LargeDeviationsA family of random variables follows the large deviations principle if the probability of the variables falling into bad sets, representing large deviations from expectations, declines exponentially insome ap
Michigan - STAT - 36-754
Chapter 31Large Deviations for I IDSequences: The Return ofRelative EntropySection 31.1 introduces the exponential version of the Markov inequality, which will be our ma jor calculating device, and shows howit naturally leads to both the cumulant gen
Michigan - STAT - 36-754
Chapter 32Large Deviations forMarkov SequencesThis chapter establishes large deviations principles for Markovsequences as natural consequences of the large deviations principlesfor IID sequences in Chapter 31. (LDPs for continuous-time Markovprocess
Michigan - STAT - 36-754
Chapter 34Large Deviations forWeakly Dep endentSequences: TheGrtner-Ellis TheoremaThis chapter proves the Grtner-Ellis theorem, establishing anaLDP for not-too-dependent processes taking values in topologicalvector spaces. Most of our earlier LDP
Michigan - STAT - 36-754
Chapter 35Large Deviations forStochastic DierentialEquationsThis last chapter revisits large deviations for stochastic dierential equations in the small-noise limit, rst raised in Chapter 22.Section 35.1 establishes the LDP for the Wiener process (Sc
Michigan - STAT - 36-754
BibliographyAbramowitz, Milton and Irene A. Stegun (eds.) (1964). Handbook of Mathematical Functions . Washington, D.C.: National Bureau of Standards. URLhttp:/www.math.sfu.ca/cbm/aands/.Algoet, Paul (1992). Universal Schemes for Prediction, Gambling a
Michigan - STAT - 36-754
Solution to Homework #1, 36-75427 January 2006Exercise 1.1 (The product -eld answers countable questions)Let D = S X S , where the union ranges over all countable subsets S of the index set T . For any event D D, whether or not asample path x D depend
Michigan - STAT - 36-754
Solution to Homework #2, 36-7547 February 2006Exercise 5.3 (The Logistic Map as a MeasurePreserving Transformation)The logistic map with a = 4 is a measure-preserving transformation, and the measure it preserves has the density 1/ x (1 x)(on the unit
Michigan - STAT - 36-754
Solution to Homework #3, 36-75425 February 2006Exercise 10.1I need one last revision of the denition of a Markov operator: a linear operatoron L1 satisfying the following conditions.1. If f 0 (-a.e.), then Kf 0 (-a.e.).2. If f M (-a.e.), then Kf M (
Michigan - STAT - 36-754
Syllabus for Advanced Probability II,Stochastic Processes36-754Cosma ShaliziSpring 2006This course is an advanced treatment of interdependent random variablesand random functions, with twin emphases on extending the limit theoremsof probability fro
George Mason - STAT - 344
Introduction to Engineering StatisticsLecture 02 TopicsCollecting engineering dataMechanistic and empirical modelsProbability and probability modelsLecture 02 Reference:Montgomery: Sec 1.2 through 1.41Basic Types of StudiesThree basic methods for
George Mason - STAT - 344
Probability ALecture 03 TopicsRandom experimentsSample spacesEventsCounting techniquesLecture 03 Reference:Montgomery: Sec 2.112ProbabilityCHAPTER OUTLINE2-1 Sample Spaces &amp; Events2-1.1 Random Experiments2-1.2 Sample Spaces2-1.3 Events2-1.
George Mason - STAT - 344
Probability BLecture 04 TopicsEqually likely outcomesProbability rulesUnions, intersections &amp; complementsSet operationsConditional probabilities in treesLecture 04 Reference:Montgomery:Sec 2.2 Axioms of ProbabilitySec 2.3 Addition rulesSec 2.4
George Mason - STAT - 344
Probability CLecture 05 TopicsMultiplication ruleTotal probability ruleIndependence of eventsReliabilityBayes TheoremRandom variablesLecture 05 Reference:Montgomery:Sec 2.5Sec 2.6Sec 2.7Sec 2.8Multiplication, total probability rulesIndepend
George Mason - STAT - 344
Discrete Probability ALecture 06 TopicsDiscrete random variables, defined &amp; graphedCumulative distribution functions, defined &amp;graphedMean and variance of a discrete random variableDefined mathematicallyGraphically explainedLecture 06 Reference:M
George Mason - STAT - 344
Discrete Probability BLecture 07 TopicsFor each of these distributions, we will examine the:Graph and parametersProbability mass and cumulative distribution functionsMean and varianceUniform distributionBinomial distribution:Negative binomial dist
George Mason - STAT - 344
Discrete Probability CLecture 08 TopicsFor each of these distributions, we will examine the:Graph and parametersProbability mass and cumulative distribution functionsMean and varianceHypergeometric distributionPoisson distributionLecture 08 Refere
George Mason - STAT - 344
Probability &amp; Statistics forEngineers/Scientists ILecture 01 TopicsIntroduction to the Syllabus, Assignment SheetBlackboard for course materials, lecture notesIntroduction to the instructorBasic ideas in statisticsIllustration of computer tools RL
George Mason - STAT - 344
Continuous Probability ALecture 09 TopicsContinuous variable distribution propertiesPDF &amp; CDF functions and graphsDerivation of the mean and varianceDesign and uses of the uniform distributionLecture 09 Reference:Montgomery:Sec 4.1Sec 4.2Sec 4.3
George Mason - STAT - 344
Continuous Probability BLecture 10 TopicsNormal distribution graphs and parametersStandard normal calculation, table and softwareApproximating discrete distributions with the normalExponential distributionFormula, graphs and parameterApplicationsL
George Mason - STAT - 344
Continuous Probability CLecture 11 TopicsBuilding on the exponential distribution of prior lectureMotivation, formula, graph, parameters andapplications of the:Erlang distribution and its extension, the gamma distributionWeibull distributionLognorm
George Mason - STAT - 344
Joint Probability Distributions ALecture 12 TopicsBuilding on the exponential distribution of prior lectureMotivation, formula, graph, parameters andapplications of the:Erlang distribution and its extension, the gamma distributionWeibull distributio
George Mason - STAT - 344
Joint Probability Distributions BLecture 13 TopicsPairwise independent random variablesRectangular ranges are necessary, but not sufficientFinding these probability distributions (&gt; 2 dimensions)Joint, marginal and conditional distributionsIndepende
George Mason - STAT - 344
Joint Probability Distributions CLecture 14 TopicsDiscrete multinomial distributionContinuous bivariate normal distributionIndependentDependent (covariance &amp; correlation)Reproductive propertyLinear combinations of random variablesSums and averages
George Mason - STAT - 344
General Bivariate Continuous DistributionsThis continuous variable example illustrates1) Finding the marginal and conditional for the two variables andcorresponding expected values, variances, and standarddeviations.2) Finding general conditional dis
George Mason - STAT - 344
Bivariate Discrete DistributionsLet X and Y be two discrete random variables defined on a samplespace S of an experiment.The joint probability mass function p(x, y) is defined for each pair ofnumbers (x, y) byIn this class the pairs of numbers can be
George Mason - STAT - 344
Gamma DistributionThe gamma distribution with parameters r and can be thought of asthe waiting time for r Poisson events when r is integer. The parameteris the expected number of Poisson events per a unit time interval. Ifincrease the typical wait for
George Mason - STAT - 344
Review:MarginalandConditionalDistributionsandCovarianceforContinuousDistributionsManytopicsinthetextbeginwithgeneralcaseexamplesandthencallattentiontofamiliesofdistribution,especiallythenormalfamily.Thefollowingusesapolynomialdensityfortworandomvariabl
George Mason - STAT - 344
Midterm 2 Overview by ChapterChapter 4 Continuous distributionsFamilies: Identification, domains, expected value variance: See SummaryProbability problems:R script: Normal Distribution, Exponential Distribution, Gamma DistributionHand integration: Si
George Mason - STAT - 344
1. Probability Density Functions from Chapter 4.In the Midterm exam, some density functions will be provided. You may be asked to fill in anyof the additional information: the family names, the domain possible values, and the expectedvalue and variance
George Mason - STAT - 344
Analysis of Paired DataThe Paired t TestThe sample consists of n independently selected items for which a pairof observations is made.We can compute the difference for each pairs and make inferencesabout the mean of these differences using a one samp
George Mason - STAT - 344
Data Type, Population Parameters and R Functionsfor Hypothesis Test and Confidence IntervalsSingle Population InferenceDataParameterR functionCount or fractionProportion pbinom.testof n itemsin class of interestContinuousMean t.testPaired co
George Mason - STAT - 344
Inference about a Difference BetweenPopulation ProportionsExample problem:Olestra was a fat substitute used in some snack foods.After some people consuming such snacks reported gastrointestinalproblems an experiment was performed.Results:90 of 563
George Mason - STAT - 344
Interpreting R Hypothesis Test and Confidence Interval OutputProblems are worth .5 points each. There are 50 problems.Directions: Most answers are very short. Round many digits answers to 2 significant digits.Write neatly giving the problem number and
George Mason - STAT - 344
Interpreting R Hypothesis Test OutputIn writing numeric values for answers, round to 3 significant digits.1.Exact binomial testdata: 12 and 24number of successes = 12, number of trials = 24, p-value = 0.03139alternative hypothesis: true probability
George Mason - STAT - 344
George Mason - STAT - 344
R Inputx = c( 25.8, 36.6, 26.3, 21.8, 27.2)t.test( x, alternative=&quot;greater&quot;, mu=25, conf.level=.95)R OutputOne Sample t-testdata: xt = 1.0382, df = 4, p-value = 0.1789alternative hypothesis: true mean is greater than 2595 percent confidence interv
George Mason - STAT - 344
Concepts of Point EstimationLecture 18 (former 17) Topics Basic properties of a confidence interval Large-sample confidence intervalsPopulation mean for measurement dataPopulation proportion for categorical data Bootstrap confidence intervals ignore
George Mason - STAT - 344
Confidence IntervalsLecture 20 TopicsVariancesProportionsPrediction intervalsLecture 19 Reference:Montgomery Sections 9-1 thru 9-3Devore Lecture 20Devore Lecture 211Hypothesis and Test ProceduresLecture 20 TopicsHypothesis tests versus confide
George Mason - STAT - 344
Risks and P-ValuesLecture 21and 22 TopicsType II errors risksP-ValuesLecture 21 Reference:Montgomery Sections 9-4, 9-1Excel WSReviewedStat 344 Lecture 221 RisksGo to file: Stat 344 Lecture 21 WSconcerning the interaction of theseinterrelate
George Mason - STAT - 344
dcfeae7461006edd771c0bf8ba9d38963497f08b.xlsDr. SimsIllustration of Defined Alternative HypothesisInput DataH0: =75H1: ==n==7491000.01Output DataIntermediate Calcs7070.470.871.271.67272.472.873.273.67474.474.875.275.67676.4
George Mason - STAT - 344
Two-Sample t-test proceduresTwo-sample t-test procedures enable inference about the difference ofmeans for two populations,Samples from the two populations denoted 1 and 2 are stored invectors called x and y for convenience.The procedures make use of
George Mason - STAT - 344
Tests concerning a population mean.The mean of a random sample from a population provides afoundation for creating a test statistic to assesses hypothesis about apopulation mean.Case 1. The population is from the normal family with meanThe standard d
George Mason - STAT - 344
Tests concerning a Population ProportionBackground: Large Sample TestsCommon large sample test statistics have form Z =.is the estimator for the population parameter of interest.is the expected value under the Null Hypothesis.is standard deviation o
George Mason - STAT - 344
Quiz1Scope ThisisaclosedbookandnotesquizrelatedtoChapter1and associatedRscripts. Thescopeisgivenbelow. Hopefullymanywillgetaperfectscope. 1. BeabletousewordstodescribedensityplotsasinFigure 1.11 2. Beabletowritethedefinitionsofthemeanandmedianon page25and
George Mason - MTH - 203
(J / jS O lUlIM ath 203-001 Spring 2011E xam 1Name: L astF irst( Problem 1 ) (25 points) F ind t he g eneral so lution o f t he linear s ystem (pleasewrite t he soluti on in t he v ector form) o r e xpla in w hy t he s ystem is inconsistent .- X2
George Mason - MTH - 203
~c7L ~T ()~JM a th 203-001 Spring 2011E xam 2N a rne: LastF irst( Prob le m 1) ( 18 point s) C ompute t h e fo llowi ng determin a nt s. Show s teps b u t tryt o avoid u nn ecessary c alcul at ions when possible.2o51-1 3237-644L il@68-
George Mason - MTH - 203
S OL U I) OJ\!M ath 203-001 Spring 2011E xam 3F irstName: L ast(P roblem 1 ) (25 points) For t he m atrix A =[~ ~]do t he following:(1) F ind all eigenvalues;(2) For each eigenvalue, find t he basis of t he eigenspace;(3) I f i t t urns o ut t h
Grand Canyon - FIN - 650
4/16/2010Chapter 15. Ch 15-12 Build a ModelReacher Technology has consulted with investment bankers and determined the interest rate it would payfor different capital structures, as shown below. Data for the risk-free rate, the market risk premium, an
Grand Canyon - FIN - 650
Chapter 22Qifeng (Danny) GuoP22-6McDowell Industries sells on terms of 3/10, net 30. Total sales for the year are \$912,500.Forty percent of the customers pay on the 10th day and take discounts: while the other 60%pay, on average, 40 days after their
Grand Canyon - FIN - 650
4/16/2010Chapter 13. Ch 13-11 Build a ModelThe Henley Corporation is a privately held company specializing in lawn care products and services. The most recentfinancial statements are shown below.Income Statement for the Year Ending December 31 (Millio
Grand Canyon - FIN - 650
Chapter 18Qifeng (Danny) GuoP18-1Axel Telecommunications has a target capital structure that consists of 70% debtand 30% equity. The company anticipates that its capital budget for the upcomingyear will be \$3,000,000. If Axel reports net income of \$2
Grand Canyon - FIN - 650
4/16/2010Chapter 11. Ch 11-18 Build a ModelWebmasters.com has developed a powerful new server that would be used for corporations Internet activities. It wouldcost \$10 million at Year 0 to buy the equipment necessary to manufacture the server. The proj
Grand Canyon - FIN - 650
Chapter 14Qifeng(Danny) GuoP14-1Baxter Video Products sales are expected to increase from \$5 million in 2007 to \$6million in 2008 or by 20%. Its assets totaled \$3 million at the end of 2007. Baxter is atfull capacity, so its assets must grow at the s
Grand Canyon - FIN - 650
Chapter 10Qifeng (Danny) GuoP10-2LL Incorporated's currently outstanding 11% coupon bonds have ayield to maturity of 8%. LL believes it could issue at par new bondsthat would provide a similar yield to maturity. If its marginal tax rate is35%, what
Grand Canyon - FIN - 650
Week 2 HomeworkQifeng(Danny) GuoChapter 2P2-1An investor recently purchased a corporatebond that yields 9%. The investor is in the 36%combined federal and state tax bracket. What isthe bonds after-tax yield?Yield before TaxTax RateYield after Ta
Grand Canyon - FIN - 650
Chapter 4Qifeng(Danny) GuoPVInterestYearFV\$10,00010%5\$16,105.10FVYearsInterestPV\$5,000207%\$1,292.10PMTYearsInterestFVFvdue\$30057%\$1,725.22\$1,845.99a.PVYearsInterestFV\$50016%\$530.00b.PVYearsInterestFV\$50026%\$561
Grand Canyon - FIN - 650
Chapter 1 Mini CaseQifeng (Danny) GuoAssume that you recently graduated and have just reported to work as an investmentadvisor at the brokerage firm of Balik and Kiefer Inc. One of the firms clients isMichelle DellaTorre, a professional tennis player
Punjab Engineering College - LALA - 222
Clayton VHS AP Physics B 05-06 Chapter 4 Quiz SolutionsClayton VHS AP Physics B 05-06 Chapter 4 Quiz SolutionsClayton VHS AP Physics B 05-06 Chapter 4 Quiz SolutionsClayton VHS AP Physics B 05-06 Chapter 4 Quiz SolutionsClayton VHS AP Physics B 05-06
Punjab Engineering College - LALA - 222
Clayton VHS AP Physics B 05-03 Chapter 4 Homework SolutionsClayton VHS AP Physics B 05-03 Chapter 4 Homework SolutionsClayton VHS AP Physics B 05-03 Chapter 4 Homework SolutionsClayton VHS AP Physics B 05-03 Chapter 4 Homework SolutionsClayton VHS AP