# APME_3 - Applied Probability Methods for Engineers Slide...

• Notes
• 62

This preview shows pages 1–14. Sign up to view the full content.

Applied Probability Methods for Engineers Slide Set 3

This preview has intentionally blurred sections. Sign up to view the full version.

Chapter 12 Simple Linear Regression and Correlation
Simple Linear Regression Used when we believe some random variable is dependent on some other factor and is a linear function of this factor We consider a random variable Y i that depends on the value of an independent variable x i Assume that an observation y i is the sum of a linear function of x i and an error term ε i , i.e., y i = β 0 + β 1 x i + ε i Generally assumed that the error terms are normally distributed (and iid) with zero mean and variance σ 2 Observations y 1 , …, y n are therefore observations of independent random variables Y i ~ N(β 0 + β 1 x i , σ 2 ) Expected value of Y i equals β 0 + β 1 x i

This preview has intentionally blurred sections. Sign up to view the full version.

Simple Linear Regression
Simple Linear Regression y is the dependent variable and x is the explanatory variable β 0 and β 1 are the slope and intercept parameters β 1 slope parameter determines how expected value of y changes as a function of x β 1 = 0 implies y and x are unrelated β 0 , β 1 , and σ 2 typically estimated from a data set Should first look at a graph of the data to make sure linearity is a reasonable assumption

This preview has intentionally blurred sections. Sign up to view the full version.

Example Data Set Factory electricity usage as a function of production
Example Data Set

This preview has intentionally blurred sections. Sign up to view the full version.

Fitting a Regression Line How do we take a data set and fit the best line to it? Consider vertical deviations
Fitting a Regression Line Given the data set of x i , y i values, we want the β 0 and β 1 values that minimize the sum of squared deviations from the line The vertical error is given by ε i = y i – (β 0 + β 1 x i ) Why not minimize the sum of the errors? Min Σ i ε i = Min Σ i {y i – (β 0 + β 1 x i )} = -Max Σ i 0 + β 1 x i } Since we don’t have sign restrictions on β 0 and β 1 , optimal solution is β 0 = β 1 = (assuming x i ’s nonnegative) This is a meaningless result that essentially interprets negative errors as a good thing

This preview has intentionally blurred sections. Sign up to view the full version.

Fitting a Regression Line We need to consider negative and positive errors as equally bad Min Σ i ε i 2 = Min Q = Σ i {y i – (β 0 + β 1 x i )} 2 Setting these to zero gives ( 29 ( 29 0 1 1 0 2 n i i i Q y x β β β = = - - + ( 29 ( 29 0 1 1 1 2 n i i i i Q x y x β β β = = - - + 0 1 1 1 n n i i i i y n x β β = = = + 2 0 1 1 1 1 n n n i i i i i i i x y x x β β = = = = +
Side Note Is Q a convex function of β 0 and β 1 ? Jointly convex in β 0 and β 1 if a ≥ 0, c ≥ 0, and ac – b 2 ≥ 0 2 2 0 2 Q a n β = = 2 2 2 1 1 2 n i i Q c x β = = = 2 1 0 1 2 n i i Q b x β β = = = 2 1 1 0 2 n i i Q b x β β = = = ( 29 ( 29 2 2 2 1 1 1 4 4 4 n n n i i i i i i ac b n x x n x x = = = - = - = -

This preview has intentionally blurred sections. Sign up to view the full version.

Back to the Fitted Line Solve the two equations simultaneously: 0 1 1 1 n n i i i i y n x β β = = = + 2 0 1 1 1 1 n n n i i i i i i i x y x x β β = = = = + 1 1 0 1 1 ˆ ˆ ˆ n n i i i i y x y x n n β β β = = = - = - ( 29 ( 29 ( 29 1 1 1 1 2 2 1 1 ˆ n n n i i i i i i i n n i i i i n x y x y n x x β = = = = = - = -
Some Simplifying Notation Fitted regression line: ( 29 ( 29 2 1 2 2 2 2 1 1 1 n i n n n i XX i i i i i i x S x x x nx x n = = = = = - = - = -

This preview has intentionally blurred sections. Sign up to view the full version.

This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern