APME_3 - Applied Probability Methods for Engineers Slide...

Info iconThis preview shows pages 1–13. Sign up to view the full content.

View Full Document Right Arrow Icon
Applied Probability Methods for Engineers Slide Set 3
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Chapter 12 Simple Linear Regression and Correlation
Background image of page 2
Simple Linear Regression Used when we believe some random variable is dependent on some other factor and is a linear function of this factor We consider a random variable Y i that depends on the value of an independent variable x i Assume that an observation y i is the sum of a linear function of x i and an error term ε i , i.e., y i = β 0 + β 1 x i + ε i Generally assumed that the error terms are normally distributed (and iid) with zero mean and variance σ 2 Observations y 1 , …, y n are therefore observations of independent random variables Y i ~ N(β 0 + β 1 x i , σ 2 ) Expected value of Y i equals β 0 + β 1 x i
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Simple Linear Regression
Background image of page 4
Simple Linear Regression y is the dependent variable and x is the explanatory variable β 0 and β 1 are the slope and intercept parameters β 1 slope parameter determines how expected value of y changes as a function of x β 1 = 0 implies y and x are unrelated β 0 , β 1 , and σ 2 typically estimated from a data set Should first look at a graph of the data to make sure linearity is a reasonable assumption
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Example Data Set Factory electricity usage as a function of production
Background image of page 6
Example Data Set
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Fitting a Regression Line How do we take a data set and fit the best line to it? Consider vertical deviations
Background image of page 8
Fitting a Regression Line Given the data set of x i , y i values, we want the β 0 and β 1 values that minimize the sum of squared deviations from the line The vertical error is given by ε i = y i – (β 0 + β 1 x i ) Why not minimize the sum of the errors? Min Σ i ε i = Min Σ i {y i – (β 0 + β 1 x i )} = -Max Σ i 0 + β 1 x i } Since we don’t have sign restrictions on β 0 and β 1 , optimal solution is β 0 = β 1 = (assuming x i ’s nonnegative) This is a meaningless result that essentially interprets negative errors as a good thing
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Fitting a Regression Line We need to consider negative and positive errors as equally bad Min Σ i ε i 2 = Min Q = Σ i {y i – (β 0 + β 1 x i )} 2 Setting these to zero gives ( 29 ( 29 0 1 1 0 2 n i i i Q y x β = = - - + ( 29 ( 29 0 1 1 1 2 n i i i i Q x y x = = - - + 0 1 1 1 n n i i i i y n x = = = + 2 0 1 1 1 1 n n n i i i i i i i x y x x = = = = +
Background image of page 10
Side Note Is Q a convex function of β 0 and β 1 ? Jointly convex in β 0 and β 1 if a ≥ 0, c ≥ 0, and ac – b 2 ≥ 0 2 2 0 2 Q a n β = = 2 2 2 1 1 2 n i i Q c x = = = 2 1 0 1 2 n i i Q b x = = = 2 1 1 0 2 n i i Q b x β β = = = ( 29 ( 29 2 2 2 1 1 1 4 4 4 n n n i i i i i i ac b n x x n x x = = = - = - = -
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Back to the Fitted Line Solve the two equations simultaneously: 0 1 1 1 n n i i i i y n x β = = = + 2 0 1 1 1 1 n n n i i i i i i i x y x x = = = = + 1 1 0 1 1 ˆ ˆ ˆ n n i i i i y x y x n n = = = - = - ( 29 ( 29 ( 29 1 1 1 1 2 2 1 1 ˆ n n n i i i i i i i n n i i i i n x y x y n x x = = = = = - = -
Background image of page 12
Image of page 13
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 62

APME_3 - Applied Probability Methods for Engineers Slide...

This preview shows document pages 1 - 13. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online