Regression2_article - 1 1.1 Logistic Regression Binary outcomes Binary outcomes Often the response is binary e.g yes or no defective or good dead

Info iconThis preview shows pages 1–7. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Logistic Regression 1.1 Binary outcomes Binary outcomes Often the response is binary, e.g., “yes” or “no” “defective’ or “good” “dead” or ”alive” often coded “0” or “1” Alternatively, the response is the number of “yes” responses in a number of “trials” Binary regression: model the conditional probability of “yes” given the pre- dictors Binary regression: data For the i th case: X i, 1 , . . . , X i,p are the predictors n i is the number of “trials” p ( X i, 1 , . . . , X i,p ) is the conditional probability of a “yes” Y i | X i, 1 , . . . , X i,p is Binomial { p ( X i, 1 , . . . , X i,p ) , n i } So Pr ( Y i = y | X i, 1 , . . . , X i,p ) = ± n i y ! p ( X i, 1 , . . . , X i,p ) y { 1 - p ( X i, 1 , . . . , X i,p ) } n i - y for y = 0 , . . . , n i
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
1.2 Example: ingots data Example: ingots data Soak Time Heat Time Not Ready n i 1 7 0 10 1 14 0 31 1 27 1 56 1 51 3 13 1.7 7 0 17 1.7 14 0 43 1.7 27 4 44 1.7 51 0 1 2.2 7 0 7 2.2 14 2 33 2.2 27 0 21 2.2 51 0 1 2.8 7 0 12 2.8 14 0 31 2.8 27 1 22 2.8 51 0 0 4 7 0 9 4 14 0 19 4 27 1 16 4 51 0 1 Y i = (number of ingots not ready) / n i n i = number of ingots prepared Let’s look at the data * * * * * * * * * * * 1.0 1.5 2.0 2.5 3.0 3.5 4.0 0.00 0.05 0.10 0.15 0.20 soaking time proportion not ready 2
Background image of page 2
* * * * * * * * * * 10 20 30 40 50 0.00 0.05 0.10 0.15 0.20 heating time proportion not ready Modeling p ( X 1 , . . . , X p ) : first attempt From previous slide: p ( X 1 , . . . , X p ) is the conditional probability of a “yes” Linear model: p ( X 1 , . . . , X p ) = β 0 + β 1 X 1 + · · · + β p X p What is wrong with this model? 1.3 The logistic regression model Logistic function -10 -5 0 5 10 0.0 0.2 0.4 0.6 0.8 1.0 x L(x) 3
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
L ( x ) = 1 1 + exp( - x ) Logistic regression model p ( X 1 , . . . , X n ) = L ( β 0 + β 1 X 1 + · · · + β p X p ) Let’s look at the simplest case, p = 1 : p ( X ) = L ( β 0 + β 1 X ) Some logistic models with one X -15 -10 -5 0 5 10 0.0 0.2 0.4 0.6 0.8 1.0 x Pr(yes|X) b0=0, b1=1 b0=0, b1=3 b0=4, b1=1 b0=0, b1=-1 Logit function p ( X 1 , . . . , X n ) = L ( β 0 + β 1 X 1 + · · · + β p X p ) 4
Background image of page 4
implies that L - 1 { p ( X 1 , . . . , X n ) } = β 0 + β 1 X 1 + · · · + β p X p L - 1 is called the “logit” function and L - 1 ( p ) = log ± p 1 - p ² Link function The “odd” for “yes” against “no” is p 1 - p So the logistic model says that the log-odds equals β 0 + β 1 X 1 + · · · β p X p The logit function is called the “link” function because it links p ( X 1 , . . . , X n ) , and β 0 + β 1 X 1 + · · · + β p X p 1.4 Fitting the ingots data R output Call: glm(formula = Y ~ soak + heat, family = binomial, data = ingots, weights = n) Deviance Residuals: Min 1Q Median 3Q Max -1.28311 -0.78183 -0.50514 -0.09701 1.71923 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -5.55917 1.11969 -4.965 6.87e-07 *** soak 0.05677 0.33121 0.171 0.863906 heat 0.08203 0.02373 3.456 0.000548 *** --- Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1 (Dispersion parameter for binomial family taken to be 1) Null deviance: 25.395 on 18 degrees of freedom Residual deviance: 13.753 on 16 degrees of freedom AIC: 34.08 Number of Fisher Scoring iterations: 5 5
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
1.5 Maximum likelihood Maximum Likelihood Estimation Let y i be the value of Y i actually observed. Then L ( β 0 , β 1 , . . . , β p ) := Pr ( Y 1 = y 1 , . . . , Y n = y n ) = = n Y i =1 n i y i p ( X i, 1 , . . . , X i,p ) y i { 1 - p ( X i, 1 , . . . , X i,p ) } n i - y i L is called the “likelihood function” Maximum likelihood estimation The maximum likelihood estimates are the values of β 0 , β 1 , . . . , β p that make L ( β 0 , β 1 , . . . , β p ) as large as possible.
Background image of page 6
Image of page 7
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 01/09/2009 for the course ORIE 312 taught by Professor D.ruppert,p.jacks during the Spring '08 term at Cornell University (Engineering School).

Page1 / 20

Regression2_article - 1 1.1 Logistic Regression Binary outcomes Binary outcomes Often the response is binary e.g yes or no defective or good dead

This preview shows document pages 1 - 7. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online