{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

regression2_article

# regression2_article - 1 1.1 Logistic Regression Binary...

This preview shows pages 1–7. Sign up to view the full content.

1 Logistic Regression 1.1 Binary outcomes Binary outcomes Often the response is binary, e.g., “yes” or “no” “defective’ or “good” “dead” or ”alive” often coded “0” or “1” Alternatively, the response is the number of “yes” responses in a number of “trials” Binary regression: model the conditional probability of “yes” given the pre- dictors Binary regression: data For the i th case: X i, 1 , . . . , X i,p are the predictors n i is the number of “trials” p ( X i, 1 , . . . , X i,p ) is the conditional probability of a “yes” Y i | X i, 1 , . . . , X i,p is Binomial { p ( X i, 1 , . . . , X i,p ) , n i } So Pr ( Y i = y | X i, 1 , . . . , X i,p ) = n i y p ( X i, 1 , . . . , X i,p ) y { 1 - p ( X i, 1 , . . . , X i,p ) } n i - y for y = 0 , . . . , n i

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
1.2 Example: ingots data Example: ingots data Soak Time Heat Time Not Ready n i 1 7 0 10 1 14 0 31 1 27 1 56 1 51 3 13 1.7 7 0 17 1.7 14 0 43 1.7 27 4 44 1.7 51 0 1 2.2 7 0 7 2.2 14 2 33 2.2 27 0 21 2.2 51 0 1 2.8 7 0 12 2.8 14 0 31 2.8 27 1 22 2.8 51 0 0 4 7 0 9 4 14 0 19 4 27 1 16 4 51 0 1 Y i = (number of ingots not ready) / n i n i = number of ingots prepared Let’s look at the data * * * * * * * * * * * 1.0 1.5 2.0 2.5 3.0 3.5 4.0 0.00 0.05 0.10 0.15 0.20 soaking time proportion not ready 2
* * * * * * * * * * 10 20 30 40 50 0.00 0.05 0.10 0.15 0.20 heating time proportion not ready Modeling p ( X 1 , . . . , X p ) : first attempt From previous slide: p ( X 1 , . . . , X p ) is the conditional probability of a “yes” Linear model: p ( X 1 , . . . , X p ) = β 0 + β 1 X 1 + · · · + β p X p What is wrong with this model? 1.3 The logistic regression model Logistic function -10 -5 0 5 10 0.0 0.2 0.4 0.6 0.8 1.0 x L(x) 3

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
L ( x ) = 1 1 + exp( - x ) Logistic regression model p ( X 1 , . . . , X n ) = L ( β 0 + β 1 X 1 + · · · + β p X p ) Let’s look at the simplest case, p = 1 : p ( X ) = L ( β 0 + β 1 X ) Some logistic models with one X -15 -10 -5 0 5 10 0.0 0.2 0.4 0.6 0.8 1.0 x Pr(yes|X) b0=0, b1=1 b0=0, b1=3 b0=4, b1=1 b0=0, b1=-1 Logit function p ( X 1 , . . . , X n ) = L ( β 0 + β 1 X 1 + · · · + β p X p ) 4
implies that L - 1 { p ( X 1 , . . . , X n ) } = β 0 + β 1 X 1 + · · · + β p X p L - 1 is called the “logit” function and L - 1 ( p ) = log p 1 - p Link function The “odd” for “yes” against “no” is p 1 - p So the logistic model says that the log-odds equals β 0 + β 1 X 1 + · · · β p X p The logit function is called the “link” function because it links p ( X 1 , . . . , X n ) , and β 0 + β 1 X 1 + · · · + β p X p 1.4 Fitting the ingots data R output Call: glm(formula = Y ~ soak + heat, family = binomial, data = ingots, weights = n) Deviance Residuals: Min 1Q Median 3Q Max -1.28311 -0.78183 -0.50514 -0.09701 1.71923 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -5.55917 1.11969 -4.965 6.87e-07 *** soak 0.05677 0.33121 0.171 0.863906 heat 0.08203 0.02373 3.456 0.000548 *** --- Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1 (Dispersion parameter for binomial family taken to be 1) Null deviance: 25.395 on 18 degrees of freedom Residual deviance: 13.753 on 16 degrees of freedom AIC: 34.08 Number of Fisher Scoring iterations: 5 5

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
1.5 Maximum likelihood Maximum Likelihood Estimation Let y i be the value of Y i actually observed. Then L ( β 0 , β 1 , . . . , β p ) := Pr ( Y 1 = y 1 , . . . , Y n = y n ) = = n i =1 n i y i p ( X i, 1 , . . . , X i,p ) y i { 1 - p ( X i, 1 , . . . , X i,p ) } n i - y i L is called the “likelihood function” Maximum likelihood estimation The maximum likelihood estimates are the values of β 0 , β 1 , . . . , β p that make L ( β 0 , β 1 , . . . , β p ) as large as possible.
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}