Regression
DOCR Logistic Course November 5, 2008
Curtis A. Parvin, Ph.D. Department of Pathology & Immunology Phone: [phone] email: [e-mail]
Regression
Relate one or more independent (predictor) variables to a dependent (outcome) variable
Ordinary linear regression
Continuous outcome variable Determine the relationship between a continuous outcome variable and the predictor variable(s)
Logistic regression
Binary outcome variable Determine the relationship between the probability of the outcome occurring and the predictor variable(s)
Other Flavors of Logistic Regression
Conditional Logistic Regression
Matched pairs data (1:1, 1:k, k1:k2 matching)
Ordinal Logistic Regression
More than two ordered groups for outcome
Multinomial Logistic Regression
More than two unordered groups for outcome
Example: Relationship between gestational age at birth and whether an infant is breast feeding at time of hospital discharge
GA 28 29 30 31 32 33 No 4 3 2 2 4 1 Yes 2 2 7 7 16 14 Total 6 5 9 9 20 15 P .33 .40 .78 .78 .80 .93
Ordinary Linear Regression
Logistic Regression
How do we get an "S-shaped" curve?
Rather than using probability as our outcome variable, we use a transformation that is a function of probability We choose our transformation so that it ranges between (-,) as probability ranges between (0,1)
We will use the logit transform Fitting a straight line using the logit transform as the outcome variable is called logistic regression
After we estimate the straight line we can transform back to get our S-shaped curve
Probability, Odds, and the Logit Transform
Probability, P, ranges between 0 and 1 Define Odds = P/(1-P)
Odds range between 0 and Note: P = Odds/(1+Odds)
The Logit transform is the logarithm of the Odds
Logit = log(Odds) = log[P/(1-P)] Logit ranges between - and Note: Odds = eLogit Note: P = eLogit/(1+eLogit)
Odds and Logit for Breast Feeding Example
GA 28
29 30 31
No 4
3 2 2
Yes 2
2 7 7
P .33
.40 .78 .78
Odds .50
.67 3.5 3.5
Logit -.69
-.41 1.25 1.25
32
33
4
1
16
14
.80
.93
4.0
14.0
1.39
2.64
Log(Odds) = -16.72 + 0.577*GA
Logistic Regression
Model the logarithm of the odds of an outcome as a linear combination of predictor variables Logit = log(Odds) = a+bX+cY+. . . Estimate the coefficients a, b, c based on a random sample of subjects' data Determine which of the predictors are "good" Assess model fit Use the model to predict future cases
Logistic Regression Coefficients
For a single predictor variable, logistic regression fits a straight line to the log of the Odds
log(Odds) = a + bX b is the slope coefficient for X Each 1 unit change in X, changes the log(Odds) by b units
Logistic Regression Coefficients
b = log[Odds(X+1)] log[Odds(X)] Note: log(A) log(B) = log(A/B) b = log[Odds(X+1)/Odds(X)] Note: Odds(X+1)/Odds(X) is called an Odds ratio
Odds and Odds Ratios
Odds defines the probability that an event occurs divided by the probability that the event doesn't occur An Odds ratio is the ratio of two Odds
An Odds ratio could represent the ratio of the odds in two different groups An Odds ratio could represent the ratio of the odds at two different values for a risk variable
Breast Feeding Example
GA 28 29 30 No 4 3 2 Yes 2 2 7 P .33 .40 .78 Odds .50 .67 3.5 Logit -.69 -.41 1.25
31
32 33
2
4 1
7
16 14
.78
.80 .93
3.5
4.0 14.0
1.25
1.39 2.64
The Odds ratio for breast feeding at hospital discharge for GA=32 compared to GA=28 is 4.0/0.5 = 8.0
Logistic Regression Coefficients and Odds Ratios
b = log[Odds(X+1)/Odds(X)] b estimates the log of the Odds ratio associated with a 1 unit increase in X eb estimates the the odds ratio for a 1 unit increase in X For the breast feeding example
log(Odds) = -16.72 + 0.577*GA the odds of breast feeding at hospital discharge increase by a factor of e0.577 = 1.78 for each additional week of GA
Logistic Regression Odds Ratios
GA 28 29 30 31 32 33 No 4 3 2 2 4 1 Yes 2 2 7 7 16 14 Odds .50 .67 3.5 3.5 4.0 14.0 OR 1 1.33 7.0 7.0 8.0 28.0 Logistic OR 1 1.78 3.17 5.64 10.05 17.89
Logistic Regression When There is Only One Binary Predictor
This situation can be handled as a classic case-control study
Disease Cases Controls Risk Yes a b Factor No c d Odds Ratio (OR)= a/c = a/b = ad b/d c/d bc
Example: CHD and Age (Dichotomized at 55 Years)
CHD
Present
55 years <55 years 21 22
Absent
6 51
Total
27 73
Total
43
57
100
2X2 table calculation: OR = (21/22)/(6/51) = 8.11
Example: CHD and Age (Dichotomized at 55 Years)
CHD
Present
55 years <55 years 21 22
Absent
6 51
Total
27 73
Total
43
57
100
Logistic Regression: log(Odds) = -0.841 + 2.094 * Age OR = exp(2.094) = 8.11 Logistic regression produces the exact same Odds ratio estimate as the 2X2 table calculation
The Real Strength of Logistic Regression is When There are Multiple Predictor Variables
The independent variables (predictors, risk factors) can be categorical or continuous Example: TDx-FLM II and gestational age as predictors of risk for respiratory distress syndrome (RDS)
TDx-FLM II measures mg surfactant/g of albumin in amniotic fluid
The Data (some of it)
TDxFLM 75 7 14.8 18.3 27 22 29 135 4 15 16.5 25 44.2 35.5 41 48 49 55.8 59 59 GA 30 31 31 31 31 31 31 31 32 32 32 32 32 32 32 32 32 32 32 32 RDS 0 1 1 1 1 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0
Logistic Regression Parameter Estimates
-----------------------------------------------------------------------------rds | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------tdxflm | -.1121656 .0163848 0.000 -7.11 -.1442792 -.0800520 ga | -.3661113 .1192559 -2.58 0.010 -.5998486 -.1323740 _cons | 15.68597 4.322678 3.63 0.000 7.213680 24.15827 ------------------------------------------------------------------------------
log(Odds) = 15.69 - 0.112*TDxFLM - 0.366*GA
Odds Ratio for a 1 g/mg increase in TDxFLM: e-0.112 = 0.894 Odds Ratio for a 1 week increase in GA: e-0.366 = 0.693
-----------------------------------------------------------------------------rds | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------tdxflm | .893896 .0154269 -7.11 0.000 .8636601 .9241324 ga | .6934256 .0871025 -2.58 0.010 .5227078 .8641434 ------------------------------------------------------------------------------
Using the Logistic Model to Predict Risk of RDS
We can use the logistic model equation to;
Identify variables that are significant predictors calculate the absolute risk (probability) of RDS (may give biased estimates) calculate the relative risk (odds ratio) of RDS develop a classifier for diagnosing RDS
Logistic Regression Parameter Estimates
Significant coefficients mean significantly different from zero
-----------------------------------------------------------------------------rds | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------tdxflm | -.1121656 .0163848 -7.11 0.000 -.1442792 -.0800520 ga | -.3661113 .1192559 -2.58 0.010 -.5998486 -.1323740 _cons | 15.68597 4.322678 3.63 0.000 7.213680 24.15827 ------------------------------------------------------------------------------
Significant Odds ratios mean significantly different from one
-----------------------------------------------------------------------------rds | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------tdxflm | .893896 .0154269 -7.11 0.000 .8636601 .9241324 ga | .6934256 .0871025 -2.58 0.010 .5227078 .8641434 ------------------------------------------------------------------------------
Absolute Risk of RDS based on TDX FLM II and gestational age (for RDS prevalence of 8.5%)
Gestational Age (weeks)
FLM 10 20 30 40 29 98% 94% 85% 64% 30 97% 92% 79% 55% 31 96% 89% 73% 46% 32 95% 85% 65% 37% 33 92% 80% 56% 29% 34 89% 73% 47% 22% 35 85% 65% 38% 17% 36 80% 57% 30% 12% 37 73% 47% 23% 8.7% 38% 17% 6.2% 12% 4.4% 38 39
50
60 70 80 90 100
37%
16% 5.8% 2.0%
29%
12% 4.1% 1.4% 0.45%
22%
8.4% 2.9% 0.96% 0.31% 0.10%
16%
6.0% 2.0% 0.67% 0.22% 0.07%
12%
4.2% 1.4% 0.46% 0.15% 0.05%
8.5%
3.0% 0.98% 0.32% 0.11% 0.03%
6.1%
2.1% 0.68% 0.22% 0.07% 0.02%
4.3%
1.4% 0.47% 0.16% 0.05% 0.02%
3.0%
1.0% 0.33% 0.11% 0.04% 0.01%
2.1%
0.70% 0.23% 0.07% 0.02% <0.01%
1.5%
0.49% 0.16% 0.05% 0.02% <0.01%
Risk
e(15.69 0.112*TDxFLM 0.366*GA) 1 e(15.69 0.112*TDxFLM 0.366*GA)
Odds ratios for RDS relative to a TDX FLM II ratio of 70 mg/g at 37 weeks gestational age
Gestational Age (weeks) FLM 29 30 >1000 >1000 541 >1000 375 31 >1000 799 260 32 >1000 554 180 33 34 35 36 37 38 189 61.6 20.1 42.7 13.9 39
10
20 30 40
>1000
>1000 384 125
>1000
818 266 86.8
>1000
567 185 60.2
>1000
393 128 41.7
837
273 88.8 28.9
50
60 70 80
176
57.4 18.7 6.09
122
39.8 13.0 4.23
84.8
27.6 9.00 2.93
58.8
19.1 6.24 2.03
40.8
13.3 4...