Note A on Adapting Propensity Score Matching and Selection Models to Choice Based Samples
James J. Heckman and Petra E. Todd November 2004 This draft, February 16, 2006
1
Introduction
The probability of selection into a treatment, or the propensity score, plays a central role in classical selection models and in matching methods (see, e.g., Heckman (1980), Rosenbaum and Rubin (1983)). Heckman and Robb (1986) discuss how the propensity score is used di erently in matching and selection models. They also show that, given the propensity score, both matching and selection models are robust to choice based sampling, which occurs when treatment group members are over or underrepresented relative to their frequency in the population. This type of sampling design is frequently chosen in evaluation studies to reduce the costs of data collection and to get a larger number of observations on treated individuals. Given a consistent estimate of the propensity score, matching and clas2
sical selection methods are robust to choice-based sampling, because both are "control function" methods and are defined conditional on treatment and comparison group status. This note extends Heckman and Robb's (1986) analysis to consider the case where population weights are unknown so that propensity score cannot be consistently estimated. In evaluation settings, the population weights are often unknown or cannot easily estimated.1 For example, for the NSW training program studied in LaLonde (1986), the population consists of all persons eligible for the program, which was targeted at drug-addicts, ex-convicts, and welfare recipients. Few datasets
The methods of Manski and Lerman (1977), Manski (1986) for adjusting for choice-based sampling in estimating the discrete choice probabilities cannot be applied when the weights are unknown and cannot be estimated from the data.
1
3
would have the information necessary to determine whether a person is eligible for the program, so it would be di cult to estimate the population weights needed to consistently estimate propensity scores. In this note, we establish that matching and selection procedures can still be applied when the propensity score is estimated on unweighted choice based samples. The idea is very simple. To implement both matching and classical selection models, only a monotonic transformation of the propensity score is required. In choice based samples, the odds ratio of the misspecified propensity score is monotonically related to the odds ratio of the true propensity scores. Thus selection and matching procedures can identify population treatment e ects using misspecified estimates of propensity scores fit on choice based samples. 4
2
Discussion of the Proposition
Let = 1 if a person is a treatment group member; =0 if a member of the comparison group. = is a realization of . In the population generated from random sampling, the joint density is ( ) = [Pr( for = 1 | )] [Pr( = = 0 | )]1 {0 1} ( )
where is the density of the data. By Bayes's theorem, we have, letting Pr( = 1) = , ( | and ( | = 0)(1 ) = ( ) Pr( 5 =0| ) (1b) = 1) = ( ) ( =1| ) (1a)
Take the ratio of (1a) to (1b), ( | = 1) Pr( = ( | = 0) 1 Pr(
=1| ) =0| )
(2)
Assume 0 Pr( = 1 | ) 1. From knowledge of the densities of the data in the two samples, ( | = 1) and ( | = 0) one can form a scalar multiple of the ratio of the propensity score without knowing . The odds ratio is a monotonic function of the propensity score that does not require knowledge of the true sample weights. In a choicebased sample, both the numerator and denominator of the first term in (2) can be consistently estimated. This monotonic function can replace ( ) in implementing both matching and nonparametric selection models. However, estimating ( | = ) is demanding of the data 6
when is of high dimension. Instead of estimating these densities, we can substitute for the left hand side of (2) the odds ratio of the estimated conditional probabilities obtained using the choice-based sample with the wrong weights. (i.e., ignoring the fact that the data are a choice based sample). The odds ratio of the estimated probabilities is a scalar multiple of the true odds ratio. It can therefore be used instead Pr( of = 1 | ) to match or construct nonparametric control functions in selection bias models. In the choice-based sample, let ~ ( = 1 | ) be the conditional probability that = 1 and be the unconditional 6= , the true popprobability of sampling = 1, where ulation proportion. The joint density of the data from the
7
sampled population is [ ( | = 1) ][ ( | = 0)(1 )]1 = 0) 1 )
Using (1a) and (1b) to solve for ( | one may write the data density as Pr( =1| ) ( ) Pr(
= 1) and ( |
=0| ) ( ) (1 (1 )
8
~ ( and ~ (
=1| )=
Pr( ( | = 1)
=1| ) ( ) + ( | = 0)(1 ) (3a)
) (3b) Under random sampling, the right-hand sides of (3a) and (3b) are the limits to which the choice-based probabilities converge. Taking the ratio of (3a) to (3b), assuming the latter is not zero,
1 Pr( = 0 | ) ( ) 1 =0| )= ( | = 1) + ( | = 0)(1
9
one obtains ~ ( =1| ) Pr( = 1 | ) 1 (4) = ~ ( =0| ) Pr( = 0 | ) 1 Thus, one can estimate the ratio of the propensity score up to scale (the scale is the product of the two terms on the righthand side of (4)). Instead of matching or estimating semiparametric selection models using Pr( = 1 | ) (as in, for example, Ahn and Powell (1993), Heckman, Ichimura, Smith and Todd (1998), and Powell (1999)), we can instead use the odds ratio of the estimate ~ ( = 1 | ), which is monotonically related to the true Pr( = 1 | ). In the case of a logit ( ), ( ) = exp( ) (1 + exp( )), the log of this ratio becomes ~ ( =1| ) ln = ~ ~ ( =0| ) 10
where the slope coe cients are the true values and the intercept ~ 0 = 0 + ( (1 )) + ((1 ) )), where 0 is the true value. In implementing nearest neighbor matching estimators, matching on the log odds ratio gives identical estimates to matching on the (unknown) Pr( = 1 | ), because the odds ratio preserves the ranking of the neighbors. In application of either matching or classical selection bias correction methods, one must account for the usual problems of using estimated log odds ratios instead of true values.
11
For discussion related to using estimated propensity scores, see Hahn (1998) Heckman, Ichimura, Smith and Todd (1998) Ichimura, Smith and Todd (1998) Imbens, Hirano and Ridder (2003)
12
References
[1] Ahn, H. and J. Powell (1993), "Semiparametric Estimation of Censored Selection Models With Nonparametric Selection Mechanism," Journal of Econometrics, 58, 3-29. [2] Hahn, J. (1998), "On the Role of the Propensity Score in E cient Estimation of Average Treatment E ects," Econometrica, 66(2), 315-331. [3] Heckman, J., (1980), "Add...