241: AMS Bayesian Nonparametric Methods (Winter 2009)
Summary and references
Instructor: Athanasios Kottas, Department of Applied Mathematics and Statistics, University of California, Santa Cruz Notes 3: Dirichlet process mixture models Applications Outline
1. Summary and references 2. Survival analysis using Weibull DP mixtures 3. Bayesian semiparametric quantile regression 4. Curve fitting using Dirichlet process mixtures 5. Modeling for multivariate ordinal data 6. Nonparametric inference for Poisson processes 7. Modeling for stochastically ordered distributions
1. Summary and references
Dirichlet process (DP) mixture models, and their extensions, have largely dominated methodological and applied Bayesian nonparametric work, after the technology for their simulation-based model fitting was introduced. References categorized by methodological/application area include: Models for binary and ordinal data (Erkanli et al., 1993; Basu and
Mukhopadhyay, 2000; Hoff, 2005; Das and Chattopadhyay, 2004; Kottas et al., 2005)
Density estimation, mixture deconvolution, and curve fitting (West et al.,
1994; Escobar and West, 1995; Cao and West, 1996; Gasparini, 1996; Mller et u al., 1996; Ishwaran and James, 2002; Do et al., 2005; Leslie et al., 2007; Lijoi et al., 2007)
Regression modeling with structured error distributions and/or regression functions (Brunner, 1995; Lavine and Mockus, 1995; Kottas and Gelfand, 2001b;
Dunson, 2005; Kottas and Krnjaji, 2009) c
AMS 241 -- Winter 2009
Athanasios Kottas
1/73
AMS 241 -- Winter 2009
Athanasios Kottas
2/73
Summary and references
Regression models for survival/reliability data (Kuo and Mallick, 1997;
Gelfand and Kottas, 2003; Merrick et al., 2003; Hanson, 2006)
Survival analysis using Weibull DP mixtures
2. Survival analysis using Weibull DP mixtures
Bayesian nonparametric work for survival analysis has focused on prior models for cumulative hazard or hazard functions (gamma processes, extended gamma processes, Beta processes), or survival functions (Dirichlet processes, Polya trees) (see, e.g., Walker et al., 1999; Ibrahim et al., 2001)
Generalized linear, and linear mixed, models (Bush and MacEachern, 1996;
Kleinman and Ibrahim, 1998; Mukhopadhyay and Gelfand, 1997; Mller and u Rosner, 1997; Quintana, 1998)
Errors-in-variables models (Mller and Roeder, 1997); Multiple comparisons u problems (Gopalan and Berry, 1998); Analysis of selection models (Lee and
Berger, 1999)
Dirichlet process mixture models? f (t; G) = k(t; )dG(), t R+ , G DP(, G0 )
Meta-analysis and nonparametric ANOVA models (Mallick and Walker, 1997;
Tomlinson and Escobar, 1999; Burr et al., 2003; De Iorio et al., 2004; Mller et u al., 2004; Mller et al., 2005) u
Time series modeling and econometrics applications (Mller et al., 1997; u
Chib and Hamilton, 2002; Hirano, 2002; Hasegawa and Kozumi, 2003; Griffin and Steel, 2004)
kernel k(t; ) that yields mixtures with flexible density (and hazard) shapes is needed Mixtures of Weibull or gamma distributions (Kottas, 2006b; Hanson, 2006)
ROC data analysis (Erkanli et al., 2006; Hanson et al., 2008)
3/73 4/73
AMS 241 -- Winter 2009
Athanasios Kottas
AMS 241 -- Winter 2009
Athanasios Kottas
Survival analysis using Weibull DP mixtures
Survival analysis using Weibull DP mixtures
Weibull Dirichlet process mixture model
ind.
Data Illustrations
Simulated data (n = 200) from a mixture of two Lognormal distributions 0.8LN (0, 0.25) + 0.2LN (1.2, 0.02) bimodal density non-monotone hazard function with 3 change points in the interval (0, 5) where essentially all the probability mass lies
ti | (i , i ) (i , i ) | G G | , , , ,
K(t | i , i ) = 1 - exp(-ti /i ), i = 1, ..., n G, i = 1, ..., n DP(, G0 ); G0 = U ( | 0, )IG( | c, ) p()p()p()
i.i.d.
full posterior inference for all functionals of interest in survival analysis, including non-linear functionals (e.g., hazard function, and median survival time) -- uses sampling from the posterior of G also, the prior distribution of functionals can be sampled (quantifies prior to posterior learning)
Remission times (in weeks) for leukemia patients (Lawless, 1982) comparison of two treatments, A and B, each with 20 patients (3 and 2 right censored survival times, respectively) "no evidence of a difference in distributions" based on classical tests that rely on approximate normality and assume proportional hazard functions (Lawless, 1982)
AMS 241 -- Winter 2009
Athanasios Kottas
5/73
AMS 241 -- Winter 2009
Athanasios Kottas
6/73
Survival analysis using Weibull DP mixtures
Prior and posterior for v
0.20 0.6
Survival analysis using Weibull DP mixtures
Gamma(2,0.9) prior for v
1.0
Gamma(2,0.1) prior for v
1.0
Gamma(3,0.05) prior for v
1.0
Posterior for n*
0.8
0.8
0.4
0.10
0.6
0.6
0.2
0.00
0.0
0.4
0.4
0
2
4
6
8
10
12
0
5
10
15
20
25
30
0.2 0.2 0.2 0 1 2 3 4 5 0.0 0
Gamma(2,0.9) prior for v
Gamma(2,0.9) prior for v
0.0
0
1
2
3
4
5
0.0
0.4
0.6
0.8
1
2
3
4
5
0.4
0.2
0.6
0.6
Gamma(2,0.1) prior for v
Gamma(2,0.1) prior for v
0.4 0.4 0.4 0 1 2 3 4 5 0.0 0 0.2
0.20
0.6
0.2
0.4
0.2
0.10
0.0
0.00
0.0
0
1
2
3
4
5
0.0
0.2
0.6
0
2
4
6
8
10
12
0.00 0
0.0
0.10
0.20
0.6
5
10
15
20
25
30
1
2
3
4
5
0
2
4
6
8
10
12
0
5
10
15
20
25
30
Gamma(3,0.05) prior for v
Gamma(3,0.05) prior for v
Figure 1: Simulated data. Histograms of posterior draws for (denoted by v in the panels) and n , under three prior choices for . The prior densities for are denoted by the solid lines.
Figure 2: Simulated data. Inference, under three prior choices for . The upper panels provide prior (dotted lines) and posterior (dashed lines) point and interval estimates for the survival function. The lower panels include the histogram of the data along with the posterior point estimate (dashed line) for the density function. In each graph, the solid line denotes the true curve.
AMS 241 -- Winter 2009
Athanasios Kottas
7/73
AMS 241 -- Winter 2009
Athanasios Kottas
8/73
Survival analysis using Weibull DP mixtures
Survival analysis using Weibull DP mixtures
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0 10 20 30 (a) 40 50 60 0
6
4
2.5
3.0
5
3.5
10
20
30 (b)
40
50
60
3
2
1.5
2.0
0.00 0.01 0.02 0.03 0.04
1.0
1
0.5
0.0
0
0
1
2
3
4
5
0
1
2
3
4
5
0
10
20
30 (c)
40
50
60...
notes-3_4