Relevant statistical model analysis of covariance

This preview shows page 8 - 11 out of 27 pages.

Relevant statistical model: analysis of covariance, multiple regression, structural equation modeling, repeated measures analysis of variance Variation in Program Effects In many cases, want to know how program effects vary according some variable o Subgroups of the target population o Geographic location o Implementation sites
Week 4: Impact Evaluation: Identifying and Interpreting Program Effects Need to bring in other variables to the model, not just the outcomes and covariates Moderator Variables Variables that define the subgroups that are being analyzed o Gender o Age o Ethnicity/race o SES o Urban/rural Best used not as a straight covariate, but as an interaction tern with program exposure in the regression model (MV*Program exposure y/n) Helps avoid premature conclusions about the average program effects—a supplemental analysis An effective program on average may not be so far some subgroups A program with no effects on average may have effects in some subgroups Can test expectations about differential effects—are the variations in line with theory? Probe the consistency of impact evaluation findings and strengthen conclusions Must be aware of how program participants and nonparticipants differ (biases comparisons among subgroups) A proximal outcome that changes as a result of exposure to the program, which then influences the outcome An intervening variable that comes between program participation and the outcomes of interest Is correlated with the variations in the outcome Represents steps on the causal pathway in program theory Looking at the correlation between intermediate and ultimate outcomes helps understand outcomes. Helps understand the change processes that the program activities initiated Test the hypothesized causal logic as depicted in the program theory—another dimension of consistency testing for the evaluation findings The mediating variables must be affected by the program to be considered as a causal step If there is no outcome effect, no need to worry about the mediator relationships Does variation in the mediating variable predict variation in the following outcome? (Does one step lead to the next?) Meta-Analysis Statistical. Synthesis of findings. From pervious impact evaluation effect estimates Systematic reviews can lead to meta-analysis if adequate information is available Start by finding all the relevant literature, using predetermined criteria (the systematic review part) o Type of program o Type of outcome
Week 4: Impact Evaluation: Identifying and Interpreting Program Effects o Type and size of samples o Location, date, language Then have to standardize the report effect sizes using an effect size statistic Systematically document the evaluation methods, study design, sampling strategy, eligibility for program participation, program components Then conduct a variety of statistical tests to assess mean effects, variation in effects, and factors associated with variation.

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture