week66009.docx - 6.2 What Do We Mean by Experimental Design...

This preview shows page 1 - 2 out of 16 pages.

6.2 What Do We Mean by Experimental Design? Part 1 So now we're going to talk about experimental designs in program evaluation, starting with a random ized controldesign, which is probably the best known. It's really important to remind ourselves that th ese experimental designsare ones that control or restrict access to program participation. And only i n this way can you remove the problem ofselection bias. So whether a sample unit is a program participant or not is actually controlled by the program. This p roduces themost statistically robust-- the most statistically valid, and the least biased-- estimate of th e program effects. But intruth, it can be very challenging to implement these kinds of evaluations in p ractice. When you use an experimental design, you create a situation where you can make the assumption t hat programparticipants and non-participants are identical, except for participation in the program. S o you are creatingequivalence between the groups at the aggregate, because they are the same, on average, except for the program.Our language, when we talk about experimental design, is to say co ntrol group, and not comparison group, whichimplies a quasi-experimental design. This concept of equivalence between the treatment and control group or the intervention and control group is thatyou have identical composition in each group. So you have the same mixes of units in te rms of the program andoutcome-related characteristics. You might think back to some of the adjustm ents and the covariates that we did--how the distribution in the different cells when we were thinking about program effect, how these mixes within thethings that are actually giving us the average, withi n that average, how those mixes are. With equivalence, we canassume that they're identical. There is also identical predispositions towards the program and as well, to actually achieving outcom es without theprogram. Remember, there are some people who might achieve programs with or with out the program. So we canthink about a situation where we're creating identical predispositions bet ween the control group and the interventiongroup. We can also think that the two groups have identi cal experiences. So over time, things that are outside thecontrol of the program, these external influe ncers-- they happen in the same way to the units in the treatment groupand in the control group. So there are essentially two experimental designs-- the randomized control design that we're going t o focus on here,or the RCT's-- randomized experiments. Sometimes, we use field trials as the term. And then there's regressiondiscontinuity design. In a randomized controlled design, you use random assignment of units to the intervention and contr ol groups. Andthis is an explicit, transparent, probabilistic process that-- you have to strictly adhere t o it. You can't make exceptions.It is not any kind of an ad hoc or casual, haphazard, arbitrary proces s. It's clearly defined as part of the evaluationmethodology. When you do this, you can assume that t

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture