100%(1)1 out of 1 people found this document helpful
This preview shows page 1 - 2 out of 16 pages.
6.2 What Do We Mean by Experimental Design? Part 1Sonow we're going to talk about experimental designs in program evaluation, starting with a randomized controldesign, which is probably the best known. It's really important to remind ourselves that these experimental designsare ones that control or restrict access to program participation. And only in this way can you remove the problem ofselection bias.So whether a sample unit is a program participant or not is actually controlled by the program. This produces themost statistically robust-- the most statistically valid, and the least biased-- estimate of the program effects. But intruth, it can be very challenging to implement these kinds of evaluations in practice.When you use an experimental design, you create a situation where you can make the assumption that programparticipants and non-participants are identical, except for participation in the program. So you are creatingequivalence between the groups at the aggregate, because they are the same, on average, except for the program.Our language, when we talk about experimental design, is to say control group, and not comparison group, whichimplies a quasi-experimental design.This concept of equivalence between the treatment and control group or the intervention and control group is thatyou have identical composition in each group. So you have the same mixes of units in terms of the program andoutcome-related characteristics. You might think back to some of the adjustments and the covariates that we did--how the distribution in the different cells when we were thinking about program effect, how these mixes within thethings that are actually giving us the average, within that average, how those mixes are. With equivalence, we canassume that they're identical.There is also identical predispositions towards the program and as well, to actually achieving outcomes without theprogram. Remember, there are some people who might achieve programs with or without the program. So we canthink about a situation where we're creating identical predispositions between the control group and the interventiongroup. We can also think that the two groups have identical experiences. So over time, things that are outside thecontrol of the program, these external influencers-- they happen in the same way to the units in the treatment groupand in the control group.So there are essentially two experimental designs-- the randomized control design that we're going to focus on here,or the RCT's-- randomized experiments. Sometimes, we use field trials as the term. And then there's regressiondiscontinuity design.In a randomized controlled design, you use random assignment of units to the intervention and control groups. Andthis is an explicit, transparent, probabilistic process that-- you have to strictly adhere to it. You can't make exceptions.It is not any kind of an ad hoc or casual, haphazard, arbitrary process. It's clearly defined as part of the evaluationmethodology. When you do this, you can assume that t