• The first, and necessary step is to establish two equivalent groups—control and treatment—so that they can be compared and the unique effect of the treatment on the dependent variable can be ascertained. While equivalence can be established through random assignment or matching, the former is preferred to the latter. Still, in many situations ethical and practical constraints preclude random assignment. Then, matched samples must be used to establish equivalence. • The researcher must carefully evaluate the study and its context to assess threats to internal and external validity. If internal validity is reasonably high, then a causal inference can be made. However, to the extent that external validity may be low, care should be taken to project the results beyond the context of the current experiment. • Conducting the appropriate statistical analysis should answer three questions: the existence of the impact (is the difference statistically significant?), the magnitude of the impact (how large is the impact?), and the direction of the impact (is the impact positive or negative?). The analysis takes the form of an ANOVA or analysis-of-variance for a basic study. However, variations such as repeated-measures ANOVA or multivariate- analysis-of-variance exist to accommodate more complex designs. The reader can refer to many books and sources that describe the appropriate analytical strategy for a given experiment. • Finally, taking into account all of the above issues, the researcher should assess the applicability of the results. Just because the results are “statistically significant” may not mean they are managerially relevant, useful, or applicable. In interpreting the conclusions of a given experiment it is also useful to consider the findings from previous
Experimental Research For Customer-Focused Insights 34 studies—experimental and non-experimental. Finally, due diligence should also consider industry knowledge—which can be subjective in nature but laden with insight. CONCLUDING COMMENTS Causal research has been gaining more and more acceptance as a method for customer- focused inquiry, despite its resource intensive nature. This description hopefully provides the reader with a flavor for what it entails and also strategies that should be employed in designing sound causal studies. This concludes Part 1 of the 2-part note. In Part 2 of the note I will provide an overview of the statistical analysis used in a basic customer-focused experiment. In addition, I will describe and discuss some customer-focused experiments. The descriptions and discussions can serve as cases to showcase issues related to internal and external validity, the pros and cons of choices that had to be made, and the limits to the conclusions that may be drawn from each experiment. I hope the reader can develop a better understanding of this abstract topic by reading the descriptions of these experiments.
You've reached the end of your free preview.
Want to read all 34 pages?
- Spring '17
- Lee, Janghyuk
- Sales, Causality, external validity