Provide raw data in the appendices or to describe

This preview shows page 227 - 229 out of 394 pages.

provide raw data in the appendices or to describe where the data can be acquired, e.g., from a website. Additional processing (or preparation) of the data set may be required. Such preparations should be discussed here. This includes, if appropriate, data transfor- mation, outlier identification and their potential removal, and handling of missing values, as well as the discussion of dropouts (i.e., data from participants who were not present for all experimental sessions). Chap. 7 details methods for dealing with missing values. For hypothesis testing, special emphasis should be placed on how the data was evaluated (e.g., by an ANOVA) and how the analysis model was validated. The violations of the statistical assumptions underlying the analysis method (e.g., normality, independence, and residuals) should also be described. The values of the resulting statistics also need to be reported. Harris outlines what has to be reported for different kinds of statistical tests (Harris, 2002). Singer (1999) recommends that “inferential statistics are reported with the value of the test (effect size), the proba- bility level, the degrees of freedom, the direction of effect,” and the power of the test. To this list, we add the alpha value and the confidence interval where appropriate (Dybå et al., 2006; Kampenes et al., 2007). 3.10. Discussion The purpose of the discussion section is to interpret the findings presented in the previous section. This includes an overview of the results, threats to validity, generalization (where are the results applicable?), as well as the (potential) impact on cost, time, and quality. Harris (2002) suggests starting this section with a description of what has been found and how well the data fit the predictions. Related to this, authors should discuss whether the hypotheses were confirmed or not. The discussion
222 A. Jedlitschka et al. section should include information about each of the following three elements: Evaluation of Results and Implications , Threats to Validity , and Inferences . 3.10.1. Evaluation of Results and Implications The purpose of the evaluation of results and implications is to explain the results. All findings, including any unexpected results, should be described in this subsec- tion. Moreover, if the null hypothesis was not rejected, authors may include reasons for why they believe this is the case. Several authors point out that it is important to distinguish between statistical significance and practical importance (Kitchenham et al., 2002) or meaningfulness (Harris, 2002). The results should also be related to both theory and practice. Although it is still very rare for SE experiments to develop theory, the implica- tions of the findings should be related to the larger theory being developed, and how they further explicate or illuminate that theory (see Chap. 12 for more information about theory). The results should be discussed in the light of the objectives stated in the introduction and also related to the previous work described in the back- ground section. These two together should help to build a broader theoretical foun-

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture