•To find out why interviewees chose certain metrics or KPIs.The context, codes and metrics collected through the literature review, survey and interviews, are usedas input for the design.Evaluation is necessary to: “Observe and measure how well the artifact supports a solution to the problem”. Therefore, an evaluation of the design is mainly done for the following reasons:•To determine the extent to which certain practitioners agree with the design and feel that it couldactually help them measure the e↵ectiveness of their EA.•To find out whether the KPIs found are accepted.•To find out whether something is still missing.In Table3.1, each subquestion for answering the main research question is linked to a researchmethod.Table 3.1: Link between research questions and research methodsResearch QuestionMethodRQ1: What methods can be used to measureenterprise architecture e↵ectiveness?Literature ReviewRQ2: What metrics can be used to measure en-terprise architecture e↵ectiveness?Literature Review, Input Survey & InterviewsRQ3: What are key performance indicators forenterprise architecture e↵ectiveness?Literature Review, Input Survey & Interviews,Evaluation (Intermediate Evaluations, Evalua-tion Session, Evaluation Survey)RQ4: How are key performance indicators forenterprise architecture e↵ectiveness linked?Literature Review, Input Survey& Interviews,Evaluation (Intermediate Evaluations, Evalua-tion Session, Evaluation Survey)A Focus on KPIs12
CHAPTER 3.METHODSMeasuring EA e↵ectivenessIn the sections below, each method is discussed in more detail.3.2Literature ReviewRelevant context, existingEA e↵ectivenessmeasurement approaches, metrics, and key performance in-dicators are identified through a semi-structured literature review. An attempt is made to gain resultsfrom both academic, and “grey literature”. It is believed that, although grey literature is not alwayspeer-reviewed, by including both, an attempt is made to overcome the gap that often exists betweenresearch and practice. This is also in line with the adopted design science research approach. Moreover,EA itself is deemed inherently practical.A more structured approach is adopted while searching for academic literature using the ACM DigitalLibrary  and the IEEE Xplore Digital Library , two well-known computer science-related digitallibraries that allow for entering detailed search queries. Although they require subscription to actuallyaccess the articles, free full text versions of titles can often be found through other channels. The searchquery used in these databases can be found in AppendixB, SectionB.1.The following steps are taken:1. Search in ACM using the defined keywords.2. Save the resulting list of titles in a spreadsheet.