7/14/201Chapter 14INTRODUCING EVALUATION1Agenda•Discuss key evaluation concepts.•Introduce different evaluation methods.•Show how different evaluation methods are usedfor different purposes at different stages of thedesign process and in different contexts.•Show how evaluators mix and modify methods.•Discuss some of the challenges.•Look at frameworks for evaluation.22
7/14/20Why, what, where, and whento evaluateIterative design and evaluation is a continuous process that examines:
7/14/204Evaluation case studies•A classic experimental investigation into thephysiological responses of players of acomputer game•An ethnographic study of visitors at the theRoyal Highland show in which participantsare directed and tracked using a mobilephone app•Crowdsourcing in which the opinions andreactions of volunteers (for example, from thecrowd) inform technology evaluation77Challenge and engagement ina collaborative immersive game•Physiological measures were used•Players were more engaged when playingagainst another person than when playingagainst a computer•Why was the physiological data collectednormalized?88
7/14/205Physiological data ofparticipants in a videogame9Source: Mandryk and Inkpen (2004), “The Physiological Indicators for the Evaluation of Co-located Collaborative Play,”CSCW’2004, pp 102-111. Reproduced with permission ofACMPublications.