CS147-2009-Evaluation

CS147-2009-Evaluation - stanford hci group http:// cs...

Info iconThis preview shows pages 1–7. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: stanford hci group http:// cs 147.stanford.edu Scott Klemmer Autumn 2009 Evaluating Designs How can we measure success? How do we know? Poor tests yield poor results Issues user sample statistical significance “newbie” effect / learning effects Source: PC World If you read a bit more carefully into the study, you'll notice that the study is about initial adoption of the iPhone keyboard compared to users' current phones. Also, it isn't a survey, it was a study with one on one interviews where users typed and were timed. The multitap (Non-QWERTY) users did the same or better with the iPhone than their current method, which suggests that multitappers may have an easier time adopting the iPhone's keyboard than QWERTY users. Which to me is interesting. The study does not at any time attempt to say that QWERTY users will be twice as slow on the iPhone for as long as they use the iPhone, but it does say they may have more difficulty than multitap users initially. Which to me is interesting. It would be interesting to see ia study some expert iPhone texters and have them switch to a QWERTY phone to see if there is a similar difference in typing efficiency. Why do User Testing? Can ’ t tell how good UI is until? people use it! Other methods are based on evaluators who may know too much may not know enough (about tasks, etc.) Hard to predict what real users will do Different claims, different methods This idea/system/method is innovative – analysis of prior work/competitors – design alternatives & rationale may solve a known problem – analysis of the problem, its context – formative technique, e.g., concept validation, case study, or (gulp) think-aloud usability study is better than another idea/system/method – summative empirical or analytic technique, e.g., controlled lab experiment or quasi- experimental field study If you don’t like the method, don’t make the claim Taxonomy of Methods McGrath et al. 1994 Interview transcripts – kind of like a judgment study? Yeah. The stimulus is the software, and the conditions under which you are measuring the interactions are controlled for (i.e. the interviewer is the same, etc.) But it is under which you are measuring the interactions are controlled for (i....
View Full Document

This note was uploaded on 01/12/2010 for the course CS 147 at Stanford.

Page1 / 25

CS147-2009-Evaluation - stanford hci group http:// cs...

This preview shows document pages 1 - 7. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online