{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

CS147-2009-Evaluation

CS147-2009-Evaluation - stanford hci group...

Info icon This preview shows pages 1–7. Sign up to view the full content.

View Full Document Right Arrow Icon
stanford hci group http:// cs 147.stanford.edu Scott Klemmer Autumn 2009 Evaluating Designs
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
How can we measure success? How do we know?
Image of page 2
Poor tests yield poor results Issues user sample statistical significance “newbie” e ff ect / learning e ff ects Source: PC World If you read a bit more carefully into the study, you'll notice that the study is about initial adoption of the iPhone keyboard compared to users' current phones. Also, it isn't a survey, it was a study with one on one interviews where users typed and were timed. The multitap (Non-QWERTY) users did the same or better with the iPhone than their current method, which suggests that multitappers may have an easier time adopting the iPhone's keyboard than QWERTY users. Which to me is interesting. The study does not at any time attempt to say that QWERTY users will be twice as slow on the iPhone for as long as they use the iPhone, but it does say they may have more di ffi culty than multitap users initially. Which to me is interesting. It would be interesting to see ia study some expert iPhone texters and have them switch to a QWERTY phone to see if there is a similar di ff erence in typing e ffi ciency.
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Why do User Testing? Can t tell how good UI is until? people use it! Other methods are based on evaluators who may know too much may not know enough (about tasks, etc.) Hard to predict what real users will do
Image of page 4
Di ff erent claims, di ff erent methods This idea/system/method is innovative analysis of prior work/competitors design alternatives & rationale may solve a known problem analysis of the problem, its context formative technique, e.g., concept validation, case study, or (gulp) think-aloud usability study is better than another idea/system/method summative empirical or analytic technique, e.g., controlled lab experiment or quasi- experimental field study If you don’t like the method, don’t make the claim
Image of page 5

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Taxonomy of Methods McGrath et al. 1994 Interview transcripts – kind of like a judgment study? Yeah. The stimulus is the software, and the conditions under which you are measuring the interactions are controlled for (i.e. the interviewer is the same, etc.) But it is
Image of page 6
Image of page 7
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern