In order to produce reliable and generalizeable results an experiment must be

In order to produce reliable and generalizeable

This preview shows page 37 - 46 out of 51 pages.

In order to produce reliable and generalizeable results an experiment must becarefully designed.Choose the hypothesis, clarify the independent and dependent variables.How many participants can you get? Are they representative of your usergroup?Which experiment method will you use:1etween-subjectWithin-subject
Background image
Gxperimentùl srr|ethod±²1etween-subjects– (or randomized) Design – each participant isassigned to a different condition• Advantages: controlled – no learning effect.• Disadvantage: Need more participants. Variations between the groupscan negate the results.°²Within-subjects–(or repeated measures) – Each user performs undereach different condition.• This design can suffer from learning effects, but this can be lessened ifthe order in which the conditions are tackled is varied between users.• For example group A does first condition followed by second and groupB does second condition followed by first.• Less costly than between subjects and fewer users are required.• Less chance of effects from variation between participants.
Background image
Gxperimentùl 'esignWithin-subjects– everyone tries all options – more work out of eachparticipant.1etween-subject– requires more people. Benefit uncorrupted and mostcommon.• Pros and Cons to each method.
Background image
ºÄypes of user evùluùtion°.Controlled experiment: usually when there is a specific question about theinterface to be answered±.Usability testing: typically observational
Background image
bservùtion• Protocols:– “think aloud” – user talks through what s/he is doing– “cooperative evaluation” – users talks through what s/he is doing, butevaluator can interrupt and ask questions (“why did you do that?”, “what if …?”)
Background image
¢¢¬rotocol ùnùlyis• Evaluator takes notes during the session• Evaluator records session using video/audio (then someone needs to transcribeit)• Computer logs user data (filtering problem – need a program to log only theactions being assessed)• User logs their activities (but then they might forget some things)
Background image
¢¢¬rotocol ±nùlysisAutomatic protocol analysis toolsAnnotate video/auto, making sure notes are recorded and tagged topositionsCan be used to synchronize data from multiple recording sourcesPost-task walkthroughEvaluator interviews users after the experimental interaction, shows videoand asks them to narrate what they did and why
Background image
Ëser query techniques• Evaluator asks user directly about the interface• Interview/questionaire– General (user background information, etc.)– Open-ended– Scalar (“rate from ° to °²”)– Multiple choice– Ranked (“pick the best of … “)
Background image
Ëser evùluùtion through physiologicùldùtù• Use medical devices to record physiological responses– Eye-tracking• Record number of fixations (user’s gaze is fixed on same location forsome period of time)• Length of fixation• Scan path
Background image
Image of page 46

You've reached the end of your free preview.

Want to read all 51 pages?

  • Fall '19

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture