In order to produce reliable and generalizeable results an experiment must becarefully designed.–Choose the hypothesis, clarify the independent and dependent variables.How many participants can you get? Are they representative of your usergroup?–Which experiment method will you use:•1etween-subject•Within-subject
Gxperimentùl srr|ethod±²1etween-subjects– (or randomized) Design – each participant isassigned to a different condition• Advantages: controlled – no learning effect.• Disadvantage: Need more participants. Variations between the groupscan negate the results.°²Within-subjects–(or repeated measures) – Each user performs undereach different condition.• This design can suffer from learning effects, but this can be lessened ifthe order in which the conditions are tackled is varied between users.• For example group A does first condition followed by second and groupB does second condition followed by first.• Less costly than between subjects and fewer users are required.• Less chance of effects from variation between participants.
Gxperimentùl 'esignWithin-subjects– everyone tries all options – more work out of eachparticipant.1etween-subject– requires more people. Benefit uncorrupted and mostcommon.• Pros and Cons to each method.
bservùtion• Protocols:– “think aloud” – user talks through what s/he is doing– “cooperative evaluation” – users talks through what s/he is doing, butevaluator can interrupt and ask questions (“why did you do that?”, “what if …?”)
¢¢¬rotocol ùnùlyis• Evaluator takes notes during the session• Evaluator records session using video/audio (then someone needs to transcribeit)• Computer logs user data (filtering problem – need a program to log only theactions being assessed)• User logs their activities (but then they might forget some things)
¢¢¬rotocol ±nùlysis●Automatic protocol analysis tools○Annotate video/auto, making sure notes are recorded and tagged topositions○Can be used to synchronize data from multiple recording sources●Post-task walkthrough○Evaluator interviews users after the experimental interaction, shows videoand asks them to narrate what they did and why
Ëser query techniques• Evaluator asks user directly about the interface• Interview/questionaire– General (user background information, etc.)– Open-ended– Scalar (“rate from ° to °²”)– Multiple choice– Ranked (“pick the best of … “)
You've reached the end of your free preview.
Want to read all 51 pages?
- Fall '19