evaluation

evaluation - Areas of Interest •  Applica&on...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 3/2/10 Areas of Interest •  Applica&on being designed and developed Evalua&ng User Interfaces CSCI 264 – Design of Human ­ Computer Interface –  Design phase –  Alpha/Beta Prototypes –  Release version •  Compe&tor applica&ons •  User percep&ons about a task or applica&on User Interface Debugging •  Starts at design review prototyping &me (if possible) •  Have typical real users involved –  keep designers gagged in background to listen, observe, learn, and sweat! –  Plan on major investments here, par&cularly for commercial products Major por&ons, adapted from notes prepared by John Lovgren and Sherry Kalin. User Interface Debugging •  •  •  •  •  Test for problems in the interface Measure for speed of learning Measure for speed of use Test usefulness of documenta&on All itera&ve, but documenta&on has dis&nct purposes for different types of informa&on Used by permission. Evaluate Interface Problems •  •  •  •  Use typical users Test typical tasks Record details of all interac&ons Record interac&ons between users Evaluate Interface Problems •  Speed of learning –  Work with typical users, performing typical sequence of typical tasks –  Use typical training –  Control training &me –  Measure performance and compare to goals set as part of user requirements •  Record thinking protocol of user •  Record signs of stress or frustra&on •  Record user's subjec&ve reac&ons 1 3/2/10 Evaluate Interface Problems •  Speed of use –  Work with typical users, performing typical sequence of typical tasks –  Record &me to comple&on, and number and types of errors –  Measure enough users to have reasonable sta&s&cal validity –  Compare &mes to goals set as part of user requirements Evaluate Interface Problems •  Documenta&on –  Apply computerized tests for: •  Reading level •  Overly long/complex sentences –  Have users read documenta&on; give them a comprehension test –  Have users carry out tasks, aided only by documenta&on –  Have non ­users (i.e. new users) read documenta&on for clarity User Observa&on •  Let users try to get past trouble spots •  If they cannot: –  help them, but not too much –  explain only what is necessary –  note how you had to explain it –  give a break every 45 min. –  debrief par&cipant at end of session about problems User Observa&on •  Mind yourself –  Sit at a distance –  Stay concealed (with permission) •  Note places users have trouble: –  where –  how they got there –  what happened –  how did they try to recover (did they try?) Evalua&on Methodologies •  •  •  •  •  Audio tape Video tape Online tes&ng Ques&onnaires Interviews –  Structured –  Unstructured Audio Taping •  •  •  •  Always obtain wriden permission to tape session Place microphone where it will not be intrusive Conceal microphone if necessary (and appropriate) Make sure that the microphone can pickup an appropriate sound level (boost or reposi&on if necessary) •  Focus groups 2 3/2/10 Video Taping •  Always obtain wriden permission to video tape session •  Place camera and microphones as unobtrusively as possible •  Use mul&ple cameras to record: –  Screen –  Hands –  Eyes –  Other por&ons of the interac&on Online Tes&ng •  User ac&ons recorded in a comprehensive file for later analysis •  Can create overwhelming amounts of data •  Most useful in later stages of development •  Great poten&al for feedback from the field •  Focus on key events –  Errors –  Interac&on pathways –  Applica&on specifics Ques&onnaires •  Can be administered by untrained personnel or passively (e.g. a public website) •  Ques&ons need to be clearly explained •  Limit the number of ques&ons –  Too many ques&ons will cause boredom or anxiety –  Some ques&ons can be combined Structured Interviews •  Do not use leading/misleading ques&ons •  Carefully design ques&ons and ask them exactly as designed –  Varia&ons may not mean the same thing •  Provide appropriate response choices –  Mul&ple choice •  Address all possibili&es or make it short answer •  Allow for "don't care" type responses –  Short answer –  Unlimited response •  Ask for comments at end –  Gives users a chance to expand on certain areas –  Can point out issues in the interview itself •  Ques&onnaires could be considered a "fixed" interview Unstructured Interview Less formal than a structured interview Have a few basic (starter) ques&ons Let the interviewee talk at will Let the interviewee ask ques&ons Ask for all impressions or comments regarding toplc •  Debrief par&cipant at end of session •  •  •  •  •  Focus Groups •  An observer asks a series of ques&ons and gets responses from a group of users •  Group may be recorded or observed covertly •  Ques&ons may be direct answer (round ­robin) or discussion based 3 3/2/10 Data Gathering •  Establish a user interface tes&ng lab •  Methods –  One par&cipant, one observer –  One par&cipant thinking aloud, one observer –  Two par&cipants, no observer For Next Week •  Nothing (NO CLASS 3/18/08) (but make sure to finish your Design Document for the week aler) 4 ...
View Full Document

{[ snackBarMessage ]}

Ask a homework question - tutors are online