3750 CS/PSY - User Interface Design ------------------------------------------------------------------------
Group Project: User Interface Evaluation and Design
Outline
Quick access to the sections of this document: * * * * * * * Project Report Book Part 0 Project/team Definition Part 1 - Understanding the Problem Part 2 - Design Alternatives Part 3 - System prototype and evaluation plan Part 4 - Evaluation Project Presentation
Project Overview
This term you will undertake a group project (usually 4 people, sometimes 3, never 5) to evaluate some computing-related task/problem, to develop interface design alternatives for the task/problem, to implement a prototype of your design, and to evaluate your design. This will provide you with hands-on experience with the user-computer interface design process. Ideally, the topic of the project will be a problem that matters to some "real-life" people. These people then will serve as your "clients", with whom you will communicate with and learn from. Sometimes we may have some specific projects for you to consider; more often you will need to develop your own project
idea. Your project idea must be approved prior to completing Project Part 0. Each project group will be graded as a team, that is, each person receives the same grade. I will poll team members, however, to determine each members individual contribution. Lack of effort will result in an individual reduction of grade. Within the team, you must negotiate on what each person will contribute. Think carefully about your team members: Where do people live and what hours do they work? Where will you meet? What skills do the different individuals bring to the group (computing, programming, design, evaluation, statistics, etc.)? I would strongly encourage you to form a heterogeneous team full of individuals with varying skills. Working with friends who happen to share your skill set is not a formula for success. For due dates, refer to the class syllabus.
Project Report Book
Each part of the project will include a deliverable report. This report will be placed on the WWW as either pdf generated from Microsoft Word, or a Microsoft Word file. The report must be a single file. Your team name and member names should be at the start of each report. Each team should have a "home" page that includes: 1) A brief (one paragraph) description of the problem/task; 2) Names and email addresses of the team members; 3) Links to the reports for project parts 1-4 (no report is needed for part 0); 4) Link to your PowerPoint presentation on Requirements Gathering Techniques and Results (if assigned); 5) Link to your PowerPoint presentation on Testing and Evaluation Plans (if assigned);
6) Link to PowerPoint Final Project Presentation. The format of the reports for the individual parts is up to you, but it should be professionally prepared, expressive, grammatically sound, illustrative of your efforts and process, and easy to read and understand. Teams sometimes use long series of short bullet items for their report; this is not satisfactory bullets are fine, but they need to be explained and motivated and interconnected. A good design effort can easily be hampered by poor communication of what was done. Note that the PowerPoint presentations you prepare are NOT the report; they are generally derived from the report but are not a substitute for a written report.
Part 0 - Topic Definition
This is your one-paragraph project description, list of team members and link to (future) project deliverables. Your TA will help you set up the Swiki for this. A simple template for your project notebook is available for you to use. Include a listing of who will (at least initially) fill each of the following roles: 1) Project leader leads meeting discussions, summarizes meeting results and who has agreed to do what by when. 2) Meeting scheduler communicates with everyone to establish meeting times/places. 3) Task schedule monitor checks with team in advance of deliverable due dates to ensure that things are on track. 4) Report integrator pull together into a coherent whole the parts of reports that are typically written by multiple people. 5) Presentation creator the PowerPoint expert who creates the teams presentations.
These roles can rotate amongst team members but it is important to identify who has each role. Also, you can re-define roles. The most important criteria is that everyone knows who is filling what role!
Part 1 - Understanding the Problem
The key goal of this first part of the project is to deeply understand the problem you are addressing, its set of pertinent users, and the issues and constraints that are involved in the problem. If there is an existing system/interface for performing the task, you should review that system to help you learn more about it. Most important is to identify the important characteristics of the user needs that will influence your subsequent design. In class we will discuss different techniques for acquiring this kind of information. You should utilize the techniques that you feel are most appropriate to the particular task you are examining. Your report and deliverable for this part should deeply examine the application domain. Who are the potential users? What tasks do they seek to perform? What functionality should the system provide to allow the users to carry out their tasks? Basically, you are establishing a set of user needs requirements and constraints for your subsequent design. You will also address criteria will be used to judge if your design is a success or not. Whatever methods you use, they must include face to face interviews with people, either individually or in focus groups. More specifically, you should develop the following items in this part, and you should communicate them through your report. The number of points assigned to each part for grading are in parentheses.
1. (5) Introduction: A one-page introduction to what your system is all about, written for the potential buyer/user of the system. Not done as an advertisement with hype, but factual. 2. (5) Requirements Summary: In one page, say as much as you can about the functional requirements that your task analysis suggests. 3. (5) Methodology: A description and justification of how the information was gathered: interview (how many, what kind of people, what questions) or focus group or questionnaire or observation, etc etc. 4. (5) Constraints: A list of the real-world constraints under which you are working, including elapsed time and persondays of effort. 5. (10) User Characteristics: A description of the important characteristics of the users of the system. This should take the form of a table of the sort shown in the lecture notes, and should include a persona description for each type of user. 6. (30) Task Analysis: Start with a written description of the major user tasks. Then continue with: 6.1 Hierarchical Task Decomposition, including plans (sequences) of steps. This might be simple taking just a page or two or more complex, depending on the complexity of your application. If in doubt consult with your TA or teacher. 6.2 Diagrams of workflows or processes. These may or may not be needed, depending on the complexity of your application. If in doubt consult with your TA or teacher.
6.3 Object model of objects, properties of objects, operations on objects, relations between objects. Students often do this with UML or E-R diagrams. 6.4 Environment issues concerning where or how your system will be used, such as in the rain, while jogging, in bright light, etc. 7. (10) Usage Scenarios: Specific usage scenarios (at least four) that you will use later for testing alternative designs and the final design. 8. (5) Current UI Critique: of the existing system/interface, if one exists. Big picture strengths, weaknesses of the current design. Not at the detailed level. In what ways will your design differ from and improve on the current system (this could be in terms of functionality, usabilty, intended audience, compute platform, etc.) If there is no current UI, then discuss how things are done now and how your system will improve on that. 9. (10) Usability Goals: A table with concrete, specific, measurable usability goals such as found in the course notes. 10. (10) Implications: A discussion of the implications of what you learned above. Don't just describe the target users, tasks, environment, etc. Tell us how these attributes should/will influence your design. Are there any implications to be drawn from the user profiles and other data you learned? We will be very careful to look for this information in your report.
11. (5) Reflections: on Part 1 such as, what was hard about this phase of the project? What was easy? What would you do differently if you were to start over? What you would do next if you had more time? Be sure to stress both the functional and non-functional (usability principles) requirements for your eventual design. Functional requirements are all about what the system should do. Nonfunctional requirements are the usability attributes that this particular design must stress, e.g., learnability, robustness, etc. A successful project report for this part would be one that could be handed to a different group of people not familiar with the topic area, and then that group could do an excellent design.
Part 2 - Design Alternatives
The key goal of part 2 of the project is to use the knowledge gained in part 1, as well as that from class, to develop a set of three design alternatives for your problem. These multiple design alternatives should explore the potential design space for the problem. When you do a design, you have lots of different decisions to make, starting with the basic conceptual model (objects, relations, actions) and then the basic interaction styles (command line, wimp , gui) and then the detailed design (names of commands, layout of menus, ..) The set of all possible designs is very very large, and is often called a 'design space.' The more different your three designs are from one another, the better. The idea is simply to make the three designs as different from one another as you can. If you can think of three different conceptual designs, that's great. Or maybe three different metaphors, or three different interaction styles - or combinations of any of these things. Again, the idea is simply to have the three designs be noticeably different from one another. A rearrangement of the screen layout for a WIMP-style UI would not lead to a noticeably different design.
In this part of the project you will develop mock-ups, storyboards, and sketches of your interface designs. Provide pencil-and-paper or electronic images of the interface at various stages of use; you do not need to build a working prototype. Your design sketches should be sufficiently detailed for a potential user to provide useful feedback about the design, however. Along with your design mock-ups, you should provide a brief narrative walk-through of how the system will work. Perhaps most importantly, you should also include your justifications for why design decisions were made, and what you consider to be the relative strengths and weaknesses of your different designs. The design process you follow in this part of the project is important. Don't do the following: The group splits up and everyone creates one design, then these become your alternatives to be turned in. This is not how a good, creative design process works. It should be more like a brainstorming session with all team members present. You should seek to create some fundamentally different design ideas, concepts all over the potential design space for the problem you have chosen. Your project report should include all the explanatory material mentioned above as well as all the design sketches, drafts, storyboards, etc., that you generated for each of the three designs. If some of your sketches are on paper, scan them. The posters used in your poster session are not a substitute for any part of the report. Make sure that your report adequately reflects the design process that your group undertook. The key in this part of the project is to come up with fundamentally different design ideas, not just a small set of variations from some basic design.
It is okay if things change from your first report, in fact it is most likely good! The design process is intended to be iterative. Remember, abandoning bad ideas early is a good thing.
1. (5) Introduction and Requirements Summary updated (if you have made changes) from the first two sections of your project report for part 1, section with changes in italics. If no changes were made, just copy and paste from your part 1 report. 2. (10) Design space Describe design the space of the potential interfaces for your system. What requirements may be difficult to realize? What are some tradeoffs that you should explore? How could your interface support some tasks easier than others? Describe the design alternatives that you considered exploring and then give a brief description and justification of the three (or more) alternatives that you did explore. 3. (60) Three interface designs With each design include: a. A rationale for this design choice. b. Illustrations of the design (sketches, storyboards ...) c. At least one use case scenario. Scenarios should show start to finish use of the system to carry out the use case. 4. (10) Assessment of this design This assessment should include action counts for the usage scenarios from Part 1. List pros/cons of this design based on your own teams discussions and also on that of some users. The pros/cons are not a formal evaluation, just opinions. Pull this all together in a table comparing the three designs on as many criteria as you can develop. 5. (5) Reflection on your process for creating and assessing the prototypes. What you would do differently next time, what you would do the same next time, your team processes.
6. (10) Poster presentation not part of report, but part of grade for part 2 of your project. Grading criteria are: a. How well poster communicates overall purpose of the project b. How well poster communicates the essence of the three designs We will utilize one full class day as a poster session near the end of this part of the project. Each group will show their three design ideas on a poster in class. Everyone will then circulate and interact with the designers. The idea here is that each group can use this opportunity to get feedback about their design ideas as they narrow their design space and head into part 3 of the project. Poster Preparation Suggestions 1) Set of bullet items to define the problem. 2) Three panels, one for each of three design alternatives. 3) Meaningful name for each alternative. 4) Use screen sketches; if an action on one screen leads to another, draw a line leading from the action (button selection, menu item, etc.) to the new screen). See these pictures as examples:
Part 3 - System Prototype and Evaluation Plan
In part 3 of the project, your group will implement a detailed prototype of your interface. Use whatever software tools that you know and that are good for prototyping, such as Visual Basic, Flash, Hypercard, Macromedia Director or a web page editor. You should be able to get much of the interface functionality working, but in most cases you will not be able to implement all the backend application functionality. Provide a set of usability specifications for your system and a plan for an evaluation of it. To develop usability specifications, consider the objectives of your design. For example, if you are working on a calendar manager, you might specify time limits in which you expect a user to be able to schedule or modify an appointment, or a maximum number of errors that you expect to
occur. Basically, you should list a set of criteria by which your interface can be evaluated. This will be a refinement of the usability goals you developed in Part 1. Describe your initial evaluation plan for the system. What kinds of benchmark tasks would you have users perform to help evaluate the interface? What kind of subjective questionnaire would you deploy to have a user critique the interface? You will need to perform this evaluation in project part 4, so you should do your best to set it up now. The key here is not to do some exhaustive description of a usability evaluation plan, but to motivate why the particular plan you propose is appropriate for this interface. Note that developing an initial evaluation plan is also a good way to figure out how much of the interface you need to develop. You should be able to build and connect to enough of the application functionality to be able to conduct an initial usability evaluation with the benchmark tasks as you are proposing here. Your write-up for this part should include a description of your system prototype. You can include screen dumps to help explain it and text to describe how a user would interact with it. Discuss the implementation challenges you faced. Were there aspects that you wanted to build but were unable to do so? The key component to include in your project report is a justification of why you settled on the design that you chose. What's special about this particular design with respect your problem? The report for this part also must include the usability specifications that you established and a description of the evaluation that you are planning. This needs not be too detailed here as the actual evaluation will occur in part 4. We will try to give you helpful feedback about your plan here to assist with the testing in part 4.
Specifically, the report should follow this outline: 1. (5) Introduction and Requirements Summary: Updated (if you have made changes) from the first two sections of your project report for part 2, section with differences from part 2 in italics. If no changes were made, just copy and paste from your part 2 report. 2. (10) Final Design Summary: A summary of the final design you selected. Often this will be a modification of one of the three initial designs. The best way to provide the summary is with one or two screen shots plus some text. Explain why you chose this final design. 3. (35) Prototype Description: This section should be detailed. Start out with a big picture overview. Include screen shots, photos, and/or sketches of everything. Have a use case, showing step by step screens. I need to know how you made the prototype (using Flash, VB, Director, carved it out of styrofoam, etc), whether it's high/low fidelity, what level of functionality the prototype has (i.e. wizard of oz, fully functional), how realistic it will seem to your users, etc. Expect the grading for this to be subjective. If it looks like you put a lot of work and thought into your prototypes, based on your project write up and the meeting with the GTA and Teacher, you will likely get a good grade. 4. (40) Evaluation Plan: Your evaluation plan should be a detailed description of what you're going to do for Project Part 4. You should list which features of your system you will be testing and why, what benchmarks your prototype will test, techniques you're going to use and why, and detailed descriptions of how you're going to use them. For example, if you're going to interview people, provide a list of questions you are going to ask. If you're performing a heuristic evaluation, provide a list of the heuristics you're going to use. For a questionnaire, list the questions!
Write out the instructions you will give users who test the system for you. Describe which types of users or evaluators you are going to use, and what you hope to learn about your system. 5. (10) Reflection: What you learned, where you think your project is going, what you would do differently, what you would do the same, how your team functioned. How do you think your prototypes will fare when used for usability testing? Has the project focus changed since you first started? What parts are you happy with, or which parts do you wish worked better? What was hard about designing the prototypes? Did you have to change any of your benchmarks due to the design of your prototype? etc, etc. I'm looking for insightful comments here.
After this part is complete, each group will demo their system for the teacher and TA. The quality of the demo will be factored into the grade for section 4 of your part 3 re...