Cognitive A Dimensions Questionnaire
Alan Blackwell and Thomas Green
This questionnaire was developed as a tool for assessing the usability of information devices by means of the Cognitive Dimensions of Notations framework. For further reading on the framework, see: http://www.cl.cam.ac.uk/~afb21/CognitiveDimensions/ To download the current version of this questionnaire, see http://www.cl.cam.ac.uk/~afb21/CognitiveDimensions/CDquestionnaire.pdf We would be extremely grateful to be kept informed of the use of the questionnaire. If you contact us before using it, we will be able to supply any recent amendments both in format and analysis techniques. Contact: Alan Blackwell [e-mail] Computer Laboratory Cambridge University Thomas Green [e-mail] Computer-Based Learning Unit Leeds University
This version: 5.1.0 November 2000 Copyright 2000 Alan F. Blackwell and Thomas R.G. Green
2000 [e-mail] & [e-mail] Before use, please contact us for latest version.
[V5.1 25-Nov-00]
Thinking about Notational Systems
This questionnaire collects your views about how easy it is to use some kind of notational system. Our definition of "notational systems" includes many different ways of storing and using information books, different ways of using pencil and paper, libraries or filing systems, software programs, computers, and smaller electronic devices. The questionnaire includes a series of questions that encourage you to think about the ways you need to use one particular notational system, and whether it helps you to do the things you need.
Section 1 - Background information:
What is the name of the system? How long have you been using it? Do you consider yourself proficient in its use? Have you used other similar systems? (If so, please name them)
Section 2 - Definitions:
You might need to think carefully to answer the questions in the next sections, so we have provided some definitions and an example to get you started:
Product Notation Sub-devices
The product is the ultimate reason why you are using the notational system what things happen as an end result, or what things will be produced as a result of using the notational system. This event or object is called the product. Any product that needs a notation to describe it usually has some complex structure.
The notation is how you communicate with the system you provide information in some special format to describe the end result that you want, and the notation provides information that you can read. Notations have a structure that corresponds in some way to the structure of the product they describe. They also have parts (components, aspects etc.) that correspond in some way to parts of the product. Notations can include text, pictures, diagrams, tables, special symbols or various combinations of these. Some systems include multiple notations. These might be quite similar to each other for example when using a typewriter, the text that it produces is just letters and characters, while the notation on the keys that you press tells you exactly how to get the result you want. In other cases, a system might include some notations that are hard for humans to produce or to read. For example when you use a telephone the notation on the buttons is a simple arrangement of digits, but the noises you hear aren't so easy to interpret (different dialling tones for each number, clicks, and ringing tones). A telephone with a display therefore provides a further notation that is easier for the human user to understand. Complex systems can include several specialised notations to help with a specific part of the job. Some of these might not normally be considered to be part of the system, for example when you stick a Post-It note on your computer screen to remind you what to write in a word processor document. There are two kinds of these sub-devices. 7KH 3RVW,W QRWH LV DQ H[DPSOH RI D helper device. Another example is when you make notes of telephone numbers on the back of an envelope: the complete system is the telephone plus the paper notes if you didn't have some kind of helper device like the envelope, the telephone would be much less useful. $ redefinition device changes the main notation in some way such as defining a keyboard shortcut, a quick-dial code on a telephone, or a macro function. The redefinition device allows you to define these shortcuts, redefine them, delete them and so on. Note that both helper devices and redefinition devices need their own notations that are separate from the main notation of the system. We therefore ask you to consider them separately in the rest of this questionnaire.
2000 [e-mail] & [e-mail] Before use, please contact us for latest version. [V5.1 25-Nov-00]
To review how we intend to use these terms, consider the example of typing business letters on a word processor. The product of using the word processor is the printed letter on paper. The notation is the way that the letter looks on the screen on modern word processors it looks pretty similar to what gets printed out, but this wasn't always the case. If you want to find and replace a particular word throughout a document, you can call up a helper device, the search and replace function, usually with its own window. This window has its own special notation the way that you have to write the text to be found and replaced, as well as buttons that you can click on to find whole words, or to find the word in upper and lower case etc.
Section 3 Parts of your system:
What task or activity do you use the system for? What is the product of using the system? What is the main notation of the system? When using the system, what proportion of your time (as a rough percentage) do you spend:
Searching for information within the notation Translating substantial amounts of information from some other source into the system Adding small bits of information to a description that you have previously created Reorganising and restructuring descriptions that you have previously created Playing around with new ideas in the notation, without being sure what will result
% % % % %
Are there any helper devices?
Please list them here, and fill out a separate copy of section 5 for each one.
Are there any redefinition devices?
Please list them here, and fill out a separate copy of section 5 for each one.
2000 [e-mail] & [e-mail] Before use, please contact us for latest version.
[V5.1 25-Nov-00]
Section 4 Questions about the main notation:
[VIJU] [VISC] [DIFF] [HMOS] [ERRP] [CLOS]
How easy is it to see or find the various parts of the notation while it is being created or changed? Why? What kind of things are more difficult to see or find? If you need to compare or combine different parts, can you see them at the same time? If not, why not? When you need to make changes to previous work, how easy is it to make the change? Why? Are there particular changes that are more difficult or especially difficult to make? Which ones?
Does the notation a) let you say what you want reasonably briefly, or b) is it long-winded? Why? What sorts of things take more space to describe?
What kind of things require the most mental effort with this notation? Do some things seem especially complex or difficult to work out in your head (e.g. when combining several things)? What are they?
Do some kinds of mistake seem particularly common or easy to make? Which ones? Do you often find yourself making small slips that irritate you or make you feel stupid? What are some examples?
How closely related is the notation to the result that you are describing? Why? (Note that in a sub-device, the result may be part of another notation, rather than the end product). Which parts seem to be a particularly strange way of doing or describing something?
2000 [e-mail] & [e-mail] Before use, please contact us for latest version.
[V5.1 25-Nov-00]
[ROLE] [HIDD] [PROG] [PROV] [PREM]
When reading the notation, is it easy to tell what each part is for in the overall scheme? Why? Are there some parts that are particularly difficult to interpret? Which ones? Are there parts that you really don't know what they mean, but you put them in just because it's always been that way? What are they? If the structure of the product means some parts closely are related to other parts, and changes to one may affect the other, are those dependencies visible? What kind of dependencies are hidden? In what ways can it get worse when you are creating a particularly large description? Do these dependencies stay the same, or are there some actions that cause them to get frozen? If so, what are they? How easy is it to stop in the middle of creating some notation, and check your work so far? Can you do this any time you like? If not, why not? Can you find out how much progress you have made, or check what stage in your work you are up to? If not, why not? Can you try out partiallycompleted versions of the product? If not, why not? Is it possible to sketch things out when you are playing around with ideas, or when you aren't sure which way to proceed? What features of the notation help you to do this? What sort of things can you do when you don't want to be too precise about the exact result you are trying to get? When you are working with the notation, can you go about the job in any order you like, or does the system force you to think ahead and make certain decisions first? If so, what decisions do you need to make in advance? What sort of problems can this cause in your work?
2000 [e-mail] & [e-mail] Before use, please contact us for latest version.
[V5.1 25-Nov-00]
[CONS] [SECN] [ABST] [NOVL] [IMPR]
Where there are different parts of the notation that mean similar things, is the similarity clear from the way they appear? Please give examples. Are there places where some things ought to be similar, but the notation makes them different? What are they? Is it possible to make notes to yourself, or express information that is not really recognised as part of the notation? If it was printed on a piece of paper that you could annotate or scribble on, what would you write or draw? Do you ever add extra marks (or colours or format choices) to clarify, emphasise or repeat what is there already? [If yes: does this constitute a helper device? If so, please fill in one of the section 5 sheets describing it] Does the system give you any way of defining new facilities or terms within the notation, so that you can extend it to describe new things or to express your ideas more clearly or succinctly? What are they? Does the system insist that you start by defining new terms before you can do anything else? What sort of things? If you wrote here, you have a redefinition device: please fill in one of the section 5 sheets describing it. Do you find yourself using this notation in ways that are unusual, or ways that the designer might not have intended? If so, what are some examples?
After completing this questionnaire, can you think of obvious ways that the design of the system could be improved? What are they? Could it be improved specifically for your own requirements?
2000 [e-mail] & [e-mail] Before use, please contact us for latest version.
[V5.1 25-Nov-00]
Section 5 Questions about sub-devices:
Please fill out a copy of this page for each sub-device in the system. This page is describing (tick one box): a helper device , or a redefinition device What is its name? What kind of notation is used in this sub-device? When using this sub-device, what proportion of the time using it (as a rough percentage) do you spend:
Searching for information Translating substantial amounts of information from some other source into the system Adding small bits of information to a description that you have previously created Reorganising and restructuring descriptions that you have previously created Playing around with new ideas in the notation, without being sure what will result
% % % % %
In what ways is the notation in this sub-device different from the main notation? Please tick boxes where there are differences, and write a few words explaining the difference.
[VIJU]
Is it easy to see different parts? Is it easy to make changes? Is the notation succinct or long-winded? Do some things require hard mental effort? Is it easy to make errors or slips? Is the notation closely related to the result? Is it easy to tell what each part is for? Are dependencies visible? Is it easy to stop and check your work so far? Is it possible to sketch things out? Can you work in any order you like? Are any similarities between different parts clear? Can you make informal notes to yourself? Can you define new terms or features? Do you use this notation in unusual ways? How could the design of the system be improved?
2000 [e-mail] & [e-mail] Before use, please contact us for latest version.
[IMPR]
[NOVL]
[ABST]
[SECN]
[CONS]
[PREM]
[PROV]
[PROG]
[HIDD]
[ROLE]
[CLOS]
[ERRP]
[HMOS]
[DIFF]
[VISC]
[V5.1 25-Nov-00]
Section 5 Questions about sub-devices:
Please fill out a copy of this page for each sub-device in the system. This page is describing (tick one box): a helper device , or a redefinition device What is its name? What kind of notation i...