Unformatted text preview: Questionnaire Development Questionnaire
Asking Questions You must ask the right question must understand your question must know the answer must be willing and able to tell you Respondents Respondents Respondents Respondents the answer the What is a Good Question? What One One One One that yields a truthful, accurate answer the asks for one answer on one dimension the accommodates all possible contingencies of response contingencies that uses specific, simple language One A good question… good One One that has mutually exclusive response options options that produces variability in response that minimizes social desirability that is pre-tested One One One Ask questions one at a time Ask Bad question: In the past 6 months, In what major appliances has your household purchased new from the store? store? Better question: Please indicate whether Please your household has purchased this type of appliance new from the store during the past 6 month. How about… How a refrigerator? a kitchen range or oven? a microwave? Ask questions one at a time Ask Bad question: Better question: Compared to one Compared year ago, are you paying more, less, or about the same for your auto and life insurance? insurance? Compared to one Compared year ago, are you now paying more, less, or about the same for .... a. auto insurance? a. b. life insurance? Use words with singular meanings Use Ambiguous How would you compare How how close you are to family in your current neighborhood to how close you were in your old neighborhood? Would you say your family is close here, further here or the same Clearer Compared to your last Compared neighborhood, do you now live closer to your family , are your further from your family, or are you about the same distance? distance? Be specific…Specify! Specify through cues: People drink beer in many places - for example, at home, at People restaurants, at bars, sporting events, at friends’ homes, etc. During the past 30 days did you drink any beer? During Specify who, what, when , where, and how. For example, whose Specify income? What’s included? Over what period of time: income? In 2007, what was your total household income, before taxes? In Please count income from all members of your household, including wages from employment, disability, social security, and pubic aid. wages Types of Survey Questions Types
1. Those that ask about behaviors or facts Non-threatening behavior questions Threatening behavior questions Demographics 2. Those that ask about psychological states 2. or attitudes or 3. Those that ask about knowledge 3. Open vs. Closed Question Open General General rule: closed questions are usually usually better better Easier for the respondent Less coding later Better to have respondent do categorizing Categories help define the question Disadvantages of Closed Question Categories may be leading to respondents May make it too easy to answer without May thinking May limit spontaneity Not best when - Not Asking for frequency of sensitive behaviors There are numerous possible responses General Principles for Response Options Options Response Response categories should be consistent with the question with Categories must be exhaustive, including Categories every possible answer every Categories must be mutually exclusive (no Categories overlap) overlap) If appropriate, include a “don’t know” If category category In the Middle In Common practice is to omit it to push respondents Common towards one end or the other, on the theory that few individuals are truly in the middle on a particular issue individuals Evidence from empirical studies shows that use of an Evidence explicit middle alternative will often be taken by Rs in a Rs forced choice situation if offered; at the same time, it does not affect the ration of pro to con responses or the size of the don’t know category size Common practice: A scale should include a middle Common category option unless there are persuasive reasons not to to Constructing Response Scales Constructing Rs can generally only remember a maximum of 5 responses unless visual cues are used responses Using graphic images such as thermometer and Using ladders and using card sorting for complex ratings is effective ratings Number of points on a scale should be Number determined by how you intend t use the data determined With scales a few points, every scale can be With labeled; in longer scales, only the endpoints are labeled labeled Ordering Response Categories Ordering Usually Usually better to list responses from the lower to the higher level lower greater responses level with greater numbers greater with the end of the scale that is the least socially desirable least Associate Associate Start Start General hints for question writing General
1. 2. 3. 4. 5. 6. 7. 8. Go from general to specific Keep your questions concise Only one topic, idea, theme at a time Where appropriate use anchors (Left, Right) Question stems should be absent of bias, if scale Question dimension is mentioned both positive and negative should be included should Words should have a singular meaning, when possible Words define term within the context of the question define Use transitional phrasing Be aware of social desirability Knowledge Questions Designed to assess current knowledge or awareness Designed of a particular topic of Ask knowledge questions before attitude Consider difficult of question Reduce threat by asking KQs as opinion or by using Reduce phrases such as “do you happen to know” or “can you recall” recall” if yes/no are appropriate ask several of these in a roll to reduce successful guessing to For KQ that require a number (#) write as an open end For to avoid giving away answer or misleading R to Consider use of picture 1. 2. 3. 4. 5. 6. Threatening Questions Threatening
1. 2. 3. 4. 5. 6. 6. 7. 8. 9. Open better than close Long better than short Use of familiar words my increase overreporting because of social Use desirability desirability For Socially undesirable behavior ask if one has EVER before For current current For Socially desirable ask about CURRENT rather than usual For Embed the threatening topic into a list of more and less Embed threatening topics, to reduce the perceived importance of the topic to the Rs topic Consider alternatives to standard questions, such as randomized Consider response, card sorting and sealed envelopes response, Avoid reliability check on the same questionnaire Ask Rs at the end of the interview about their perception of Ask threatening questions threatening Attitude Questions Attitude Components of an attitude 1. Affective or evaluative – like/dislike, 1. favor/disfavor, pro/con favor/disfavor, 2. Cognitive – what R knows or thinks 3. Action – R willingness or intention to do 3. something something Strength of an attitude Strength The strength of an attitude measured by 1. Building a strength dimension into the 1. question itself, so that evaluations and strength are measured at same time are 2. Use separate questions 3. Consider asking a series of independent 3. questions questions Attitude Questions Attitude
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Clearly specify attitude object (what you are trying to Clearly measure/find out about) measure/find Decide which aspect you are measuring (evaluative, cognitive, Decide action) action) Measure strength by building strength dimension into question by Measure asking a series of questions or separate questions asking Avoid double-barreled questions Consider unipolar items if bipolar item might miss individual Consider dimension dimension Standardize answer categories Ask general before specific attitude questions are related When asking questions about varying degree of popularity ask When about the least first about Pretest attitude questions If measuring over time ask the exact same question each time ...
View Full Document
This note was uploaded on 06/11/2010 for the course EPSY 589 taught by Professor Carolynanderson during the Spring '09 term at University of Illinois at Urbana–Champaign.
- Spring '09