“A survey is only as good as the clarity of your terms, the directness of your questions, and the relevance of your answer choices.” -inow
A questionnaire or survey takes the form of carefully selected questions that want to elicit honest and specific opinions from the participant. These opinions are important to generate statistics about a product or service for analysis. If the opinions are dishonest or vague, it will badly affect the outcome of our analysis. The above quote tells us a few key points to look out for when we create our questions.
2. Good Questions vs. Bad Questions
Here are examples of bad questions as opposed to their good counterparts.
- Do you think you will take a long time to use this function?
- Rate on a scale of 1 to 7 how much time you think it will take you to use this function. (1-short, 7-long)
In this example above, the good example not only asks for a rating so participants can more specifically rate their response, but it also balances rating scale with 1-4-7 where 4 is the middle-point. The good example also avoids asking with a biased leading question, which the bad example does when it implies to the user to think that he will take a long time to use that function.
- Which design do you think is best?
- Which design do you think is the prettiest and most easy to use?
In this second example, the bad part asks a very generic question that may be understood differently for each user. For some users, ‘best’ might mean prettiest, whereas for others it might mean the easiest to use. To clarify this, a good question should ask specifically and clearly what the user needs to consider.
- Which feature in the list would you dislike to see in a product you would find hard to use?
- Which feature in the list would you like to see in an easy-to-use product?
In this third example, the bad part asks a double negative question, and by doing so will confuse the participant and cause misunderstandings in the question. This question can be worded properly in the good part without using double negatives.
- Answer Yes or No. There is no sorting mechanism you can see in this tool.
- Answer Yes or No. Do you see no sorting mechanism in this tool?
- Answer Yes or No. Do you see a sorting mechanism in this tool?
In this fourth example, the first bad part is not even a question, but an assertion. As such, it is important to remember to ask questions rather than assert them. Also in both bad parts, the answer to be given is ambiguous in relation to the question. Some people would answer ‘No’ to a question asking for a ‘no’ as their way of confirmation, whereas other people confirm by answering ‘Yes’ to a ‘no-question’. This is how people interpret differently so misunderstandings will cause many to answer wrongly.
Not all the important parts were covered in the above examples, but in summary, a good survey or questionnaire needs:
- Use simple words
- Correct grammar
- Common understanding with and about the participants
- Avoid leading questions
- Avoid double negatives
- Ask questions, not assertions
- Balance rating scales
- Make choices list simple and short
- Alignment of questions, choices and answer boxes should be consistent
- Avoid open-ended questions (give choices instead of written)
- Put questions in logic order (helps mind flow)
What happens when you do not have a common understanding about the participants: