The Health Canada Policy Toolkit for Public Involvement in Decision Making

Level 2 Technique:
Questionnaires (also see "Surveys")

What Is It?

Questionnaires are a method of data collection using a sample of people and are generally used to gather information on the telephone, face to face, or by selfadministered means through the mail. The ultimate utility of a questionnaire will depend in part on proper problem definition and clear objectives.

How It Works

Questionnaire Construction

The questionnaire must translate the research objectives into the specific questions and convince the respondent to provide the information. The major considerations involved in formulating questions are their content, structure, format and sequence. Depending on the degree of sophistication being sought, an expert may be best able to design the questionnaire.

1. Content: Most questions can be classified into either factual questions or questions about subjective experiences.
   • Factual: Designed to gather information from a respondent's background and his or her habits
   • Subjective: Inquires about the respondent's beliefs, attitudes, feelings and opinions
2. Structure: There are three main types of question structures; open-ended, closed-ended and contingency questions.
   • Open-ended questions: There are no choices offered, and the respondent's answers are recorded. The advantage of open-ended questions is that respondents are not forced into abiding by categorized answers, yet they can be difficult to answer and may be hard to analyse.
   • Closed-ended questions: Respondents are asked to choose, among a set of answers, the one that most closely represents their views. They are quick to answer and easy to analyse, yet may introduce bias by making respondents choose from a set of predetermined options.
   • Contingency questions: This is a type of closedended question that applies only to a subgroup of respondents. The subgroup is decided by the answer of all respondents to a preceding filter question, which will either include or exclude them from the subgroup. They are useful because it may be necessary to include some questions that may be applicable only to some respondents.
3. Format: There are many techniques for structuring the response categories of closed-ended questions.
   • Rating scales capture the intensity of the respondent for a set of ordered categories, such as "strongly agree," "favourable" or "very often."
   • Semantic differential is a rating scale that measures reactions to objects or ideas in terms of a bipolar scale defined with contrasting adjectives on each end, such as (Good 3 2 1 0 1 2 3 Bad)
   • Ranking is used to gain information regarding the degree of importance or the priorities that people attach to attitudes and objects. It helps to grant relative order, but does not give any information about the distance between the ranked numbers.
4. Question sequence: The two most common question sequences are the funnel sequence and the inverted funnel sequence.
   • Funnel sequence: Each question is related to the previous question which has a gradually narrower scope. This sequence is useful when the survey's objective is to gather detailed information. When the survey is more exploratory in nature, the funnel sequence works effectively to ask the broad questions first.
   • Inverted funnel sequence: In this sequence, narrower questions are followed by more general ones. This method is used when a researcher wants to make a generalization regarding a specific situation and if the researcher is unfamiliar with the facts but the respondents know them.

When Is It Most Useful?

The questionnaire can be useful as a means of gathering information to measure attitudes and obtain opinions. As a general rule, questionnaires should not exceed six pages in length due to respondents' time constraints.

Logistics and Limits

• If a questionnaire has a low response rate, its reliability will depend upon verifying that the non-respondents are similar to the respondents.
• Questionnaire format: Proper investment in format and typography will likely result in a higher response rate.
• Covering letter: Identify the sponsor of the study, explain its purpose, tell the respondents why they should fill out the questionnaire, how they were selected, and ensure the study's confidentiality.
• The lack of a self-addressed, prepaid envelope will reduce the response rate.
• A follow-up phone call (call back) for those unavailable the first time may be necessary to raise telephone interview response rates.

Cost Implications

Mail questionnaires are low in cost relative to personal interviews or telephone surveys, yet they are not inexpensive, since additional mailings may be required.

Expectation for Feedback or Follow-up

• For mail questionnaires, a letter reminder requesting that the respondent return the questionnaire is almost always necessary to raise the response rate.
• A follow-up questionnaire may also be required.
• Incentives may have to be offered and may depend on the length of the questionnaire.

Timelines

In terms of the speed of data collection, telephone interviews are the quickest, followed by personal interviews and mail surveys. The researcher essentially has no control over the return of the self-administered mail survey.

Potential Pitfalls

• It is important that the question is worded so the respondent understands it.
• Vary the question or issue format to avoid the tendency for some respondents to answer all questions in a specific direction regardless of their content.
• Avoid questions that the respondent interprets as leading to a specific answer. An example of a leading question is: "You would not say that you were in favour of capital punishment, would you?"
• Try to minimize threatening questions that the respondent may find embarrassing and difficult to answer.
• Avoid double-barrelled questions; those that combine two or more questions in one.
• An expert is likely required to conduct questionnaires that use sophisticated methods.