Appendix 5

The Purpose Of Evaluation

The purpose of evaluation is to Ainfluence the actions and activities of individuals and groups who have, or are presumed to have, an opportunity to tailor actions on the basis of the results of the evaluation effort"¹. Evaluations generally focus on the operation and the outcome of programs or projects and seek to answer a standard set of fundamental questions. Most program evaluations also seek to be participatory in nature.

What is Participatory Evaluation?

The Health Canada Guide to Project Evaluation: A Participatory Approach² provides the following principles for participatory evaluation, and many are applicable to Care for the Caregiver:

• Participatory evaluation focuses on learning, success and action. A key question to ask in an evaluation is what we learned about what worked and what did not work. These questions are appropriate to ask to the Care for the Caregiver workshop participants because they are caregivers. Their responses can be used to inform future workshops.
• The evaluation is useful to the people who are doing the work that is being evaluated. The program's goals and objectives B what the program intends to accomplish B must be the standards against which the program work is measured. Attention must be paid to the program=s specific needs and available resources.
• The evaluation process is ongoing and includes ways to let all participants use the information from the evaluation throughout the program, not just at the end. Since there are a series of six workshops in the Care for the Caregiver program, it is important that each workshop is monitored with participant evaluations.
• Recognition of the progression of change B in knowledge, attitudes, skills and behaviour B is built into the evaluations. It is important to think in advance about the ways that the program strategies and activities can lead to changes in people's knowledge, attitudes, skills and behaviour; and, to consider changes that can be recognized and measured in a practical manner and within the timeframe and resources available to the program.
• The project/program sponsors are responsible for defining the specific project/program evaluation questions, the indicators of success and realistic timeframes. Ideally, the sponsor of the program (i.e., Care for the Caregiver) would be active in the evaluation process in several ways. First, identifying the expected project outcomes based on stated objectives and goals; second, selecting indicators that will provide a measure of the success in meeting the objectives, third, determining the appropriate and feasible method of data collection and measurement (i.e., what questions to ask, to whom, how and when); and ensuring appropriate analysis and dissemination of results.
• Participatory evaluation makes it possible to recognize shared interests among those doing the work, the people the work is designed to reach, the program funders and other stakeholders.

What are the Fundamental Questions of Evaluation?

All evaluations attempt to answer a set of five fundamental questions. These questions are listed below:

• What?
  Did we do what we said we would do?
• Why?
  What did we learn about what worked and what didn't work?
• So what?
  What difference did it make that we did this work?
• Now what?
  What could we do differently?
• Then what?
  How do we plan to use evaluation findings for continuous learning?

What are the Common Types of Evaluation?

Two common types of evaluation are process evaluation and outcome evaluation.

1. Process Evaluation

A process evaluation looks at the operation of a program. For the Care for the Caregiver program this requires that the participants evaluate each of the workshops as they are conducted. The timely analysis of these evaluation results will enable facilitators to become aware of any participant concerns as they occur, and to address them, if feasible, before the next workshop in the series.

1. Outcome Evaluation

An outcome evaluation determines if the program goals and objectives have been met with the anticipated outcomes (as well as identify any unexpected outcomes). An outcome evaluation is a source of valuable information for workshop facilitators, the program team and the funder.

1. Some Implications of Evaluation

Not all questions can be answered given the program timeframe and the available resources. Often, the stated program goal is very broad and given the timeframe for the program it is not possible to determine if it has been achieved. For example, a broad objective of Care for the Caregiver is to improve the well being of caregivers. However, to determine if there has been any change in quality of life of caregivers as a result of the information and skills acquired at the workshops would require follow-up surveys or interviews in the community several months after the program's completion.

What are the Expectations of the Care for the Caregivers Facilitators Related to Evaluation?

Briefly, the expectations of the Facilitators in the evaluation of Care for the Caregivers include:

• Collect information on workshop participants, i.e. the Caregiver Questionnaire.
• Evaluate each workshop. Hand out (and explain) the evaluations at the end of each workshop. Collect the completed evaluations from all participants and compile the results. Read and act on workshop evaluation feedback, if feasible and appropriate, prior to the next workshop.
• Evaluate the overall Care for the Caregiver program. Hand out the evaluation forms to all participants, collect the completed forms and compile results.
• Submit the participant questionnaires, all completed evaluations and the compiled results to the sponsors and/or retain with the workshop documents.

¹Peter H. Rossi and Howard E. Freeman. Evaluation: A Systematic Approach.1993, New York: Sage, p. 46.

²Excerpts from Guide to Project Evaluation: A Participatory Approach.1996, Ottawa: Population Health Directorate, Health Canada, p. 6.