This document contains the entire text of the policy as
revised on April 1, 2001. This policy replaces Chapters 1-1, 1-2,
3-1 and 3-2 of the "Review" volume of the Treasury Board
Manual dated July 31, 1994.
This policy supports the generation of accurate, objective and
evidenced-based information to help managers make sound, more
effective decisions on their policies, programs and initiatives
and through this provide results for Canadians.
This policy clarifies the role of evaluation within the
government's management framework.
The policy is based on three fundamental principles:
- first, that achieving and accurately reporting on results is
a primary responsibility of public service managers;
- second, that rigorous and objective evaluation is an
important tool in helping managers to manage for results; and
- third, that departments with the support of the Treasury
Board Secretariat, are responsible to ensure that the rigour and
discipline of evaluation are sufficiently deployed within their
jurisdictions.
Managing for Results
Managing for results is the prime responsibility of public
service managers. As outlined in the management framework for the
federal government, Results for Canadians, public service
managers are expected to define anticipated results, continually
focus attention towards results achievement, measure performance
regularly and objectively, and learn and adjust to improve
efficiency and effectiveness.
Managers must be accountable for their performance to higher
management, to ministers, to Parliament and to Canadians.
Evaluation - like internal audit, risk management capacity and
other management tools - helps managers to operate effectively in
this environment. Evaluation can support managers' efforts to
track and report on actual performance and help decision-makers
objectively assess program or policy results. This distinguishes
evaluation from internal audit - a function that provides
assurances on a department or agency's risk management strategy,
management control framework and information, both financial and
non-financial, used for decision-making and reporting.
Evaluation as a Management Tool
Evaluation has two main purposes:
- to help managers design or improve the design of policies,
programs and initiatives; and
- to provide, where appropriate, periodic assessments of policy
or program effectiveness, of impacts both intended and
unintended, and of alternative ways of achieving expected
results.
Evaluation operates in a complex environment that involves
partnerships with other federal organisations, with other levels
of government, with the private sector or with not-for-profit
entities. In addition, as the delivery of programs is devolved to
other jurisdictions, the evaluation of underlying policies
increases in importance. Evaluation should contribute to
improvements in policy, as well as program design and
delivery.
In this milieu, departments should embed the discipline of
evaluation into the lifecycle management of policies, programs
and initiatives to:
- develop results-based management and accountability
frameworks for new or renewed policies, programs and
initiatives;
- establish ongoing performance monitoring and performance
measurement practices;
- evaluate issues related to the early implementation and
administration of the policy, program or initiative, including
those that are delivered through partnership arrangements
(formative or mid-term evaluation); and
- evaluate issues related to relevance, results and
cost-effectiveness.
Departments should undertake an appropriate balance of
evaluation work. To achieve this balance, evaluators should
develop a strategically focussed plan that is based on
assessments of risk, departmental priorities and priorities of
the government as a whole.
Effective Deployment of Evaluation
The Government of Canada is committed to becoming a learning
organisation. Evaluation supports this aim by helping to find out
what works and what does not, and by identifying cost-effective,
alternative ways of designing and improving policies, programs,
and initiatives.
The success of evaluation depends on a number of important
factors. It requires clarity of roles, application of sound
standards, ongoing support for rigorous, professional practice,
and developing a conducive environment where managers embed the
discipline of evaluation into their work. The organisational
positioning of evaluation should reflect the unique needs of the
department. Evaluation discipline should be used in synergy with
other management tools to improve decision-making. Heads of
evaluation should work to ensure that evaluation in their
organisation is healthy in respect of all these factors.
The Treasury Board Secretariat, in its role as the
government's management board, should support the practice of
evaluation in departments by providing advice on best practices,
setting standards, monitoring the evaluation capacity in
departments and using the products of evaluation to inform
decision-making at the centre.
To ensure that the government has timely, strategically
focussed, objective and evidence-based information on the
performance of its policies, programs and initiatives to produce
better results for Canadians.
It is government policy that departments and agencies embed
evaluation into their management practices:
- to help design policies, programs, and initiatives that
clearly define expected results and that embody sound performance
measurement, reporting and accountability provisions at their
outset; and
- to help assess in a rigorous and objective manner the results
of government policies, programs, and initiatives, including
their impacts both intended and unintended, and alternative ways
of achieving expected results.
This policy applies to all organisations considered to be
departments within the meaning of section 2 of the Financial
Administration Act.
Deputy heads must establish an appropriate evaluation
capacity, tailored to the needs and resources of the department,
to evaluate policies, programs and initiatives - including those
of an inter-organisational nature. To that end they should:
- appoint a senior head of evaluation to conduct strategically
focussed evaluation studies and work with managers to embed the
discipline of evaluation into departmental management practices;
- establish an evaluation committee and designate a senior
departmental executive to chair it; and
- ensure that, in addition to accountability practices internal
to the department, the Treasury Board Secretariat is given access
to evaluation plans and early warning of evaluation findings that
indicate major concerns respecting the management or
effectiveness of policies, programs and initiatives.
Departmental heads of evaluation must provide
leadership and direction to the practice of evaluation in the
department and, to that end, should:
- ensure that strategically focussed evaluation plans - founded
on assessments of risk, departmental priorities, and priorities
of the government - appropriately cover the organisation's
policies, programs and initiatives;
- work with managers to help them enhance the design, delivery
and performance measurement of the organisation's policies,
programs, and initiatives;
- conduct evaluation studies of policies, programs and
initiatives in accordance with established plans;
- inform senior management and the appropriate departmental
players (e.g. internal audit) without delay of any findings that
indicate major concerns respecting the management or
effectiveness of policies, programs, or initiatives;
- make completed evaluation reports available to the Treasury
Board and to the public with minimal formality in both official
languages; and
- apply the evaluation standards appended to this policy.
Departmental managers must manage for results and to
that end, should:
- draw on the organisation's evaluation capacity, where
appropriate, and ensure that they have reliable, timely,
objective and accessible information for decision-making and
performance improvement; and
- incorporate approved evaluation findings and measures for
improvement into priority setting, planning, reporting and
decision-making processes.
Treasury Board Secretariat must provide central
direction for evaluation in the Government of Canada and, to that
end, should:
- establish a Centre of Excellence to provide leadership,
guidance and support to the practice of evaluation;
- use evaluation results where appropriate in decision-making
at the centre;
- set standards; and
- monitor evaluation capacity in the government.
The Treasury Board Secretariat will monitor this policy to
ensure its success in meeting its intended objectives.
To monitor the policy, the Treasury Board Secretariat, in
consultation with its stakeholders, will develop a results-based
management and accountability framework that sets out, among
other things, the objectives and key results to be achieved and
the performance measurement strategy.
The Treasury Board Secretariat will conduct an evaluation of
this policy within five years of the coming into force of the
policy.
In monitoring the implementation of this policy, the Treasury
Board Secretariat will be guided by the requirements of the
Evaluation Standards for the Government of Canada, appended to
this document.
Financial Administration Act
Access to Information Act
Privacy Act
Results for Canadians: A Management Framework for the
Government of Canada
Official Languages Act
The Social Union Framework Agreement
Transfer Payments Policy
Study of Program Evaluation in the Federal Government
Managing for Results
Modernisation of Comptrollership in the Government of
Canada
Communications Policy of the Government of Canada
Alternative Service Delivery Policy Framework
This policy and related standards supersede Chapters 1-1, 1-2,
3-1 and 3-2 of the "Review" volume of the Treasury Board
Manual dated July 31, 1994.
Enquiries about this policy should be directed to the
Planning, Performance and Reporting Sector of the Treasury Board
Secretariat.
Approved (or completed) (approuvé (ou
réalisé)) - when referring to an evaluation
product means approved by the organisation's evaluation
committee.
Department (or departmental) (Ministère (ou
ministériel)) - includes a department, agency or any
other organisation covered by this evaluation policy.
Inter-organisational (Inter-organisationnels) -
refers to inter-departmental, inter-governmental, or other
relationships, including those with the private or not-for-profit
sectors.
Monitor (Surveiller) - means to observe and
oversee and if necessary provide warning.
Performance (Rendement) - means actual
achievements measured against defined goals, standards or
criteria.
Performance measurement (Mesure du rendement) -
is the ongoing monitoring of the results of a program, policy or
initiative, and in particular, progress towards pre-established
goals.
Results (Résultats) - relate to what was
achieved. They are the collection of impacts and outcomes
associated with a program, policy or initiative.
Results-based management and accountability frameworks
(Cadres de gestion et de responsabilisation axé sur
les résultats) - are a mandatory requirement for all
transfer payment programs (Reference: Policy on Transfer
Payments, June 2000) and are commonly required by Treasury Board
in the approval of new or renewed programs and initiatives. They
generally include:
- a clear statement of the roles and responsibilities of the
main partners involved in delivering the policy, program or
initiative;
- a clear articulation of the resources to be applied and the
objectives, activities, outputs and key results to be achieved,
along with their linkages;
- an outline of the performance measurement strategy, including
costs and performance information (key indicators) that will be
tracked;
- the schedule of major evaluation work expected to be done; and
- an outline of the reporting provisions as appropriate for
funding recipients and those for the department, including
parliamentary reporting.
Introduction
The government's management framework, Results for Canadians,
commits managers to sharpening their citizen focus, to applying
sound public service values, to managing for results and to
spending in a responsible manner.
Managers have a responsibility to monitor the performance of
policies, programs and initiatives - to make sound decisions and
to report on performance to higher departmental authorities, to
Parliament and to the Canadian public.
Evaluation is a management tool that can operate throughout
the lifecycle of a policy, program or initiative. It can help
managers design and implement reliable performance measurement
systems. It can also periodically assess effectiveness in
achieving objectives, impacts, both intended and unintended,
continued relevance and alternative ways of achieving expected
results.
Treasury Board policy is that departments are responsible for
establishing and supporting evaluation capacity that is
appropriate to their needs and is capable of fulfilling these
responsibilities in a timely and professional manner.
These standards provide clear expectations for evaluation in
an environment, where policies, programs and initiatives often
involve partnerships with other federal organisations, with other
levels of government or with the private or not-for-profit
sectors. Evaluation should contribute to improvements in policy,
as well as in programs and on a departmental as well as
inter-organisational basis.
By using common criteria, the standards provide departments
with a basis for improving the quality of evaluation practice.
They also provide the Treasury Board Secretariat with a basis for
monitoring the implementation of its evaluation policy.
Evaluation Standards
Evaluation Planning and Issues
Standard
The department must apply the discipline of evaluation to
assess the performance of its policies, programs and initiatives,
both departmental and inter-organisational, taking into account
its priority concerns as well as those of its partners and the
government as a whole.
Guidance
Evaluators should develop a strategically focussed plan that
is based on assessments of risk, departmental priorities and
reporting requirements, and priorities of the government as a
whole.
The full range of evaluation issues should be considered at
the planning stage of an evaluation:
- does the policy, program or initiative continue to be
consistent with departmental and government-wide priorities and
does it realistically address an actual need? (relevance);
- is the policy, program or initiative effective in meeting its
objectives, within budget and without unwanted outcomes?
(success); and
- are the most appropriate and efficient means being used to
achieve objectives, relative to alternative design and delivery
approaches? (cost-effectiveness).
Evaluators should address issues that are needed for
accountability reporting, including those involving key
performance expectations (a) identified and conveyed to the
Treasury Board or (b) resulting from Cabinet decisions requesting
evaluation information.
Competency
Standard
The person or persons carrying out evaluations, or evaluation
related work, must possess or collectively possess the knowledge
and competence necessary to fulfil the requirements of the
particular evaluation work.
Guidance
Evaluators should possess or ensure the provision of content
knowledge appropriate for the evaluation and continuously strive
to improve their methodological and practice skills. Evaluators
should possess the knowledge, skills and experience in:
- the application of sound research design able to answer the
chosen questions;
- the collection and analysis of reliable quantitative and
qualitative data; and
- the development of valid, credible and unbiased conclusions
and recommendations.
Objectivity and Integrity
Standard
Individuals performing evaluation work must be free from
impairments that hinder their objectivity and must act with
integrity in their relationships with all stakeholders.
Guidance
Evaluators should:
- accurately represent their level of skills and knowledge; and
- declare any matter that could compromise the objectivity of
either evaluator or stakeholder before embarking on an evaluation
project or at any point during the project.
- evaluators should be accountable for their performance and
their products and be responsible for:
- ensuring that their work addresses the priority concerns and
accountability requirements of departmental management and the government;
- conferring with stakeholders on decisions such as
confidentiality, privacy, communications and ownership of
findings and reports;
- ensuring sound fiscal decision-making so that expenditures
are accounted for and clients receive good value for their
dollars; and
- completing evaluation work within a reasonable time as agreed
to with the clients.
Consultation and Advice
Standard
Evaluation work must incorporate sufficient and appropriate
consultation and, where appropriate, apply the advice and
guidance of specialists and other knowledgeable persons.
Guidance
Evaluators should consult major stakeholders in the conduct of
their work.
Where appropriate, peer review groups should be organised to
review evaluation products to improve their quality and enhance
the sharing of best practices.
Measurement and Analysis
Standard
Evaluation work must produce timely, pertinent and credible
findings and conclusions that managers and other stakeholders can
use with confidence, based on practical, cost-effective and
objective data collection and analysis.
Guidance
Evaluation products should be made available at the most
appropriate time to aid management decision-making.
Evidence should be sufficient in relation to the
decision-making context. Evaluation findings should be relevant
to the issues addressed and follow from the evidence.
Evaluation products should be demonstrably useful to managers
in improving performance and reporting on results achieved.
In planning and conducting the evaluation, the department
should comply with government policies related to information
collection, use, preservation and dissemination.
Reporting
Standard
Evaluation reports must present the findings, conclusions and
recommendations in a clear and objective manner.
Guidance
Evaluation reports should be written so that senior managers
and external readers can readily focus on and understand the
important issues being reported. They should:
- be concise and clearly written;
- include only information that is needed for a proper
understanding of the findings, conclusions and recommendations;
- present the conclusions and recommendations so that they flow
logically from evaluation findings;
- clearly expose the limits of the evaluation in terms of
scope, methods and conclusions; and
- satisfy, where applicable, Cabinet, Treasury Board submission
or external reporting requirements.
Additionally, evaluation reports should:
- provide the reader with appropriate context by describing the
objectives and timing of the work, the policy, program or
initiative that was evaluated, how it fits into the overall
operations of the organisation, and its importance;
- provide an accurate assessment of the results that have been
achieved by the policy, program or initiative;
- contain clear and actionable recommendations, and timing for
management action; and
- provide relevant analysis and explanation of the exposure to
risks for any significant problems identified and in respect of
key recommendations.
|