Thursday, September 15th, 2005
Agenda
Opening Remarks
David Moloney,
Senior Assistant Secretary,
Expenditure Management Sector, Treasury Board of Canada Secretariat (TBS)
- Provided update on the status of the reorganization at TBS that has
resulted in the Expenditure Management Sector being the new home for the
CEE. This sector now houses all budget management divisions, as well as the
Results-based Management Directorate. The Sector will perform the role of
“Budget Office”, and is taxed with the responsibility of the entire
management cycle from A to Z.
- Key roles for evaluation in the Budget Office: evaluation’s role in
expenditure management decision making, as well as supporting ongoing
accountability for results. Evaluators are key players from the point of
view of the Budget Office.
- It is critical that managers and policy planners know that their programs
are achieving objectives – evaluators are also key to the program design
stage to ensure measurable objectives are in place.
- Discussion of Expenditure Review – There will be ongoing phases that
will help decision makers do better with the money they have. Evaluation
will be a critical part of the next phase.
- Priorities – need to start with renewing and strengthening the
evaluation policy, this will be a commitment of the Management Agenda to be
presented this Fall by the TB President; need for re-orienting the
evaluation function and its place in expenditure management (“Phase II”
of ERC)
- On the relative roles of audit and evaluation – audit has been under the
political limelight and has received more resources. With the new Management
Agenda, this will remain an emphasis of TB. Evaluation however must be
supported as both complementary and a distinct pillar of management –
evaluation isn’t going away.
- Policy renewal must also support quality and independent evaluations. Need
greater clarity and transparency re: the Management Agenda; importance of
horizontal evaluation – independence must be strengthened, but evaluators
can’t work in an isolated manner.
Questions/comments from the audience:
- Thoughts on the new internal audit policy and implications for evaluation:
- Is not fully aware of all details of the audit policy. Realizes that
there are similar issues from the perspective of evaluation – best
practice questions such as whether there should be two distinct
committee structures governing functions in departments, participation
of external members on committees, having an external chair.
- Would like to have the evaluation function more integrated within the
department; to be able to learn from audit is important, but too often
it is defensive – how much separation of Church and State do we need
for evaluation. We need independence, but there could be a different
approach equally justifiable.
- What is his sense for how the policy review will be undertaken?
- Importance of communication with the evaluation community, must not
forget to consider people’s multiple responsibilities.
- Thoughts on review of the evaluation function:
- Need to take a top down look to see how evaluations are used, as well
as a “plumbing and wiring” investigation to observe operations of
the function.
- Timeframe for when TBS might begin review of function and practice –
Pre-ERC, 9 reviews were launched, followed by other horizontal reviews
to determine production function side. By winter, to provide a schedule
of reporting on reviews as well as others to follow.
- Will TBS be taking on a larger role in assisting/directing horizontal
evaluations?
- We need to work on best practices and tools for conducting horizontal
evaluations. However, ownership of product belongs with departments. TBS
could do it, or the department could do it, or a team approach could be
used, but ownership should lie with the department
Government of Canada Evaluation Plan:
Demonstrating Value for Money and Accountability for Results, 2005-2006
Lee McCormack
Executive Director,
Results-based Management Directorate, TBS
Response to David’s comments, re: policy renewal:
- Definitely a need for change, as the world has changed since the policy
was established in 2001
- Still in process of developing a workplan for the policy renewal.
On the reorganization and the new Budget Office:
- Bringing better performance information to better decision making.
Presented his deck “Government of Canada
Evaluation Plan, 2005-06 Re-orienting and Strengthening the Evaluation Function”.
Questions/comments from the audience:
- Major tool for reporting is the DPR, however the agency is experiencing
pressure to change wording and to remove less than perfect information. TBS
needs a balanced report, but such a report doesn’t go through the system
well.
- Must respect the fact that the Minister signs the document. We’ve
been preaching to cabinet that it’s better to be frank and open about
achievements, but this is theoretical, because at the end of the day you
hope there will be accountability for this.
Evaluation Policy Renewal
Client Satisfaction Survey of the CEE: Summary Findings and Implications for
Evaluation Policy Renewal
Pierre Lacasse
Director General,
Evaluation and Internal Audit, TBS
- Background information provided: Internal Audit and Evaluation Division at
TBS, at the request of the Secretary of the Treasury Board, was asked to
oversee the evaluation of the CEE to assess how well CEE clients have been
served and the impacts of a reduction or discontinuation of CEE operations
- Methodology: 27 interviews with departmental heads of eval, 11 interviews
with TBS representatives
- Key findings regarding varying levels of satisfaction among clients (small
agencies and new heads find CEE more useful than more established
functions); leadership viewed as energetic and committed; level of
satisfaction correlated to CEE analyst; need to strengthen in-house
technical experience
- Feedback from TBS program sectors was discussed
- Future roles for TBS in evaluation were proposed, based on strong support
among heads of evaluation and within TBS program sectors for TBS to continue
to provide a leadership role for evaluation across the federal government.
- Roles that are suggested include: continue to support and strengthen the
policy, set standards for monitoring overall performance the evaluation
function, support community building, focus on capacity development, and
coordinate horizontal evaluations for major policy issues.
Impacts of Evaluation on Executive Decision-Making and Implications for
Evaluation Policy Renewal: Overview of Findings of Ad-hoc Evaluation Community
Working Group
Elizabeth Murphy-Walsh
Director General,
Review Services, Department of National Defence
Presented her deck “Impacts of Evaluation on
Executive Decision-Making and Implications for Evaluation Policy Renewal:
Overview of Findings of Ad-hoc Evaluation Community Working Group”.
Impacts of RMAFs on Executive Decision-Making and Implications for
Evaluation Policy Renewal
Rebecca Leslie
Policy analyst,
Centre of Excellence for Evaluation (CEE), TBS
Presented her deck “Implementation of RMAFs in the
Government of Canada: Status Report”.
Questions/comments from audience:
- Advice from Health Canada and CIDA in terms of establishing a clear RMAF
liaising unit, to work with program managers as well as TBS to support
timelines, sign off processes, etc.
- Concern regarding sign off by heads of evaluation: situation has emerged
where a head has signed off, but TBS analyst had comments on RMAF that
required changes by program manager. This seriously challenges credibility
of evaluation within the department
- Response: this is a case by case situation; practical experience is
showing to us that many heads are not signing off on RMAFs before copies
are sent to analysts for review; trying to follow up with this sign off,
but departments are awaiting final comments before sign off occurs; need
to work better together on this point, and re-communicate expectations
- Concern regarding the recommended role of evaluation services as providing
quality control function on RMAF: how can head of evaluation effectively
provide quality control while ensuring that the program manager maintains
ownership of the document?
- Perhaps the wording is strong, but there is an expectation that
evaluation heads must be involved to review and assess the quality of
the RMAF before it is seen by TBS analysts
Update on the Policy on Transfer Payments Renewal and Consultation Processes:
Implications for Evaluation Policy Renewal
Pierre Laflamme
Director,
Financial Management Policy, TBS
Presented his deck “Renewal of the Transfer
Payment Policy”.
Management Accountability Framework: Implications for Evaluation and
Managing for Results and Update on MAF 2005-2006
Ivan Blake
Executive Director,
MAF Development and Implementation,
Corporate Priorities and Planning Sector, TBS
- Highlights of changes to the MAF process: new database now on-line, with
historical information on assessments; timelines for current process (new
feature of departmental input into data)
- Priority will be to make this a focus for departments, no longer a TBS
exercise
- New indicators have been included (nothing changes for evaluation); still
aiming to reduce the total to 12 (currently close to 40)
Overview of Findings of Evaluation Community Studies and Implications for
Evaluation Policy Renewal
Terry Hunt
Senior Director,
CEE, TBS
Last three slides of his deck “Renewal of the Transfer
Payment Policy” were discussed.
- Emphasised the role of evaluation and the mandate to renew the TB
Evaluation Policy and reorient it to better meet deputy and central agency
information needs on value-for-money
- Discussed potential approaches to policy review and asked for volunteers
to sit on a variety of policy for a, including interested ADMs to sit on an
ADM advisory committee.
|
