Home : Reports and Publications : Audit & Evaluation : Risk-Based Review of WD’s Project M&P; Function - Nov 2003
Recommendations
The recommendations presented in this section are intended to address control deficiencies noted in Appendix A. Each deficiency in the appendix is cross-referenced to one of these recommendations.
A. Department-Wide Risk Management Framework
WD should develop a department-wide risk management framework.
The risk framework must provide guidance around the level of risk that WD is willing to tolerate, and provide input to the development and implementation of RBAFs, and specific monitoring and claim verification standards and activities.
WD has started to develop an integrated risk management framework as required by Treasury Board. The action plan in the next section provides the status of this initiative.
B. Project Performance Measurement Function
WD should design a project performance measurement function.
In designing the function, WD must assign responsibilities for defining, monitoring and measuring project outcomes. In doing so, WD should assess the anticipated work effort and skill requirements associated with measuring project performance and make any organizational changes necessary.
Risk is not a major factor in determining which departmental unit should be responsible for project performance measurement. Each option WD considers will have inherent risks that must be mitigated with appropriate controls. By way of example, the following table presents three possible options for allocating responsibility for performance measurement, potential benefits, inherent risks and mitigating controls. The table is not intended to be a comprehensive analysis of the options WD should consider, nor of the inherent risks and required controls associated with each.
| Options: Unit Responsible |
Possible Benefit or Rationale | Example Inherent Risks | Possible Mitigating Controls |
|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
Additional options could be derived by considering combinations of the options presented.
In addition to assigning responsibility for project performance measurement, and addressing organizational implications, WD must also develop specific processes and systems to support performance measurement activities.
C. Implementation of RMAFs and RBAFs
WD should complete the development and implementation of RMAFs and RBAFs for all its programs.
In completing and implementing the RMAFs and RBAFs, WD should consider combining similar programs (e.g. WDP and ICIP) under shared RMAFs and RBAFs. WD should define specific monitoring and auditing requirements for each program and incorporate these requirements into documented M&P policies and procedures.
D. Standards for Monitoring and Claims Verification
WD should define standards for monitoring and claims verification activities.
For each program and sub-program, WD should define specific standards relating to the amount and nature of monitoring and claims verification required. The standards should be based on WD's corporate risk profile, program RBAFs and the need for efficiency. The standards must be incorporated into documented policies and procedures for M&P.
Monitoring activities should be driven by a monitoring plan prepared at the beginning of each project based on the nature and risk of the project. Monitoring activities must be documented to provide WD with a record in the event of client disagreements or staff turnover, and to formally assess the effectiveness of such activities.
E. Standard Policies and Procedures for M&P
WD should develop and maintain standard policies and procedures for M&P across all regions.
WD should review current processes to ensure that they are efficient and serve the interests of WD with respect to client service, support to other departments, and risk management.
As part of reviewing its processes, WD should review the necessity and appropriateness of all forms used by M&P and revise and standardize these forms as appropriate. In particular, WD should revisit the risk assessment form to ensure that it is useful and necessary to use for each claim, and that the risk elements and scores accurately predict the level of risk associated with each project.
Policies and procedures must reflect central agency regulations and guidelines, RBAFs and RMAFs and WD's interpretation of such documents in the form of risk tolerances and acceptable practices and standards.
Once documented, WD should distribute its policies and procedures along with supporting forms and tools using a central Intranet site to ensure consistency and currency across all regions.
F. M&P Involvement in Reviewing Contribution Agreements
WD should clarify the intent and process for involving M&P in the review of draft contribution agreements.
WD should review current regional practices to involve M&P in the review of contribution agreements (e.g. buddy system, project review committees, M&P-developed training for Development and Assessment officers) and agree upon best practices. Director Generals in each region need to clarify and communicate M&P's role (e.g. one of advising on the structure and wording of Attachment A, and mentoring/training Development and Assessment Officers) to both the Development and Assessment, and M&P officers. WD should incorporate the review process formally into the procedures for both Development and Assessment, and M&P.
G. Training Requirements
WD should identify and address training requirements for M&P.
WD should determine the specific training requirements for M&P staff, and provide training, as needed, on the new RMAFs and RBAFs, central agency regulations and guidelines (for new staff and for significant changes), and M&P policy and procedural changes.
In addition, WD should provide training to both Development and Assessment, and M&P officers on preparing Attachment A's. This would reduce the former group's reliance on M&P and help ensure greater consistency between regions and M&P officers in their interpretations of such policies as appropriate contribution stacking and in-kind contributions.
Training should be centrally coordinated to avoid duplication in effort between regions.
H. Central Coordination of M&P
WD should define and implement a central coordinating role for the M&P function.
WD should consider dedicating a headquarters resource to overseeing the M&P function, or coordinating the function using an M&P management committee or a rotating unit chair. Regardless of the mechanism used, the central body must have sufficient time and resources to ensure that the M&P function performs efficiently and effectively across all regions, and that it keeps current with departmental changes. Initially, the required effort will be greater until the recommended changes are addressed.
At a minimum, the central role should be responsible for:
- Reviewing and redesigning as necessary the processes used by M&P;
- Ensuring consistent policies and procedures, forms and practices are used across all regions;
- Making resources available to all regions (e.g. audit logs, training materials, policy clarifications and precedents);
- Determining and addressing the training requirements of M&P;
- Defining required position competencies for M&P staff;
- Helping the regions with succession planning and the determination of appropriate staffing levels and mix; and
Monitoring the M&P function through the use of performance measures, quality assurance reviews and internal audits.
I. Operating Standards and Measures
WD should develop operating standards and performance measures for M&P.
WD should develop metrics, standards and measures to manage the performance of M&P, identify regional best practices, identify potential process weaknesses and assess staffing levels.
Operating metrics should include measures of M&P activity (e.g. number of claims, files, site visits, audits). Performance standards and measures should include client service measures, turnaround times, productivity/efficiency measures, and impacts on risks (e.g. incident tracking).
J. Quality Assurance Review Process
WD should implement a quality assurance review process and conduct periodic internal audits of M&P.
WD should design and implement a formal quality assurance review process to ensure consistent application of policies and procedures throughout the regions, and to identify training requirements and best practices. The M&P quality assurance review process should be integrated with a broader review process that includes all stages of the project lifecycle.
WD should also request periodic internal audits of the M&P function to test compliance with regulations and policies, and to verify the effectiveness of controls.
K. Service Delivery Partners
Address the recommendations from the Audit of the Community Futures Program (Grant Thornton, 2003) and consider their applicability to all types of service delivery partners.
As it renews terms and conditions over the next year governing its contributions under the Service Delivery Network Program, Community Futures Program, and the Women's Enterprise Initiative, WD must address the recommendations outlined in the audit of the Community Futures Program. In particular, it should give consideration to:
- The extent to which it can rely on external auditors to provide assurance that service delivery partners are complying with the terms and conditions; and
- Standardizing performance reporting from service delivery partners, and verifying the accuracy and reasonableness of information submitted.
L. Management Information
WD should improve its management information reporting capabilities.
WD should review the capabilities of its management reporting systems (including "switched off" functionality) against management's information requirements. WD should consider developing reports suited to management's needs such as exception reporting, comparison and trend reporting, summary reporting, and ad hoc reporting and analysis. In doing so, WD must address information requirements that use data beyond that contained in GX (e.g. performance reporting, operational reporting).
M. Project Management System
WD should consider implementing a project management system.
WD should assess the costs and benefits associated with implementing a department-wide, project management system that would support its operations more fully.
Functionality of a project management system might include:
- Contact tracking;
- Tracking of requests/proposals/applications processed (including rejected);
- Automated workflow routing and on-line approvals;
- Workflow tracking (e.g. processing times, backlogs);
- Document management;
- Monitoring activity planning, prompting and recording; and
- Performance measurement tracking.
N. M&P Structure and Staffing
WD should review the structure of M&P, and its staffing levels and mix.
WD should review the organization structure of M&P, its required staffing levels and its resource mix following the development of an appropriate mandate and policy framework for the department and an associated redesign and standardization of M&P processes.
WD should prepare a succession plan to ensure that retiring staff are identified sufficiently early to hire replacements and allow for a smooth transition.
WD should review the use of full-time contractors in BC following decisions regarding the appropriate number and nature of required M&P staff.