Western Economic Diversification Canada | Diversification de l'économie de l'Ouest Canada

Home : Reports and Publications : Audit & Evaluation : Risk-Based Review of WD’s Project M&P; Function - Nov 2003

Appendix A: Voting Results by Risk (Cont'd)

Group: Performance Management (Rated as a group)

Risks

  1. WD Programs are not achieving intended outcomes.
  2. Project planned (DDR) and actual outcomes not linked to program objectives
  3. Insufficient focus on program results and outcomes
  4. Lack of appropriate management information reports on program performance and outcomes.
 

Risk Rating:

Critical
Risk Assessment
  • Performance measurement is a new area for WD and it has not been fully implemented.
  • Establishment of planned project outcomes is the responsibility of Development and Assessment
Controls Ref. Description
12. Terms and conditions now include planned project outcomes and measures
Deficiencies Description Rec.
WD has not formally assigned project evaluation and outcome tracking activities to any organizational unit
 B
Results-Based Management and Accountability Frameworks only approved for two programs
 E
Project Development Officers and M&P Officers not formally trained on new Results-Based Management and Accountability Frameworks
 G
WD does not have a system to track and aggregate performance-related information
 B

 

Group : Management and Coordination

Risk

  1. M&P approach jeopardizes client relationships with WD.
 

Risk Rating:

Mod.
Risk Assessment
  • Most concerns cited with respect to client concerns related to claims not being processed promptly or inconsistent messages with respect to allowable costs and required documentation. These concerns largely stem from ambiguities in WD's policies and the Attachment A's of projects
  • Claim processing activities are not tracked (e.g. average processing time, backlog, productivity, pending and rejected claims) to quantify impacts on client service
Controls Ref. Description
6. Terms and conditions outline project objectives and eligible expenses
Deficiencies Description Rec.
WD does not formally track client satisfaction or complaints
 I
M&P lacks operating standards and measures by which it can monitor its efficiency
 I
M&P lacks any current and specific policies and procedures to govern its activities and processes in accordance with regulations and WD interpretations E

 

Risk

  1. Inconsistencies in M&P processes across regions and programs (e.g. risk assessments, project monitoring, claim verification, advance payments, project evaluations).
 

Risk Rating:

High
Risk Assessment
  • There is limited, informal coordination provided by headquarters, but no dedicated leadership role or accountability
Controls Ref. Description
  No department-wide, formal and current controls exist
Deficiencies Description Rec.
There is no central coordination of the M&P function
 H
M&P follows different procedures and policies (e.g. audit policy, thresholds for supporting claims documentation) between regions
 E
M&P lacks any current and specific policies and procedures to govern its activities and processes in accordance with regulations and WD interpretations
 E
There is no training of M&P staff on regulations and WD interpretations, including on-going training on changes
 G
There is no quality assurance review process or internal audit of the M&P function
 J

 

Risk

  1. Structure of M&P does not promote communication and collaboration with Project Development Officers and Finance.
 

Risk Rating:

Mod.
Risk Assessment
  • Most interviewees do not believe that there is anything intrinsic about the structure that prevents M&P from working effectively with Project Development Officers and Finance
  • Some regions have implemented steps to encourage communication (e.g. a "buddy system" and regularly scheduled meetings)
Controls Ref. Description
  No department-wide, formal and current controls exist
Deficiencies Description Rec.
No systemic deficiency noted  

 

Risk

  1. RMAF's & RBAF's (for ICIP and WDP) not clearly understood and effectively utilized.
 

Risk Rating:

High
Risk Assessment
  • The RMAF's and RBAF's were only recently approved
Controls Ref. Description
  No department-wide, formal and current controls exist
Deficiencies Description Rec.
There is no central coordination of the M&P function
 H
There is no training of M&P staff on regulations and WD interpretations, including training on the new RBAF's and RMAF's
 C
Results-Based Management and Accountability Frameworks and Risk-Based Audit Frameworks are not incorporated into M&P policies and procedure (e.g. project audit practices not defined, project evaluation responsibility not assigned) C

 

Risk

  1. QAR process not effective
 

Risk Rating:

High
Risk Assessment
  • QAR process was not designed to be a quality assurance review process. It refers primarily to the checklists to ensure proper documentation exists on file
  • The QAR checklists are used inconsistently between regions and files, and in many cases, are completed after the project has been completed rather than as it progresses
Controls Ref. Description
  No department-wide, formal and current controls exist
Deficiencies Description Rec.
There is no central coordination of the M&P function
 H
There is no quality assurance review process or internal audit of the M&P function J

 

Risk

  1. File documentation not appropriate and adequate.
 

Risk Rating:

Critical
Risk Assessment
  • File documentation pertaining to claims is reasonably complete
  • File documentation pertaining to formal monitoring activities is absent from many files reviewed
  • The use of forms (e.g. claims verification checklist and risk assessment form) vary significantly between regions from very limited use to possibly excessive use
  • Files are structured differently between regions
Controls Ref. Description
7. M&P Manager, or another manager with appropriate authority, reviews all claims and applies Financial Administration Act Section 34 authorization indicating payment has been reviewed in accordance with the FAA. This review typically includes a review of supporting documentation on file
Deficiencies Description Rec.
There is no central coordination of the M&P function H
M&P follows different procedures and policies (e.g. audit policy, thresholds for supporting claims documentation) between regions
 E
M&P lacks any current and specific policies and procedures to govern its activities and processes in accordance with regulations and WD interpretations
 E
There is no training of M&P staff on regulations and WD interpretations, including on-going training on changes
 G
There is no quality assurance review process or internal audit of the M&P function
 J

 

Risk

  1. Services delivered through third-parties not managed or monitored effectively.
 

Risk Rating:

Critical
Risk Assessment
  • WD delivers very few programs through third-parties. An example of a program that will be delivered on WD's behalf is the Forestry Adjustment program in BC that will be delivered through the Community Futures Development Association of B.C. Therefore, this risk was interpreted to refer to the management of Service Delivery Network Partners, Community Futures Development Corporations and Women's Enterprise Centres (collectively referred to as "service delivery partners")
  • WD is primarily concerned about tracking the performance of its service delivery partners
Controls Ref. Description
11. WD requires audited financial statements and a statement of compliance from external auditors
25. Service delivery partners must submit annual operating plans and quarterly performance reports against operating objectives
Deficiencies Description Rec.
WD may be relying inappropriately on external auditors' compliance reports for Service delivery partners
 K
Performance reports from service delivery partners are not always useful and WD does not verify their accuracy
 K

 

Risk

  1. Lack of coordination, clear accountability and risk assessment between contributing partners (e.g. provincial government).
 

Risk Rating:

Mod.
Risk Assessment
  • This risk pertains to WEPA's that can be structured such that: the province takes the lead in monitoring the project; WD takes the lead in monitoring the project; or each party contracts separately with the client
  • Interviewees did not see this risk being significant as WD has the ability to negotiate appropriate controls (including control over administering the contribution agreement) with the province
Controls Ref. Description
13. The respective accountabilities and responsibilities of the province and WD are outlined in the WEPA agreements for each individual project
Deficiencies Description Rec.
Insufficient number of WEPA's reviewed to identify control deficiencies  

 

<< previous | next >>