Conclusion & Recommendations: Monitoring System

Observations

Because of the regional differences in the control systems, it is difficult to give conclusions as if there was one common system. However, it is possible to make observations of practices that are specific to certain regions and common across regions as well.

Best Practices:

The differing approach taken in each of the regions has allowed WD Staff to innovate their own practices. Certain of these are highlighted below as best practices:

Financial Statement Review and Reconciliation Program: employed in British Columbia and Alberta, which is done according to a detailed program and thoroughly documented in the files. (However, it should be noted that Saskatchewan and Manitoba also perform financial statement reviews for many of the same items).
CFDC Operational Review: done only in Saskatchewan for each CFDC approximately every three years, the three day on-site review of the operations, loan files, policies and financial reporting gives a great deal of information about each CFDC and assurance to WD that the CFDC is in compliance with provisions of the CF Contract and properly furthering the objectives of the CF Program.
Risk Assessments and Site Visits: While all regions appear are in the process of implementing these processes, Manitoba has implemented the systematic assessment of risk assessment and site visits to a greater extent than the other jurisdictions. The risk assessment program allows attention to be focused on CFDC’s at greater risk (and therefore in need of greater monitoring); the routine and systematic site visits allow timely information regarding the operations of each CFDC.

Opportunities for Improvement:

Although the regions have developed their own approaches to address their control and monitoring responsibilities, across the regions there are several areas where improvement is possible:

Informal and Undocumented Monitoring: while the use of informal means of monitoring CFDC activities (i.e. phone calls, casual visits) is of benefit, there is a lack of documentation of these activities. More importantly, the review of annual operating plans, which is a key factor in monitoring CFDC activities, is largely undocumented (or inconsistently documented). The 2001 Report of the Auditor General also raised concerns in relation to the level of file documentation.
Regional Inconsistency: recognizing that each region will have its own areas of focus and priorities based on differences in the economic environments in each, there is a difference in the approach taken in each region. Given that the program objectives and structure is identical in each region, one would expect that the monitoring systems would be virtually identical as well. Differences in the jurisdictions may lead to weaknesses in certain areas in each region, inefficiencies in implementing new processes which may already be employed in other regions, and the perception of unequal treatments of CFDC’s from different regions.

Conclusions

It is important to note that a long-term relationship exists between each of the CFDCs and WD. The CFDCs are funded on five-year contracts for their base funding on a continuing basis. In most cases, the CFDCs have been in existence for approximately 10 years, with the range of ages being between approximately 5 and 15 or more years (when the CF Program was administered by Human Resources and Development Canada).

The ongoing nature of the relationship is a double-edged sword. On one hand, this familiarity with CFDCs and their management gives WD Staff a broader base of knowledge regarding each CDFC, and reduces the risk of projects failing or funds being misappropriated. However, this same familiarity could lead to a degree of complacency in the monitoring of CFDC activities, and lack of diligence.

Similarly, as WD Staff work with CFDC management and boards over longer time frames, they are able to identify problem areas on a timely basis. While a very rigorous system of controls would be necessary if one-time projects were being funded, the nature of an ongoing multi-year program with strong relationships would lend itself to a less rigorous control structure. Therefore, although the 2001 Report of the Auditor General indicated that reliance on informal ways of checking on CFDCs was inadequate, these informal means nevertheless form a valuable part of the control structure, in concert with more formal monitoring.

It is therefore important that, while the program officers who directly monitor the CFDCs may utilize informal means and their past experience as tools, proper diligence and discipline to this process should be imposed within the control system. It is also the responsibility of senior WD Staff to ensure that this is the case.

The practical aspects of a control system must also be considered. An extremely thorough process could be envisioned which incorporates the best practices of each jurisdiction plus stringent controls and penalties in relation to filings. However, such a process would be extremely onerous not only on the CFDCs, but on WD Staff as well.

In summary, the monitoring procedures set out in the “CFDC, WEI and FEDO Contributions – WD Policy” provide an adequate system of monitoring the CFDCs.

The actual performance of WD in relation to the key aspects of this policy is summarized as follows:

Annual Project Risk Assessment Form completed
- Degree of implementation varies in each jurisdiction, but all jurisdictions are assessing risk as part of the monitoring process.
Update and review of CFDC policies (Investment Fund Policy, Confidentiality, Human Resource Management, Conflict of Interest) and retain copies on WD files
- None of the 20 CFDC files reviewed contained copies of the CFDC policies in these four areas. Staff indicated that the policies would have been reviewed during the initial application.
Review of quarterly performance results and variance to forecast
- All quarterly reports are reviewed. However the extent and focus of the review is for the most part at the discretion of the Client Services Officer. These reports are discussed in greater detail in the next section of the report.
Review for compliance to contract terms and conditions
- As part of the annual financial statement review, all jurisdictions were performing some review to determine compliance. Also, compliance certificates provided by each CFDC’s external auditor were relied upon as evidence of compliance as well, and were received with only on exception for the CFDC files reviewed. These compliance certificates are discussed in greater detail as well in a later section of the report.
Review of interim financial statements and annual audited financial statements, including a review of auditor’s report, ineligible expenses, contribution payments in excess of eligible expenditures, other government funding, and interfund transfers
- All jurisdictions are performing a review of financial statements, with varying degrees of formality. The key areas indicated above are the focus of WD Staff’s review.
Additional monitoring procedures should be undertaken based on an annual risk assessment, including a minimum of one site visit, with additional procedures including additional site visits, review of accounts, review of board minutes, compliance audits / program evaluations, review of other information and meeting with CFDCs auditors
- Implementation of site visits based on risk assessment varies between jurisdictions. The availability of resources needed to complete these visits has been raised as a concern.

Overall, the two significant areas of concern with the administration of the program are the inconsistency in the systems used in each jurisdiction and the lack of formal documentation in relations to certain key processes and informal monitoring practices.

Recommendations

A. Consistency Across Jurisdictions

Although regional differences exist, the basic framework for monitoring the CFDCs is the same, and the approach to monitoring the CFDCs should also be consistent. A common approach should be implemented in respect to key control items, which would follow the monitoring procedures already outlined in WD policy and address the concern of file documentation as well. These key control items are:

Formal and documented financial statement reviews should be implemented across all jurisdictions: This would be based on a standardized program with any issues raised and method of resolution recorded as part of the program.
Formal and documented operating plan review should be implemented across all jurisdictions: A standard program may not be appropriate for each of 90 CFDCs, so the review process would likely still depend on the experience and judgement of the program officers. However, a checklist of key items as well as documentation of queries and responses (whether in the form of letters, emails, faxes) would provide support that the operating plan has been reviewed and that further funding is appropriate.
Site visits to each CFDC for the control and monitoring purposes should be completed (and based on completed CFDC risk assessments): The current process ties site visits to the risk assessment process. However, the practice across the regions regarding carrying out these visits varies widely, presumably because of lack of resources to complete these visits. Also, many of these visits are for purposes other than a monitoring function. However, the CFDC management of their loan portfolios is a risk area for the program, and the assessment of individual CFDC’s lending practices should be undertaken on a routine and systematic basis. Since WD resources are limited, one alternative would be to perform a detailed review of CFDC operations on a rotating basis (similar to the approach taken in Saskatchewan).

For the most part, the current practices of the regional offices cover all the areas noted above. Certain jurisdictions have developed their own practices covering the above points, and a coordinated means of sharing practices between the regions would reduce any implementation inefficiencies.

B. Auditor’s Compliance Reports

A financial statement audit requires the auditors to satisfy themselves that the enterprise being audited is in compliance with contracts material to the financial statements. In the case of the CFDCs, this would clearly apply to the CF Contract. Therefore, it would seem that the additional step of having an external auditor provide WD with a certificate of compliance would be an efficient way for WD to gain assurance regarding CFDC compliance. However, there are problems with this approach:

Materiality: Financial statements are completed with the concept of materiality. The level of assurance required by an auditor to assess compliance in the context of a financial statement audit may be significantly less than the level of assurance required to provide the compliance certificate. Therefore, the CFDC may incur additional costs that take away from the funds available to complete their mandate.
Expertise: The CF Contract contains many aspects that may be outside the expertise of a financial auditor. An auditor may only be qualified to give an opinion on the financial aspects of the contract.
Substitute Judgement: While the auditor may give an opinion that a CFDC is in compliance, it is ultimately the responsibility of WD to make this assessment. WD may be relying on an assessment that would differ from their own.

We recommend that the utilization of compliance statements should be revisited in order to address the above issues. One alternative would be to limit the scope of the auditor’s compliance report to cover only the financial aspects of the CF Contract. Another alternative would be to determine specific procedures for the auditor to perform, and report the results. WD would then be in the position to make the determination regarding compliance themselves. The latter alternative would reduce costs to the CFDC since the procedures would be predefined for the auditor, and would eliminate the issues of “Expertise” and “Substitute Judgement” noted above.

C. Quarterly Performance Reports

Several comments arose in relation to the quarterly reports filed by the CFDCs from both WD Staff and the CFDCs themselves (certain comments arising as part of the Evaluation of the CF Program). The following summarizes these, some of which were noted earlier:

Preparation of the reports is onerous on the CFDCs, who receive little feedback on the reports themselves.
Some relevant information may not be captured.
Some irrelevant information may be captured (i.e. there are too many variables being reported).
Reporting may be inconsistent between the different CFDCs because of differing interpretations of what is required.
Reporting may be inconsistent over time within individual CFDCs.

Also, based on the findings of this audit, it appears that the reports are used largely as a tool for reporting on the results of the program itself, and utilization of the reports as a tool for monitoring the CFDC activities is minimal.

Therefore, the purpose and content of these reports should be revisited. For example, as a tool for monitoring the CFDCs, quarterly reporting may be necessary. However, all of the variables being reported on may not be needed. Similarly, the data necessary to report on the performance and results of the CF Program on a quarterly basis may not require as great of detail as annual reporting, such that the quarterly burden on CFDCs could be reduced.

If the reports are still to have the dual purposes of being tools for monitoring individual CFDCs as well as tools for reporting on program results, the fields being reported on should be revisited and the reports streamlined to the greatest extent possible.

British Columbia has developed an automated tool for performing a variance analysis using the quarterly reports, which best utilizes WD Staff’s time in reviewing the variances and using them as discussion points with the CFDCs rather than spending time manually doing calculations. Also, Manitoba has developed a tool for the CFDCs to report goals, activities and results on an annual basis in a concise format. In keeping with the earlier recommendation for the regions to achieve greater harmonization of practices, these tools may be utilized in revisiting the quarterly reporting process.

D. CFDC Policies

The terms of the CF Contract require each CFDC to have corporate policies in four key areas:

Investment Fund Management
Confidentiality Assurance
Human Resource Management
Conflict of Interest Guidelines

However, none of the 20 CFDC files reviewed contained copies of these policies. Staff indicated that the policies would have been reviewed as part of the initial approval process for the CFDC. However, there was no indication that these policies would have been reviewed in the subsequent period, or if there had been changes to them.

WD has been working with the CF Association to develop standard policies in these areas. Given that all of the CF Contracts end as of March 31, 2004, the CFDC policies should be revisited for each CFDC as part of the renewal process. The standard policies could be used as either a basis for comparison to existing policies, or as templates by which each CFDC could draft new polices.

E. Improved File Documentation

As noted earlier, a number of the monitoring procedures are informal. These informal practices may include:

telephone conversations with CFDC directors or management,
casual visits to CFDCs,
attendance at CFDC events, and/or
discussions with CFDC staff regarding possible initiatives.

WD Staff indicated that they were in frequent contact with CFDCs as a matter of routine. However, this contact is not well documented. While it would not be practical to document every telephone call or conversation, this is essential if important issues are discussed, such as CFDC initiatives, performance or loans. Improved file documentation would allow those with oversight responsibility to monitor work of subordinates and to facilitate transition if new project officers are assigned to monitor the CFDCs.

<< previous