Home : Reports and Publications : Audit & Evaluation : Contribution Audit Policies and Processes for WDP and ICIP – March 2004
Recommended Audit Policies and Processes
This section outlines recommended policies and specific processes, where necessary, to achieve the objectives for contribution audits. Brief rationale for recommended policies is provided following each group of policies.
The policies are intended to be mandatory. In some cases, specific processes must be followed in order to comply with the policy. Any such mandatory processes follow the policies they support.
Policies in this section relate to:
- Audit selection;
- Auditor selection;
- Coordination;
- Scope of Work;
- Approach;
- Deliverables; and
- Results tracking.
1. Audit Selection
1.1. Process for Selecting the Audit Sample
Recommended Policies
1.1.1 The audit sample will be selected from WDP and ICIP projects that are active during the fiscal year, and that received at least one contribution payment. The sample will include projects from both programs.
1.1.2 WD must receive audit opinions for a sufficient number of projects each year to estimate the overall number of projects that are in compliance with their terms and conditions with a confidence level of 90% and a margin of error of plus or minus 5%.
1.1.3 The audit sample should be comprised of two strata:
- Stratum 1: High-risk projects. All projects with a high risk rating (100% sample); and
- Stratum 2: Randomly-selected, projects not already included in stratum 1.
1.1.4 The sample size of stratum 2 must allow reporting of separate results for the stratum to a confidence level of 90% and a margin of error of +/- 5%.
Rationale for Policies
Polices related to the sample of projects to audit are intended to:
- Demonstrate WD's stewardship over public funds in an unbiased and defensible manner. WD will be able to report the estimated percentage of projects in receipt of contribution payments in the current year that were in material compliance with their agreements;
- Obtain meaningful (i.e. statistically valid) data as input as to the effectiveness of its project assessment, monitoring and claims verification activities; and
- Result in a probability of being audited that is sufficiently large that contribution audits may become a deterrent against inappropriate claims.
The terms and conditions for WD and ICIP, which form the basis for contribution agreements, are similar. Additionally, WD uses substantially the same monitoring and verification activities for WD and ICIP projects. Therefore, in the interest of efficiency, a single audit sample can incorporate projects from both programs.
WD must demonstrate extra vigilance around high-risk projects. Also, assuming an accurate assessment of risk, these projects should have an inherently higher probability of not complying with their contribution agreements than the population of projects as a whole. Therefore, both to demonstrate diligence and to improve the effectiveness of performing contribution audits, it is important to stratify the sample based on project risk.
While the sample size of the random sample will be sufficient to provide significant results for WD as a whole it would be too costly to achieve statistical significance in each region or in each program. The sample criteria would give projects in each region and program an equal probability of being selected.
The recommended policies will result in a sample size of at least 35 audits each fiscal year. In addition to the projects included in the sample, WD may elect to audit projects that meet special criteria for the purpose of addressing high-risk or problem areas (e.g. specific regions or programs), or create a further deterrence (e.g. type of recipient).
The sample is based on projects, not claims. This allows WD the flexibility to audit groups of claims for greater efficiency.
Required Processes
1.1.a Central Coordination
As the sample will be based on all eligible projects, regardless of region, it must be generated centrally. WD must develop specific, mandatory processes and responsibilities for:
- Maintaining a registry of projects and necessary information to generate the sample;
- Generating samples for each stratum; and
- Allocating funds to regions for contracting auditors based on the sample selected.
1.1.b Sample Size Determination
Stratum 1 is comprised of a 100% sample of high-risk projects. The sample size for stratum 2 must be based on a simple random sampling method and must be calculated based on a confidence level of 90% and a margin of error of +/- 5%. To minimize the number of audits performed, WD should consider the following:
- Assume a 100% compliance rate (percentage of unqualified audits) and draw additional samples as required should the compliance rate be lower. This assumption is reasonable given that all WD reviews all project claims. Any compliance issues should be detected and rectified through this process; and
- Determine if the selected projects have already been audited by another agency in a manner that meets WD requirements (as outlined in recommended policies 3.1.1 and 3.1.2).
Subsequent years' sample sizes should be based on a compliance rate equal to or less than the prior year's rate.
1.2 Selection Criteria
Recommended Policy
Selection criteria for projects to audit are outlined by stratum:
Selection Criteria |
|
|---|---|
Stratum 1: High-Risk |
All active projects with contribution payments made during the current fiscal year that have a risk rating of "high" as calculated using the WDP and ICIP Risk Assessment Tool. |
Stratum 2: Random |
Simple random sampling of all projects with contribution payments made during the current fiscal year and not eligible for inclusion in stratum 1. WD could consider an alternative, probabilistic sampling method such as systematic sampling based on a calculated dollar interval. This would bias the sample significantly towards large dollar projects and would need to be disclosed when reporting estimates of WD's overall compliance rate. |
Rationale for Policy
The selection criteria directly support the sample requirements for each stratum, and eliminate ambiguity and judgement in how projects should be selected for auditing.
1.3 Risk Assessment
Recommended Policies
1.3.1 High-risk projects must be identified using a risk assessment tool designed for WDP and ICIP projects.
1.3.2The risk assessment for each project must be reviewed annually and as new information becomes available (e.g. through regular monitoring activities).
1.3.3The risk rating "high" must denote projects that have a significant risk of breaching financial or non-financial terms of their contribution agreement.
Rationale for Policies
The risk assessment tool formalizes officers' perceptions of project risk, and supports a uniform approach to assessing risk.
Required Processes
1.3.a Revising and Maintaining the Risk Assessment Tool
The current risk assessment tools for WDP and ICIP need to be standardized and revised to more accurately reflect project risk. Appendix C provides suggested risk assessment criteria grouped by three types of risk elements:
- Recipient Risk;
- Project Risk; and
- Compliance Risk.
In addition, risk elements need to be weighted to ensure their impact on the total risk score reflects their impact on the risk of a project not complying with its terms and conditions. WD must track incidents of temporary or complete failures to comply and correlate them to the risk elements and scores. It must adjust the risk elements and weightings as necessary to improve the risk assessment's predictive ability over time.
1.3.b Tracking Risk Assessments
In order to identify high-risk projects requiring an audit, and to correlate failure incidents to risk elements and scores, WD must maintain a central registry of current risk ratings for each project.
1.4 Frequency and Timing
Recommended Policies
1.4.1 Contribution audits should occur after WD has completed its internal review of claims submitted to date on the project and within 90 days of the fiscal year end in which the claims were processed.
1.4.2On a quarterly basis, WD should identify projects for audit while ensuring that all projects that comprise the population for stratum 2 have an equal probability of being selected over the course of the year.
Rationale for Policies
Conducting audits after a desk review will:
- Increase the effectiveness of the audits at helping to identify potential weaknesses in the claim verification process;
- Allow WD project monitoring officers to provide input into the focus of the audits, particularly around areas of concern and compliance with key terms and conditions; and
- Allow WD to assume an initial compliance rate of 100% for the purpose of determining its sample size.
The recommended timing of audits is intended to distribute the audits throughout the year. However, WD may determine that it is simpler to select and audit all projects following the end of the fiscal year. This may strain CAC audit resources.
Required Processes
WD should select projects to audit following the timing and frequency in the table following:
Selection Criteria |
|
|---|---|
Stratum 1: High-Risk |
Projects to be audited at least once annually:
|
Stratum 2: Random |
Projects to be audited once annually, selected quarterly based on one quarter of the estimated total annual number of qualifying projects excluding projects considered in previous quarters of the current fiscal |
2. Auditor Selection
Recommended Policy
2.1.1 Contribution audits must be performed by an auditor that is qualified and permitted to issue a financial audit opinion in the province of the recipient, and that is independent of the recipient and all of its partners, including WD and other funders.
2.1.2 WD must contract external auditors to follow specified terms and conditions for conducting its contribution audits
Rationale for Policy
In addition to complying with central agency requirements, this policy follows best practice to ensure objective and professional audits.
3. Coordination
Recommended Policy
3.1.1 WD should coordinate its contribution audits of projects included in its sample with other funding agencies when feasible and appropriate.
3.1.2 WD may accept the results of contribution audits performed by another funding agency on a selected project if the audit and associated work meets WD's audit objectives and all elements of the required scope of work (see 4. Scope of Work).
Rationale for Policy
This policy helps to reduce the number of redundant audits performed on a single project within a given year by multiple funding partners. Coordinating audits, particularly between federal government departments is recommended in central agency guidelines.
4. Scope of Work
Recommended Policies
4.1.1 The scope of the work to be completed must be defined such that it will meet the objectives of the contribution audits.
4.1.2 The scope of the audit and the extent of responsibility assumed by the auditor must be clearly defined in writing for each audit. The nature of the claims and the specific terms and conditions of the Agreement subject to audit could vary considerably and accordingly, it is important that there be a clear understanding and agreement as to the scope of the audit.
4.1.3 The scope of the audit and the matters to be reported on must be within the auditor's professional competence.
4.1.4 The audit engagement should clearly state the criteria, which the auditor will use to evaluate compliance. Criteria are benchmarks against which the subject matter of the engagement can be evaluated. Without suitable criteria, inappropriate conclusions may be drawn. Characteristics of suitable criteria are relevance, reliability, neutrality, understandability, and completeness.
4.1.5 WD must clearly define any additional areas that will be subject to review, in particular the key terms and conditions, which are to be reviewed in the scope of the work being completed.
Rationale for Policies
The recommended policies are consistent with best practices followed by the auditing profession.
Required Processes
Following the selection of the audits, WD must clearly define the scope of the work required to the auditor. This needs to be done in context of the specific objectives of the WD contribution audits.
For each project or claim selected for audit, WD needs to define the specific scope of the audit to be completed with respect to the specific claim(s) to be audited and an accounting period for auditing contributions received.
All audits at a minimum should consider the following:
- Project funding received from all sources has been completely and accurately reported;
- Claimed project costs are accurate and valid costs; and
- Claimed costs are eligible and in accordance with the contribution agreement, and departmental and central agency guidelines.
For each project or claim selected, WD needs to clearly define additional areas that are to be reviewed.
Responsibility for reporting on compliance with the terms and conditions of the funding agreement rests with the funding recipient. The responsibility of the auditor is to report on whether the recipient has met the terms of conditions of the funding agreement. The recipient must prepare and provide sufficient reporting and evidence that they have met the terms and conditions of the agreement. Insufficient reporting and/or evidence to support that compliance has been met should be reported.
5. Approach
Recommended Policies
5.1.1 All audits are to conducted in accordance with generally accepted auditing standards established by Section 5815 of the Canadian Institute of Chartered Accountants Handbook ("CICA HB") Audit reports on Compliance with Agreements, Statutes and Regulations ("Section 5815"). Section 5815 provides guidance to an auditor engaged to express an audit opinion as to a client's compliance with criteria established by provisions of agreements, statutes or regulations.
5.1.2 The auditor should comply with the general and examination standards of Section 5100 CICA HB as follows:
General standard
The examination should be performed and the report prepared by a person or persons having adequate technical training and proficiency in auditing, with due care and with an objective state of mind.
Examination standards
- The work should be adequately planned and properly executed using sufficient knowledge of the entity's business as a basis. If assistants are employed they should be properly supervised.
- A sufficient understanding of internal control should be obtained to plan the audit. When control risk is assessed below maximum, sufficient appropriate audit evidence should be obtained through tests of controls to support the assessment.
- Sufficient appropriate audit evidence should be obtained, by such means as inspection, observation, enquiry, confirmation, computation and analysis, to afford a reasonable basis to support the content of the report.
Rationale for Policies
The recommended policies are consistent with best practices followed by the auditing profession and are intended to ensure audits are performed consistent with Canadian generally accepted auditing standards.
Required Processes
Audit
The audits should be supported by an adequate plan. As part of the planning phase the auditor should confirm the scope of the audit in accordance with the direction provided by WD.
The auditor should plan and perform the audit to obtain reasonable, but not absolute, assurance that the client's claim is in compliance with the criteria established by provisions of the funding agreement taken in all material respects. Absolute assurance in auditing is not attainable because of such factors as: the nature of audit evidence which is based on the use of testing and where much of the evidence available to the auditor is persuasive, rather than conclusive; the inherent limitations of internal control; and the characteristics of fraud.
Because of the nature of fraud, including attempts at concealment through collusion and forgery, an audit planned and performed in accordance with Canadian generally accepted auditing standards may not detect fraud. Further, while effective internal control reduces the likelihood that errors, fraud, or illegal acts will occur and remain undetected, it does not eliminate that possibility. Accordingly, there is a risk that material errors, fraud, and other illegal acts may exist and not be detected by an audit performed in accordance with Canadian generally accepted auditing standards. Also, the audit will not be designed to detect matters that are immaterial to the claim.
Additional Areas
The auditor should consider the control environment and whether appropriate reliance can be placed on internal controls in planning the audit.
While not being engaged to report on the recipients internal control the auditor should communicate to WD, any significant weaknesses in the recipient's internal control structure that come to the auditors attention during the audit.
The auditor should confirm that any additional work to be completed at the direction of WD on key terms and conditions is within the auditor's professional competence.
The auditor and WD should agree on the form of the reporting to be provided on matters relating to compliance with key terms and conditions. The auditor should plan the work accordingly such that it will support the required reporting.
6. Deliverables
6.1 reports
Recommended Policy
6.1.1 The auditor must provide a written opinion that the claim or groups of claims are in compliance, in all material respects, with the criteria as described in the contribution agreement. The opinion should be attached to the statement of claims and contributions audited.
6.1.2 The auditor should provide support for recommended claim adjustments and conclusions.
6.1.3 The auditor should provide specific compliance items reviewed, a schedule of evidence considered and related findings.
6.1.4 The auditor should provide an overview of the performance tracking process and findings related to the reasonableness of the process and results reported.
6.1.5 The auditor should provide general observations and findings related to matters not specifically within the scope of the audit, but which may be of interest to WD in its assessment or management of the project.
Rationale for Policy
The auditor's report must satisfy the specific contribution audit objectives. The auditor should not make recommendations in its report, but rather leave the interpretation of the audit results and the decision to take any necessary, remedial action to WD. This affords WD the latitude to respond to the audit results in a manner that is most appropriate given its desire to see each project succeed, and to maintain relations with other funders and the recipient.
It is not generally accepted practice to request access to or copies of the working papers of auditors. There is no compelling reason for WD to review auditors' working papers.
6.2 Audit Opinion
Recommended Policies
6.2.1 The auditor's report should cover the audit work relating to the following objectives:
- Project funding received from all sources has been completely and accurately reported;
- Claimed project costs are accurate and valid costs; and
- Claimed costs are eligible and in accordance with the contribution agreement, and departmental and central agency guidelines.
6.2.2 The auditor's report should follow the standards established by CICA HB Section 5815.
6.2.3 The auditor should clearly state the specific Sections of the Agreement, which have been considered by the audit. The auditor should also provide any interpretation of the provisions of the Agreement that the auditor has made in order to conclude on the report.
Required Processes
A suggested auditor report consistent with the recommended policies is provided on the next page. WD should request auditors use such a standard report for all contribution audits, subject to modifications required as a result of specific circumstances.
AUDITOR'S REPORT to WD I have audited the eligibility of Claim number for XYZ, for compliance with the criteria established by Sections of the WD Agreement ("the Agreement") dated xx . Compliance with the criteria established by the provisions of the Agreement is the responsibility of the management of XYZ and the interpretation of the provisions as further described in Note 1 attached. My responsibility is to express an opinion on this compliance based on my audit. I conducted my audit in accordance with Canadian generally accepted auditing standards. Those standards require that I plan and perform an audit to obtain reasonable assurance whether XYZ complied with the criteria established by the provisions of the agreement referred to above. Such an audit includes examining, on a test basis, evidence supporting compliance, evaluating the overall compliance with the Agreement, and where applicable, assessing the accounting principles used and significant estimates made by management. In my opinion, as at December 31, 20XX, XYZ is in compliance, in all material respects, with the criteria as described in Sections ...... to ...... of the Agreement. (signed)............................................. City Date |
6.3 Reporting Process
Recommended Policies
6.3.1 The auditor must submit the results of a contribution audit to WD within 15 days of completing the audit.
6.3.2 The auditor must provide WD with an opportunity to discuss the report within 10 days of receiving it.
6.3.3 Audit reports must be submitted to the WD officer responsible for the projects being audited and to WD headquarters.
Rationale for Policies
It is important to establish a turnaround time for contribution audits to ensure that WD receives results in time to take any necessary, remedial action, and to aggregate and report results. Many interviewees expressed a desire to have a debriefing with the auditors to clarify any findings in the audit report.
It is important that WD officers responsible for monitoring projects and an audit coordinator in headquarters receive the audit reports. The officers are the most knowledgeable people to interpret the results and take any necessary action. However, for the purpose of tracking audit results and to avoid issues of officer bias, it is equally important for headquarters to receive a copy of each audit.
7. Results Tracking
Recommended Policy
7.1.1 Contribution audit results must be tracked and aggregated organization-wide such that WD can:
- Identify trends or recurring problems;
- Adjust selection and risk assessment criteria;
- Identify any opportunities to improve monitoring and claim verification procedures;
- Identify the need to further stratify the audit sample; and
- Estimate and report on project compliance overall.
Rationale for Policy
Audit results must be aggregated and used in order for WD to obtain benefits from contribution audits beyond those associated with auditing individual projects.