About the CFIA > Reporting to Parliament > Annual Report > Annual Report 2002 - 2003  

5.0 AUDITOR GENERAL'S ASSESSMENT OF PERFORMANCE INFORMATION

AUDITOR GENERAL'S ASSESSMENT
of Performance Information in the 2002-2003 Annual Report

To the President of the Canadian Food Inspection Agency
and the Minister of Agriculture and Agri-Food

PURPOSE AND SCOPE

The Canadian Food Inspection Agency Act requires the Auditor General to assess the fairness and reliability of the performance information in the Agency’s annual report with respect to the annual and overall objectives established in its corporate business plan.

The corporate business plan and the performance information in the annual report are the responsibility of the Agency’s management. My responsibility is to provide an assessment of the fairness and reliability of the performance information in the Agency’s 2002-03 Annual Report. To do so, I assessed the information against the criteria described in Annex 1 for fairness and reliability, which were discussed with the Agency. I conducted my assessment in accordance with the standards for assurance engagements established by the Canadian Institute of Chartered Accountants. The assessment included an analysis of the information and an examination on a test basis of the evidence supporting that information. I conducted my assessment to obtain an audit level of assurance for the relevant, meaningful, attributable, and balanced criteria. However, I conducted my assessment work to support a review level of assurance for the accuracy criterion. I did not assess or comment on the Agency’s actual performance.

My assessment relates to the performance information contained in the report and not to information referenced by Web site links.

CONCLUSION

In my opinion, while several good improvements have been made this year, overall the information on the performance of the Agency does not yet adequately meet my expectations for fair and reliable reporting.

SUMMARY ASSESSMENT

The following are some of the key observations against the criteria for fair and reliable reporting, excluding the human resources performance information, which is discussed separately.

Relevant

The role of the Agency is clearly outlined. I am pleased that the performance information generally covers all aspects of the context within which the Agency operates. It also includes information on some of the external factors that the Agency must manage in delivering its services, including such matters as increased globalization and demand for services. In addition, the Agency has spent considerable effort to develop logic models to provide structure in telling its performance story in relation to strategic outcomes and key results.

More outcome information is needed and better explanations are required. For a number of key results, there is a focus on activity information with a lack of outcome measures. An example of this is the Agency’s description of its education and awareness activities. In addition, in many cases where outcomes are reported, the Agency does not clearly explain the importance of that information. This weakness limits the relevance of the performance information in assessing the Agency’s performance.

More cost information is required. This year, the Agency reports the current year’s cost of each of the three business lines and major programs. However, there is no information or analysis related to the total cost of operations over time. For example, the net cost of the Agency’s annual operations has increased fifty-six percent from $324 million in 1998-99 to $506 million in 2002-03. It is difficult to determine how this additional $182 million in net costs relates to the business lines and programs. In addition, there is no reporting or analysis of actual costs against planned costs to deliver the Agency’s services either in terms of total costs or by individual business line or program.

Meaningful

The compliance-based approach is explained. This year, the Agency has clearly reported that it strives to promote 100 percent compliance with regulatory requirements and strives for year over year improvements in cases where it is less than 100 percent. In most cases, actual compliance rates are less than 100 percent.

More explanations are required on performance trends and gaps. The Agency does not consistently compare current and previous years’ compliance rates. Nor does the Agency generally provide a meaningful interpretation of the gap between actual performance and expected performance, and its strategy for year over year improvements.

Performance is not compared against expectations in all cases. The performance information lacks clear and concrete expectations in several key result areas. An example of this is the meat industry’s adoption in federally registered establishments of hazard analysis critical control point (HACCP) systems. In 2002-03, the Agency reports that 58 percent of federally registered meat facilities now have this recognition. This is an increase of thirty facilities or 10 percent in 2002-03 from the year before. Without targets for this indicator and no discussion of actual results against planned results it is difficult for Canadians to hold the Agency accountable for its performance in promoting the implementation of HACCP.

Attributable

It is difficult to determine the Agency’s contribution to reported results. The Agency has included general information on various partners that work with it to achieve its mandate. However, overall it is difficult to determine the Agency’s contribution to most of the reported results. The influence of industry, other government organizations, consumer and interest groups, and the Agency in achieving key results (for example, industry compliance rates) is not consistently addressed. I encourage the Agency to continue to work on clarifying its contribution and that of its partners with respect to the reported results.

Accurate

Limitations of the reported information are outlined in some cases. In this year’s food safety business line, the Agency improved its disclosure by describing the limitations of the data including its reliability and availability. For example, the report notes that in the dairy program, national compliance rates for the Agency’s microbiological sampling are not available. In addition, the performance information indicates that available establishment compliance rates are not fully validated including reported compliance rates for: meat slaughter and processing; fish and seafood processing; and fruit and vegetable processing. The Agency has committed to the development of improved strategies for data collection, management and reporting in 2003-04.

More information is required on the validity of reported information. The performance information generally does not discuss the statistical validity and variability of the information. For example, the Agency has reported as a new indicator of its performance, the number of inspections carried out versus the number planned (verification delivery rate). In the meat hygiene program, the Agency carried out 35 percent of its planned partial audits of facilities that are recognized as having a food-safety enhancement program in place. There is insufficient discussion of any impact of the actual number of audits, which are significantly less than planned levels, upon the validity of reported outcomes (e.g. compliance rates), and the strategies to address this challenge.

Additional work remains to implement quality assurance. This year we noted that the Agency has made improvements in its quality assurance function in verifying the accuracy of the information that is reported. However, the Agency has more work to do to develop and implement a fully satisfactory quality assurance function.

Balanced

There is improved balance in reporting upon performance. I am pleased that this year the performance information has improved balance in reporting results (e.g. compliance rates, verification delivery rates) that the Agency identifies as either satisfactory or unsatisfactory and, to some extent, indicates areas where greater effort is required.

More information on risks and challenges is needed. In some cases, the level of explanation surrounding significant risks and challenges is incomplete including, for example: emergency funding issues; resource challenges; the mandate of Health Canada in evaluating the Agency’s food safety activities; and more specific information on challenges related to the Agency’s information systems.

PERFORMANCE INFORMATION ON THE AGENCY’S HUMAN RESOURCE MANAGEMENT

The human resources performance information is fair and reliable. Based upon my assessment, I believe that the human resources performance information reasonably meets my expectations for fair and reliable reporting. The information in this year’s report related to human resource management has improved over that of previous years. The Agency continues to produce corporate reports on important human resource trends in order to monitor performance and to identify key risk areas. The methods the Agency is implementing to collect and report training and employee satisfaction information should improve its capacity to report against human resource objectives and to identify areas of risk for the Agency.

However, further improvements in the reporting of this information are desirable. For example, greater consistency in the reporting of precedent, actual, and expected results would provide a more meaningful assessment of human resource-management performance.

IMPROVING THE AGENCY’S PERFORMANCE REPORTING

In this - the Agency’s sixth year of reporting on its performance - I noted improvements in various aspects of reporting, including: development of the performance management framework; improved balance in disclosure of performance; transparency related to the quality of data; enhanced description of the Agency’s role in the various sectors; and a better description of the compliance-based approach.

I was particularly pleased to note that this year the Agency reported the results related to a key-pilot initiative to improve its performance reporting in the meat hygiene and fish programs. The Agency has reported the results of these programs including information that more closely captures the spirit of fair and reliable reporting. There is much work left to be done in completing the pilots, including refining the indicators and targets, and expanding data collection and analysis. This is an indication that the Agency is on a path to improved performance reporting in future years. I encourage the Agency to continue to implement its performance framework building on the results of its pilots.

The Agency needs to improve the usefulness of its performance information to better serve as an accountability document. This could be accomplished by focusing on: reporting on outcomes as opposed to activities; providing clear and concrete performance expectations; discussing gaps between actual and expected performance, including planned action to improve performance; enhancing description of the significance of the performance information; improving financial information; and reporting more comprehensive descriptions of risks and challenges.

In 2003-04, I would like to see the Agency’s efforts result in performance information that comes closer to reasonably meeting my expectations for fair and reliable reporting.

Sheila Fraser, FCA
Auditor General of Canada

Ottawa, Canada
November 7, 2003

ANNEX 1

Criteria for the Assessment of Fairness and Reliability Office of the Auditor General

The following criteria were developed to assess the fairness and reliability of the information about the Agency’s performance with respect to the objectives in its corporate business plan. Two major concerns were addressed: Has the Agency reported on its performance with respect to its objectives? Is that information fair and reliable? Performance information with respect to objectives is fair and reliable if it enables Parliament and the public to judge how well the entity or program in question is performing against the objectives it set out to accomplish.

More information on the criteria is available on our Web site at www.oag-bvg.gc.ca.