Formative Evaluation of the Avian Influenza and Pandemic Influenza (AI/PI) Preparedness Initiative

March 23, 2010

Management Response and Action Plan

Table of Contents

• Executive Summary
• 1. Introduction
• 2. Overview of the AI/PI Initiative
  • Vaccines and antivirals
  • Surge capacity
  • Prevention and early warning
  • Emergency preparedness
  • Critical science and regulation
  • Risk communications to enhance public awareness
  • Federal, provincial/territorial, and international cooperation
  • Resources
  • Stakeholders and beneficiaries
  • Governance
• 3. Methodology
• 4. Evaluation Findings
  • Implementation
  • Governance and process
  • Performance measurement and reporting
  • Success
  • Cost effectiveness and alternatives
  • Relevance
• 5. Conclusions and Recommendations
• Appendix A – Detailed Context for the AI/PI Initiative
• Appendix B – Evaluation Framework
• Appendix C – Interview Guide
• Appendix D – Supplementary Data Tables
  • Table 1 Status of implementation of AI/PI components and activities
  • Table 2 Recommendations from past outbreaks
  • Table 3 Areas of support and stakeholders in Alberta FADES agreement
  • Table 4 Pandemic influenza and emergency response activity of other federal departments
  • Table 5 AI/PI activities and responsibilities (as of June 2007)
  • Table 6 Major documents reviewed by the evaluation

---

Executive Summary

Overview of the Avian Influenza and Pandemic Influenza Preparedness Initiative

In response to the animal and public health threats posed by avian influenza (H5N1), the 2006 federal budget, under the umbrella “Preparing for Emergencies,” described commitments of $1 billion over five years to improve Canada’s pandemic preparedness, including $600 million to be allocated to departments and agencies and $400 million as a contingency fund. The $600 million includes funding for the Public Health Agency of Canada ($367 million), the Canadian Food Inspection Agency ($195 million), Health Canada ($16.5 million), and the Canadian Institutes of Health Research ($21.5 million), as part of the Avian Influenza and Pandemic Influenza Preparedness (AI/PI) Initiative.

The AI/PI Initiative is intended to be a “coordinated, multi-disciplinary approach to mitigating the health impacts of AI/PI on Canadians, protecting critical Canadian infrastructure, and ensuring Canadians of essential services.” At the CFIA, the Initiative consists of seven components:

1. Vaccines and antivirals
2. Surge capacity
3. Prevention and early warning
4. Emergency preparedness
5. Critical science and regulation
6. Risk communications to enhance public awareness
7. Federal, provincial/territorial, and international cooperation

The objectives of the Initiative are “to respond to the immediate animal health impacts of AI while increasing preparedness for a potential pandemic.”

Purpose of the evaluation

In accordance with Treasury Board requirements, the Canadian Food Inspection Agency (CFIA) conducted this formative¹ evaluation of the AI/PI Initiative. The primary purpose of the evaluation was to assess whether the Initiative was implemented as planned and, to the extent possible, to report on outcomes. This report summarizes the evaluation findings, draws conclusions, and provides recommendations to guide decision making.

Methodology

The evaluation methodology consisted of an extensive document review and in-depth interviews with 44 key informants, including current and former CFIA personnel as well as external stakeholders.

Key findings and conclusions

Overall, the evaluation found that the AI/PI Initiative was implemented mostly as planned and has greatly enhanced the CFIA’s capacity to respond to avian influenza emergencies and animal health emergencies in general. However, there are opportunities to enhance the overall management of the Initiative through better coordination of Initiative activities within the CFIA and through improvements to performance monitoring and financial reporting, without which the Initiative’s future success will be difficult to demonstrate.

Implementation

• The Initiative has been implemented mostly as planned.
• Some activities, such as biosecurity, risk assessment, and risk communications, among others, spent less than half of what was planned, while others spent more than twice the planned amount.

Governance

• Relationships with the CFIA’s external partners—including other federal departments, the provinces and territories, regional laboratories, and industry—have been strengthened as a result of the Initiative.
• The Agency’s practice of systematically reviewing its response to AI outbreaks and adjusting its activities based on lessons learned has produced numerous improvements to its response to avian influenza.
• Performance monitoring is not centrally managed, indicators are not centrally tracked, and there appears to be a lack of basic reporting templates.
• Financial reporting is not directly aligned with the Initiative outputs and outcomes.
• No formal risk assessment of the Initiative has been undertaken.

Success

• The infrastructure created as a result of AI funding has enhanced the CFIA’s animal health emergency response capacity.
• The Initiative has produced a more comprehensive and harmonized response to avian influenza.
• The surveillance/detection network has been effective.
• The CFIA appears to have an adequately trained, skilled, and equipped workforce, and adequate surge capacity to face AI outbreaks of the size experienced to date.
• The Initiative’s ability to demonstrate success will be limited without more comprehensive collection of both qualitative and quantitative performance data.

Cost effectiveness

• The infrastructure created as a result of the Initiative is applicable to other animal health emergencies.
• The Initiative has leveraged significant other public and private resources.

Relevance

• While there are other initiatives in Canada that address zoonotic disease and pandemic influenza, the AI/PI Initiative is unique in focusing on animal health.
• The broad objectives of the Initiative—achieving a better level of emergency preparedness and improving public safety—are consistent with current Agency and federal priorities.
• Although AI has faded from the public consciousness, the threat remains. This suggests an ongoing need for preparedness initiatives at the national level.

Recommendations

Recommendation 1. To address gaps in central oversight, the Operations Branch and the Policy and Programs Branch should take steps to improve the internal coordination of the AI/PI Initiative by, for example, ensuring data collection and expenditures are overseen by no more than one Agency office each and that regular Initiative status reports are distributed to those responsible for its outputs.

Recommendation  2. The CFIA should review its performance measurement system for the AI/PI Initiative and present the results of the review to the Evaluation Committee at its September 2010 meeting. The following items should be addressed in the new approach:

1. review the Initiative’s performance measurement strategy to identify the most critical performance indicators, both existing and new, that can most realistically be collected immediately and retroactively, eliminating unessential indicators and ensuring that tracking begins;
2. implement a system for reporting performance information, which could include development of standardized templates or reporting tools;
3. assign responsibility for performance reporting to specific positions within the Agency; and
4. ensure financial information is linked to Initiative outputs and outcomes.

1. Introduction

In response to the animal and public health threats posed by avian influenza (H5N1), the 2006 federal budget, under the umbrella “Preparing for Emergencies,” described commitments of $1 billion over five years to improve Canada’s pandemic preparedness, including $600 million to be allocated to departments and agencies and $400 million as a contingency fund. The $600 million includes funding for the Public Health Agency of Canada ($367 million), the Canadian Food Inspection Agency ($195 million), Health Canada ($16.5 million), and the Canadian Institutes of Health Research ($21.5 million) as part of the Avian Influenza and Pandemic Influenza Preparedness (AI/PI) Initiative.

In accordance with Treasury Board requirements, the Canadian Food Inspection Agency (CFIA) conducted this formative evaluation of the AI/PI Initiative. The primary purpose of the evaluation was to assess whether the Initiative was implemented as planned and, to the extent possible, to report on outcomes. This report summarizes the evaluation findings, draws conclusions, and provides recommendations to guide decision making.

The report is organized into five sections: Section 2 provides a descriptive overview of the AI/PI Initiative; Section 3 presents the methodology used to complete the evaluation; Section 4 presents the evaluation findings; and Section 5 summarizes the findings and draws conclusions.

2. Overview of the AI/PI Initiative

The CFIA’s AI/PI Initiative was announced in 2006 in response to the animal and public health threats posed by the H5N1 influenza virus.² The Initiative involves a variety of elements including import restrictions, border security, surveillance, emergency preparedness measures, and international cooperation. It is intended to be a “coordinated, multi-disciplinary approach to mitigating the health impacts of AI/PI on Canadians, protecting critical Canadian infrastructure, and ensuring Canadians of essential services.”³ The objectives of the CFIA component of the Initiative are “to respond to the immediate animal health impacts of AI while increasing preparedness for a potential pandemic.”

The CFIA’s AI/PI Initiative consists of seven components:

1. Vaccines and antivirals
2. Surge capacity
3. Prevention and early warning
4. Emergency preparedness
5. Critical science and regulation
6. Risk communications to enhance public awareness
7. Federal, provincial/territorial, and international cooperation

Vaccines and antivirals

The purpose of this component is twofold:

1. to conduct research on vaccination as a strategy against the spread of AI with the objective of developing a policy on the effective use of vaccination; and
2. to make plans to protect front-line responders to an AI outbreak through access to antivirals.

Surge capacity

This component determines the surge capacity – or level of additional human resources, equipment, and facilities – necessary to address a specific AI threat, and to develop contingency plans to add resources and capacity as needed.

Surge capacity planning focuses on four main areas:

1. Internal staffing reserve: ensuring the CFIA has enough staff, back-up staff, and staff rotation for the duration of an AI outbreak.
2. External staffing reserve: ensuring the CFIA has identified and trained external human resources, with relevant experience, who can be deployed in an outbreak.
3. Equipment: ensuring the CFIA can obtain and deploy the disposal and depopulation equipment necessary to address an AI outbreak, including personal protective equipment for CFIA staff and vehicles.
4. Laboratories: ensuring AI is detected as early as possible to permit timely diagnosis and effective responses.

Prevention and early warning

This component includes several measures to limit the introduction and spread of AI into Canada’s domestic poultry populations.

Wild bird surveillance: The CFIA works with federal, provincial, and industry partners. The goal is to better understand the presence and characteristics of typical strains of AI in North America’s wild bird population, especially those that are, or have the potential to become, highly pathogenic.

Commercial bird surveillance: Design and implementation of the CFIA as a national AI poultry survey/surveillance program in collaboration with industry. Objectives include providing early detection of and response to the AI virus to minimize the impacts of an outbreak, helping to maintain access to the international market for Canadian poultry products, and preserving the international reputation of Canada’s foreign animal disease program.

Biosecurity: The CFIA’s Office of Animal Biosecurity (OAB) works in collaboration with stakeholders from industry, provincial/territorial and federal government departments, and academia to lead and coordinate the development and implementation of biosecurity strategies, programs, and on-farm guidelines and standards.

Intelligence gathering: Through international collaboration, Canada monitors AI developments around the world and makes appropriate adjustments to import controls and disease response plans.

Emergency preparedness

The CFIA’s emergency preparedness approach focuses on five main areas:

1. emergency response teams;
2. development of detailed procedures for response;
3. AI scenarios and exercises;
4. federal, provincial/territorial, and international cooperation (described below); and
5. surge capacity (described above).

Emergency response teams: Develop response capacity in each of the CFIA’s operational areas (Atlantic, Quebec, Ontario, and Western). The teams would be responsible for the CFIA’s emergency response and for coordinating actions with federal, provincial, and municipal partners.

Development of detailed procedures for response: Develop contingency plans for every activity associated with an outbreak.

AI scenarios and exercises: Conduct ongoing emergency preparedness workshops, exercises, and training events, as well as participate in other government departments’ (OGD) and industry exercises to test responses to AI in different regions of Canada.

Critical science and regulation

This component is intended to achieve long-term preparedness for the assessment and mitigation of the consequences of AI.

Critical science: The CFIA carries out scientific risk assessments and animal-disease modeling, which contribute to improved preparedness and response plans. Other areas of research include virus transmission, pathogenesis and ecology, vaccine efficacy, humane euthanasia, and effective disposal methods.

Regulation: The CFIA, in conjunction with provincial/territorial governments, animal and public health experts, and industry representatives, develops models and conducts cost-benefit analyses for regulatory submissions, as required. Other activities include reviewing regulatory amendments pertaining to AI, developing indicators for regulatory pieces, and preparing background policy documents in support of AI regulations.

Risk communications to enhance public awareness

The CFIA is undertaking two main national risk communication campaigns to “…bolster Canada’s AI readiness and response capacity, fully engage the critical participation of stakeholders and provincial and territorial governments, and inform and reassure Canadians.” The campaigns focus on the two likely pathways for the introduction and spread of AI into Canada—international trade and travel, and transmission from migratory birds to domestic poultry.

Federal, provincial/territorial, and international cooperation

The CFIA pursues partnerships with other federal government departments, provincial/territorial governments, and international organizations such as the World Organisation for Animal Health (OIE) and the Food and Agriculture Organization of the United Nations (FAO). The aim of these partnerships is “achieving national and international collaboration with respect to the creation of /commitment to strategies to control AI.”

Resources

Table 1 summarizes the resources allocated to the CFIA’s AI/PI Initiative, adjusted for the material events which impacted the funding’s profile.

Table 1: AI/PI resource profile ($)

	2006–07	2007–08	2008–09	2009–10	2010–11	Total	2011–12 & ongoing
Vaccines and antivirals	630,000	630,000	630,000	630,000	630,000	3,150,000	130,000
Surge capacity	17,180,000	51,470,000	12,960,000	12,590,000	12,540,000	106,740,000	12,490,000
Prevention and early warning	2,540,000	3,110,000	3,110,000	3,110,000	3,110,000	14,980,000	3,060,000
Emergency preparedness	2,840,000	5,460,000	5,460,000	4,600,000	4,700,000	23,060,000	4,700,000
Critical science and regulation	4,440,000	6,650,000	6,420,000	5,960,000	5,900,000	29,370,000	5,855,000
Risk communications	2,450,000	2,850,000	1,860,000	1,550,000	1,550,000	10,260,000	1,550,000
F/P/T/ International collaboration	920,000	1,830,000	1,560,000	1,560,000	1,570,000	7,440,000	1,570,000
sub-total (as per TB Submission)	31,000,000	72,000,000	32,000,000	30,000,000	30,000,000	195,000,000	29,355,000
adjusted for:
Strategic Review	0	0	-16,950,000	-4,050,000	-4,050,000	-25,050,000	-4,050,000
INTERvac Transfer	0	-2,000,000	-4,000,000	0	0	-6,000,000	0
ARLU Reprofiled Capital	0	-20,000,000	20,000,000	0	0	0	0
TOTAL	31,000,000	50,000,000	31,050,000	25,950,000	25,950,000	163,950,000	25,305,000

Source: FAIT, Resource Management

Note:

• the above figures do not reconcile to the RMAF table because the latter does not capture the $6M INTERvac transfer
• as well, the above table does not reconcile to the information published in the DPR for the $2M 2007-2008 INTERvac transfer is not captured in the latter document

Stakeholders and beneficiaries

Numerous partners and stakeholders are involved in the AI/PI Initiative. Federal departments include Agriculture and Agri-Food Canada (AAFC), Health Canada (HC), the Public Health Agency of Canada (PHAC), Environment Canada (EC), Foreign Affairs and International Trade Canada (DFAIT), and Public Safety Canada (PSC). Stakeholders include other federal organizations, provincial/territorial governments, industry associations, animal health practitioners, a network of provincial and university laboratories, and the Canadian poultry industry. Immediate beneficiaries are poultry producers, but ultimately the entire Canadian public is expected to benefit.

Governance

The Avian Influenza Working Group (AIWG) was established in February 2006, with membership from across the CFIA. The AIWG provided direction, project oversight, and advice to senior management on all AI-related issues until July 2007, when lead responsibilities for all aspects related to ongoing activities associated with AI preparedness were transferred to the branches. At that time, the Avian Influenza Preparedness Oversight Committee (AIPOC) was also established, with leadership from the Operations Branch and the Policy and Programs Branch to provide oversight to the AI/PI Initiative activities stemming from Budget 2006 funding and to ensure the smooth transition of files, monitoring, and reporting functions. AI/PI Initiative activities and the CFIA branches responsible for them are itemized in Table 5 of Appendix D.

In March 2008, the scope of the AIPOC was expanded to include oversight for all of the CFIA’s animal health emergency planning. Now known as the Animal Health Emergency Preparedness Oversight Committee (AHPOC), the Committee is an inter-branch forum responsible for:

• “Monitoring animal health emergency preparedness and identifying issues that require input from CFIA senior management.
• Receiving, reviewing and assessing performance information as provided by Audit Evaluation and Risk Oversight (AERO).
• Reporting to VPs and branch heads on the status of animal health emergency preparedness and any resource or key delivery issues.”⁴

Expected results

The logic model for the AI/PI Initiative contains seven immediate results, two intermediate results, and three ultimate results. The formative evaluation of the AI/PI Initiative is focused on the extent to which the immediate outcomes have been achieved, in addition to assessing whether the Initiative has been implemented as planned.

Table 2: Expected results of the AI/PI Initiative

Immediate results	Comprehensive and harmonized response Increased capacity to address regulatory and economic options associated with AI Trained, skilled, equipped workforce Increased surge capacity Mitigated risk associated with importation Improved diagnostics/avian vaccines, targeted AI detection of high-risk poultry groups Proactive coordinated communications to address AI-related concerns among industry and public
Intermediate results	Increased virus protection for poultry Maintained confidence among industry and public
Long-term results	Protected Canadian economy against impacts of AI virus outbreak Increased protection of public health Protected domestic poultry

3. Methodology

The evaluation framework contained in the RMAF for the AI/PI Initiative was refined and elaborated. The framework is the foundation on which the evaluation study is based: it covers issues of relevance, design and delivery, governance, performance measurement and reporting, success, and cost-effectiveness. The framework is in Appendix B. An Advisory Committee composed of senior CFIA managers was established to provide strategic and technical advice to the CFIA’s Head of Evaluation.

The evaluation methodology consisted of two main data collection methods: document review and key informant interviews.

Document review

The document review focused on materials relevant to the development, implementation, and management of the AI/PI Initiative, as well as externally produced materials pertaining to avian and pandemic influenza. The document review provided important contextual information for the Initiative and helped answer many of the evaluation questions in the framework. The major documents reviewed are listed in Table 6 of Appendix D.

Key informant interviews

A total of 44 key informants were interviewed using a combination of group and individual interviews. Key informants were identified by the CFIA and included current and former CFIA personnel, as well as a small number of external stakeholders. The interview guide is in Appendix C.

4. Evaluation Findings

Implementation

Overall, based on the evidence available to this evaluation, the AI/PI Initiative has been implemented mostly as planned. Most activities under all seven components of the Initiative have been implemented.⁵ Detailed information on the implementation status of each component and activity can be found in Appendix D.

Approximately three quarters (74%) of funds allocated to the Initiative was spent on Initiative activities in its first three fiscal years (see Table 3, below).

There have been significant differences in spending patterns among the components of the Initiative, as itemized in Table 3.

• Actual spending exceeded budgeted allocations for two components: vaccines and antivirals (123%), and prevention and early warning (141%).
• Actual spending approximated allocations for the emergency preparedness component (98%), but only because expenditures were significantly greater than planned on AI‑enhanced management capability (335%). Conversely, spending on field training and updated emergency response plans was about half of budgeted allocations (55% and 51%, respectively).
• Actual spending was considerably less than planned for federal/provincial/territorial and international cooperation (71%), surge capacity (66%), critical science and regulation (61%), and risk communications (48%).

There are also significant discrepancies in the ratio of actual to planned spending within components. The emergency preparedness component described above is one example. Another is found within the surge capacity component, where spending ranged from 13% of allocations for specialized equipment to 210% for field surge capacity.

Certain activities have been especially affected by lower-than-planned spending, including activities in support of a strengthened economic and regulatory framework (7%), specialized equipment (14%), AI risk assessment and modeling (30%), avian biosecurity on farms (46%), risk communications (48%), the national veterinary reserve (49%), and updated emergency response plans (51%).

Table 3: Comparison of planned and actual spending by component and activity of the AI/PI Initiative, fiscal years 2006–2007 to 2008–2009
(rounded to the nearest hundredth thousand $)

Vaccines and Antivirals - 2006-2007 to 2008-2009

Component/ activity	Planned spending ($)	Actual spending ($)	Actual/planned spending (%)
Animal vaccine bank	1,000,000	1,080,000	108%
Priority access to antivirals	200,000	400,000	200%
Subtotal	1,200,000	1,480,000	123%

 

Surge Capacity - 2006-2007 to 2008-2009

Component/ activity	Planned spending ($)	Actual spending ($)	Actual/planned spending (%)
Specialized equipment	20,500,000	2,800,000	14%
AI lab surge capacity/capability	14,600,000	12,200,000	84%
Field surge capacity	3,100,000	6,500,000	210%
National veterinary reserve	4,900,000	2,400,000	49%
Enhancement enforcement measures	3,700,000	5,000,000	135%
Avian biosecurity on farms	12,400,000	5,700,000	46%
Real property (quarantine stations)	4,000,000	7,000,000	175%
Subtotal	63,200,000	41,600,000	66%

 

Prevention and Early Warning - 2006-2007 to 2008-2009

Component/ activity	Planned spending ($)	Actual spending ($)	Actual/planned spending (%)
Domestic and wildlife surveillance	8,200,000	11,600,000	141%
Subtotal	8,200,000	11,600,000	141%

 

Emergency Preparedness - 2006-2007 to 2008-2009

Component/ activity	Planned spending ($)	Actual spending ($)	Actual/planned spending (%)
Field training	4,700,000	2,600,000	55%
AI-enhanced management capability	2,000,000	6,700,000	335%
Updated emergency response plans	5,700,000	2,900,000	51%
Subtotal	12,400,000	12,200,000	98%

 

Critical Science and Regulation - 2006-2007 to 2008-2009

Component/ activity	Planned spending ($)	Actual spending ($)	Actual/planned spending (%)
AI risk assessment and modeling	7,400,000	2,200,000	30%
AI research	3,500,000	4,100,000	117%
Strengthened economic and regulatory framework	2,800,000	200,000	7%
Performance and evaluation	2,700,000	3,500,000	130%
Subtotal	16,400,000	10,000,000	61%

 

Risk Communications - 2006-2007 to 2008-2009

Component/ activity	Planned spending ($)	Actual spending ($)	Actual/planned spending (%)
Risk communications	6,900,000	3,300,000	48%
Subtotal	6,900,000	3,300,000	48%

 

F/P/T and International Cooperation - 2006-2007 to 2008-2009

Component/ activity	Planned spending ($)	Actual spending ($)	Actual/planned spending (%)
Support for international commitments	4,100,000	2,920,000	71%
Subtotal	4,100,000	2,920,000	71%
TOTAL	112,400,000	83,100,000	74%

Source: adjusted planned spending figures provided by FAIT, Financial Services whereas actual spending sourced from PHAC’s DPRs (as lead for this horizontal initiative, the data are accumulated and published in their documents).

Note: the above figures do not reconcile to those in Table 1, for the former are rounded to the nearest hundredth thousand

Adequacy of activities and resources

While most key informants believe that activities and resources are adequate, the following concerns were raised:

• Although the CFIA has demonstrated its ability to respond to AI outbreaks of the size experienced in Canada to date, it may not have the surge capacity to handle a larger outbreak, and may not be able to maintain business continuity in that event.
• Emergency exercises have been less frequent in recent years, but should be ongoing, particularly in light of high staff turnover. Indeed, more than half (28) of the 47 emergency exercises carried out to date were conducted in 2006 and 15 in 2007. Only four exercises were conducted in 2008 and none in 2009, and no planning documents identified how many were and are still planned.
• Ongoing or additional efforts and resource commitments are required to maintain the state of preparedness attained as a result of the Initiative, and to achieve a satisfactory level of broader pandemic preparedness.

Governance and process

The governance structure of the AI/PI Initiative was appropriate at the outset, with clear roles, responsibilities, and accountabilities. However, recent changes have reduced some oversight capacity.

Governance of the Initiative within the CFIA

Most key informants said that roles, responsibilities, and accountabilities within the CFIA are clear. Key informants saw the governance structure as particularly effective at the outset of the Initiative, when it was the responsibility of the AIWG. The AIWG Project Plan Manual, which included detailed work plans and time frames for all AI activities, outlined a high level of planning and coordination in the first year of the Initiative.

When the AIWG was disbanded in June 2007, responsibility for implementation of activities was transferred to the branches. At that time, the Avian Influenza Preparedness Oversight Committee (AIPOC) was also established to provide oversight for the CFIA’s AI activities.

In March 2008, the scope of the AIPOC was expanded to include oversight for the CFIA’s entire animal health emergency planning, and the Committee was renamed the Animal Health Emergency Preparedness Oversight Committee (AHPOC). According to the Terms of Reference for the AHPOC, the Committee is an inter-branch forum co-chaired by the Director, Office of Emergency Management (Operations Branch), and the Director, Animal Health and Production Division (Policy and Programs Branch). Membership includes the Executive Director, National Operations Directorate; the Director, Research and Development (Science Branch); the Senior Communications Officer (Public Affairs); the Director, Corporate Planning and Risk Management (Corporate Secretariat); the Chief Information Officer (FAIT); the Director, Horizontal Policy Division; and the Director, Human Resources Operations Division. Under the terms of reference, the Committee was mandated to meet monthly.

Some key informants observed that there have been fewer high-level discussions pertaining to AI since the AIWG was disbanded. Only seven meetings of the AIPOC/AHPOC were held between November 2007 and September 2009. Some key informants suggested that this may be because most activities have been implemented or were well under way, and frequent discussions were no longer necessary. These key informants believe that Initiative activities are effectively coordinated within the CFIA.

However, others believe that a greater degree of coordination is required, and that linkages among all of the CFIA’s business units are not as strong as they should be. There does not appear to be one office at the CFIA that can ensure the Agency speaks with one voice on AI/PI issues. They emphasized the importance of an integrated approach that engages all internal stakeholders (including all branches) in program design, development, implementation, and ongoing delivery. In a related vein, it is important to note that some CFIA key informants, including some senior managers, were unaware of or unfamiliar with the AI/PI Initiative when approached to participate in an interview. However, they were familiar with the components for which they were responsible or in which they were involved. This could be an indication that, as some key informants suggested, internal coordination and linkages within the CFIA are not as effective as they could be.

Finally, a few key informants observed that, when an animal health emergency arises, some branches of the CFIA do not always operate within the Incident Command Systems (ICS) structure, but continue with usual management practices. Emergency response teams are organized using the ICS concept, which is based on functions. Generally, within the ICS structure, there are five functional sections: command, operations, planning, logistics, and finance. Each section may have several branches, which in turn may have a number of units. Each section is responsible for functional tasks as part of the management of an incident situation.⁶ Informants perceived a need for more training and exercises to ensure a seamless transition from normal practices to the ICS structure.

Internal CFIA management practices

In addition to considering the effectiveness of the CFIA’s governance structure, the evaluation examined the extent to which the CFIA undertakes risk assessment and lessons learned review to manage and make adjustments to the AI/PI Initiative.

Risk assessment

The evidence available for the evaluation shows that the CFIA has undertaken some risk analysis activities in relation to avian influenza. In May 2006, an Integrated Interdepartmental Avian Influenza Risk Analysis with the following objectives was undertaken: bringing together available information about the risks posed by introduction of HPAI H5N1 (Eurasian strain) in Canada; identifying knowledge gaps; developing strategies to address these gaps; providing advice on joint mitigation measures and risk management; establishing a network and leveraging resources for preparedness; contributing to a one-window approach to risk communication.⁷ This exercise, which involved the CFIA, PHAC, HC, EC, PSC, and the Canadian Cooperative Wildlife Health Centre, was completed in June 2006.

Subsequently, the December 2008 RMAF for the AI/PI Initiative identified a number of risks and mitigation strategies associated with avian influenza based on the CFIA’s Report on Plans and Priorities 2006–07. The RMAF identified the AI/PI Initiative as the main mitigation strategy in relation to the risks it identified, but noted that a true risk analysis was not carried out for the purpose of the RMAF.

Beyond these activities, the CFIA’s Science Branch, which is responsible for conducting scientific risk assessments related to AI, has initiated or completed various risk assessments on AI, including those related to importation of live birds from AI-infected countries and animal product that might contain the AI virus. It could also be argued that lessons learned reviews (described below) and business continuity planning are also aspects of the CFIA’s approach to managing risk. However, based on the evidence available for the evaluation, it is not clear whether and how often CFIA management systematically reviews and updates the risks associated with the Initiative.

Lessons learned review

Lessons learned review has become a feature of the CFIA’s management approach. The CFIA systematically reviews and reports on its response to past outbreaks and emergency exercises and makes adjustments to its AI activities in response to recommendations, lessons learned, and best practices identified through the review process. Similarly, plans and procedures are evergreen documents and subject to continual review and updating as a result of lessons learned from review of past outbreaks and emergency exercises. Examples of recommendations implemented as a result of the BC outbreak in 2004 (which preceded the AI/PI Initiative) are as follows:

• develop and exercise Foreign Animal Disease Emergency Support (FADES) plans in all provinces;
• develop collaborative arrangements with HC and PHAC to increase federal capacity to respond to zoonotic disease outbreaks;
• review and formalize emergency response protocols;
• implement ICS during emergencies;
• involve the poultry industry in development of biosecurity programs; and
• develop Standard Operating Procedures to ensure human resources, finance, IT/IM and administrative support to Emergency Operations Centres.

More recently, some of the recommendations implemented or in progress as a result of the lessons learned review of the 2007 outbreak in Saskatchewan were as follows:⁸

• work with key partners at the regional level to better coordinate inter-jurisdictional flows and triggers with other federal departments and agencies (see below);
• propose plans for increasing internal surge capacity to handle future emergencies at both the national and regional levels (see Implementation above and Appendix D); and
• strengthen and continue to practise roles and responsibilities in advance through increased exercises of the Incident Command System at both the national and regional levels (see Implementation above and Appendix D).

See Appendix D for a more complete listing of recommendations from past outbreaks implemented or in progress.

Roles and responsibilities with respect to federal partners

The CFIA’s key federal partners in the AI/PI Initiative are AAFC, Canada Border Services Agency, the Canadian Wildlife Service, EC, PHAC, Public Safety Canada, and Transport Canada. Key informants generally believe that roles, responsibilities, and accountabilities of all federal partners are clear. However, it was noted that much work has been done through the Initiative to clarify their respective roles and responsibilities in the event of an AI outbreak. To this end, as described below in Annex D–Table 1, a variety of frameworks, plans, and protocols have been developed in collaboration with other federal departments to govern Canada’s response to animal disease emergencies.

Roles and responsibilities with respect to provinces

Although the CFIA had partnerships in place with the provinces prior to the Initiative, the FADES agreements, developed under the AI/PI Initiative, have contributed significantly to clarifying roles and responsibilities. Key informants generally believe that roles, responsibilities, and accountabilities with respect to the provinces are clear, and that these partnerships are effective in delivering Initiative programs.

The purpose of the FADES agreements is to “…provide a concept with which federal, provincial and local governments will provide the assistance required by the CFIA for an eradication of a FAD outbreak”⁹ in a given province. As such, the FADES agreements describe the structure of emergency response in a province, the procedures to activate the plan, and the roles and responsibilities of federal, provincial, and local authorities, as well as industry associations, in the event of a foreign animal disease outbreak. Review of two FADES agreements (Alberta and British Columbia) suggests that these agreements do clearly articulate roles and responsibilities in such emergencies.¹⁰

The recommendation to develop and exercise FADES agreements with all provinces was made in the Lessons Learned Review report on the 2004 AI outbreak in British Columbia,¹¹ and to date, such agreements have been signed with all of the provinces except Newfoundland and Labrador. Key informants agreed that there is now much better preparedness to respond to AI and other animal health emergencies as a result of the FADES agreements. All areas and virtually all regions have had occasion to test the agreements during emergency exercises and, in the case of some provinces, during actual AI events.¹² The CFIA has also established and maintains AI communications and liaison networks in each of its four areas (Atlantic, Quebec, Ontario, and Western). Area communications protocols and contact lists are kept current and made accessible to Area/Regional Management Teams, and these networks enable each area to partner with stakeholders to support CFIA efforts.¹³ The provinces are also, of course, involved in the Canadian Animal Health Surveillance Network (CAHSN) (described below).

Roles and responsibilities with respect to regional laboratories

The CFIA has also partnered with provincial and university laboratories involved in the CAHSN to enhance diagnostic surge capacity for AI. As reported below in Appendix D–Table 1, 10 provincial or university laboratories that are part of the CAHSN have received AI funding for upgrading their facilities and have been accredited for AI testing. Key informants believe that roles, responsibilities, and accountabilities with respect to the regional laboratories are, for the most part, clear.

However, key informants raised several issues with respect to the regional laboratories. First, there is the need to provide ongoing training to laboratory analysts and to update laboratory equipment that is nearing the end of its lifespan. Key informants noted that the CAHSN’s capacity to do AI diagnostic testing will be at risk if other sources of funding are not found. Second, there is still considerable work to be done on information management, data sharing, and reporting. Key informants noted challenges in obtaining federal-provincial agreement on data sharing and privacy matters can been challenging. Third, it is not clear who will be responsible for the ongoing costs of operating AI diagnostic testing. Finally, the role of the laboratories with respect to testing for other types of (non-avian) influenza is not clear.

Performance measurement and reporting

The CFIA has taken some steps towards implementing a performance measurement system for the AI/PI Initiative. An RMAF has been developed consisting of the necessary elements of such a system, including a description of the expected results associated with each component, a logic model, a description of planned performance measurement and evaluation strategies, a performance measurement framework (including outputs and outcomes, indicators, sources of information, responsibility, and timing of monitoring), and an evaluation framework.

The CFIA reports on the performance of the AI/PI Initiative in its annual Departmental Performance Reports and PHAC’s Departmental Performance Reports (Horizontal Initiatives). However, there was no evidence of the number of performance indicators the CFIA is tracking, as identified in the RMAF, or that it is carrying out all of the planned performance monitoring activities (e.g., producing quarterly, semi-annual, or annual reports on specific indicators). It will be difficult to assess the success of the Initiative with incomplete tracking of performance information.

In addition, based on the experience of the evaluation, obtaining data on the status and impact of the Initiative and its activities is difficult and time-consuming. As noted above, it is not clear if all planned performance indicators are being tracked. However, information on outputs and outcomes is evidently being collected and held within the various the CFIA branches responsible for specific activities, rather than within a centralized repository or information management system. Several key informants identified a need for IT/IM tools to facilitate and support performance monitoring and reporting. In most cases, basic tools such as Excel spreadsheets and Access databases are adequate to support tracking of performance information, and no additional IM/IT support should be required.

It is unclear what mechanisms, if any, are in place to track the intermediate and long-term outcomes of the Initiative. These include increased virus protection for poultry, maintained confidence among industry and the public, protection of the Canadian economy against impacts of an AI virus outbreak, increased protection of public health, and protection of domestic poultry. This is an area that may need attention, since the summative evaluation will be required to report on these outcomes. This will be challenging if the CFIA is not tracking these outcomes on an ongoing basis.

Finally, it is important to note that the CFIA’s approach to financial reporting on the AI/PI Initiative is not entirely consistent with its approach to performance reporting. More specifically, the Initiative activities identified for the purpose of financial reporting (see Table 1) do not exactly match the activities identified in the RMAF. Some activities appear under different components, while others are listed for the purpose of financial reporting but are not mentioned at all in the RMAF. In other cases, slight differences in the way in which activities are identified make it difficult to know if the same activity is being referenced.¹⁴ These discrepancies make it difficult to understand how Initiative resources have been allocated and spent. Indeed, many key informants noted that there is a need to improve financial reporting on the Initiative, including linking funding to outputs and outcomes.

Success

This formative evaluation examined the extent to which the immediate outcomes of the AI/PI Initiative have been achieved. The main points are highlighted here.¹⁵

Comprehensive and harmonized response: Overall, the AI/PI Initiative has produced a more comprehensive and harmonized response to avian influenza. Within the CFIA, numerous protocols and detailed procedures for response have better prepared the Agency to respond to AI outbreaks. Relationships with other stakeholders such as other federal departments, provinces, regional laboratories, and industry have also been strengthened as a result of the various frameworks and agreements that have been put in place. Moreover, stakeholders have had the opportunity to practise emergency response procedures through AI scenarios and exercises, as well as in actual outbreaks, and the CFIA regularly reviews lessons learned following each outbreak and exercise and makes adjustments to emergency response plans.

Increased capacity to address regulatory and economic options associated with AI: The CFIA has explored regulatory options that could result in more effective control of disease spread, assessed compensation options for industry, and initiated the development of appropriate cost valuation models.

Trained, skilled, and equipped workforce: The evidence shows that there is an adequately trained, skilled, and equipped workforce to face AI outbreaks of the size experienced to date. The external staffing reserve (204 trained), the emergency exercises with provinces (42 across the country) and the creation of the Canadian Veterinary Reserve represent a significant capacity for skilled response. The Agency has a successful record in responding to several AI outbreaks in the past.

Adequate surge capacity: The CFIA has the surge capacity in terms of external reserve staff, infrastructure, equipment, and laboratory capacity to handle AI outbreaks of the size that have been experienced to date. However, some parts of the country, particularly the West, have better surge capacity than others because they have had several AI outbreaks and, therefore, have had the opportunity to put emergency response procedures into practice.

Effective surveillance/detection network: The surveillance/detection network has been effective to date. Provincial laboratories, through passive surveillance, detected the index (or initial) case in both the 2004 British Columbia and 2007 Saskatchewan outbreaks.¹⁶ Two other components of the commercial surveillance system (pre-slaughter serological surveillance and voluntary enhanced active notifiable avian influenza surveillance) have been implemented, with three other components to follow. The wild bird survey, which pre-existed the Initiative, is continuing as part of the surveillance/detection network, and new import requirements have been put in place for all countries to mitigate risks associated with importation.

Adequate diagnostic techniques and avian vaccines: Key informants believe that while there are adequate diagnostic techniques as a result of the veterinary infrastructure and laboratory capacity that have been developed, the laboratory network could become overwhelmed if the scope of the outbreak were extensive. With respect to vaccines, the CFIA’s strategy has evolved away from vaccination stockpiling as a strategy, but has ensured a reasonable supply through a vaccine sharing agreement between Canada, the US, and Mexico.

Effective and timely communications mechanisms: The CFIA Web site includes extensive information on AI for the general public and for poultry producers. The Agency is currently doing proactive communications work on traveller biosecurity and farm biosecurity. As public concern about AI has dissipated and the public’s attention has shifted to other concerns, the CFIA no longer regularly tracks public opinion on AI.

Unintended impacts

Key informants identified several positive unintended impacts of the AI/PI Initiative. Most importantly, they noted that the infrastructure and understanding of roles and responsibilities created as a result of AI funding has been instrumental in enhancing Canada’s preparedness for animal health emergencies in general, and not just for AI. Key informants reported that aspects of the CFIA’s approach to AI emergency response have been applied to H1N1 and in future will apply to other animal health diseases such as foot-and-mouth disease. They also noted that public awareness of the human health implications of animal health diseases has been heightened and that, internationally, perceptions of Canadian food safety have improved.

Cost effectiveness and alternatives

Based on the evidence available for the evaluation, it is difficult to determine if the Initiative is optimizing outputs or if resources are appropriately allocated. While all the CFIA representatives interviewed believe their own branches are using AI resources efficiently, many identified a need for improved tracking of AI resource allocations and a need to link these to outputs and outcomes. This would facilitate an objective assessment of the efficiency of the Initiative.

Key informants believe that the infrastructure created as a result of AI funding has enhanced the ability of the CFIA and its partners to respond to all animal health emergencies, not just avian influenza. The Initiative has also leveraged a variety of other public and private resources, among them veterinary schools, private veterinarians, provincial and university laboratories, research partnerships with academic institutions, and in-kind contributions from industry with respect to biosecurity measures. For example, some provincial and university laboratories within the CAHSN are enhancing their laboratory infrastructure over and above what AI funding supported. As another example, the CFIA participated in and completed a collaborative project with AAFC, Alberta Agriculture and Rural Development, and Iowa State University to study the disposal of large ruminant carcasses via composting during a disease outbreak. With such leveraging, the value realized by the CFIA and the Government of Canada has exceeded the dollar amount spent on the AI/PI Initiative.

Relevance

Finally, the evaluation compared the AI/PI Initiative to other similar initiatives, considered its relevance in comparison to current federal objectives and priorities, and assessed the ongoing need for the Initiative.

Relationship to similar initiatives

The evaluation evidence suggests that the AI/PI Initiative does not duplicate but rather is complementary to other federal, provincial, local, and industry initiatives. Other federal departments and agencies are also involved in the Government of Canada’s response to zoonotic disease, most notably PHAC, HC, and PSC. However, whereas the CFIA’s mandate is focused on animal health, other federal departments are focused on human health. Furthermore, the CFIA appears to be the only federal department with an initiative focused specifically on avian influenza; other departments have undertaken activity with respect to pandemic influenza or emergency response in general (see Appendix D for a summary of activities undertaken by other departments). Despite the different mandates of the CFIA’s federal partners, key informants believe the AI/PI Initiative has improved coordination among them with respect to Canada’s response to zoonotic diseases, which pose a risk to both animal and human health.

The provinces, for their part, do not appear to have comparable initiatives focused on AI. Rather, like federal departments, their focus is pandemic preparedness in general (results of an Internet search indicate that many provinces evidently have developed pandemic preparedness plans). With respect to animal health emergencies, the respective roles and responsibilities of the CFIA, other federal departments and agencies, and provincial authorities are spelled out in the FADES agreements, which have been signed with all the provinces, except for Newfoundland and Labrador.

Finally, the evaluation found no evidence of comparable initiatives being undertaken by industry. Industry participates on a voluntary basis in the CFIA’s biosecurity measures. Key informants believe that industry has become more aware of its roles and responsibilities in the event of an AI outbreak, and is more prepared to deal with an AI event as a result of the AI/PI Initiative.

Relevance to current priorities and ongoing need

The Initiative’s broad objectives of achieving a better level of emergency preparedness and improving public safety remain consistent with current Agency and federal priorities. While recent federal budgets and throne speeches have not emphasized emergency or pandemic preparedness, the federal government is still active in this area, although its focus has shifted from avian influenza to H1N1 (swine flu) in response to the current threat.

Although AI has faded from the public consciousness, the threat of AI has not disappeared. This  suggests an ongoing need for preparedness initiatives at the national level. World Health Organization data show that outbreaks of avian influenza and associated cases of human illness continue to occur around the world, most recently in Egypt in July 2009. In light of this reality, key informants agreed that there will always be a need to maintain the level of preparedness that has been achieved and to regularly reassess the threat and the level of preparedness. Many key informants also noted that the Initiative has served as a model for response to other zoonotic diseases. Some were of the view that the CFIA should take an “all hazards” approach to emergency preparedness, rather than focus on a specific threat.

5. Conclusions and Recommendations

Overall, the AI/PI Initiative was implemented mostly as planned and has greatly enhanced the CFIA’s capacity to respond to avian influenza emergencies and animal health emergencies in general. However, there are opportunities to improve the overall management of the Initiative through better coordination of Initiative activities within the CFIA and through improvements to performance monitoring and financial reporting. The Agency’s ability to demonstrate Initiative success will be limited without more comprehensive collection of both qualitative and quantitative performance data.

Implementation

The evaluation found that the AI/PI Initiative has been implemented mostly as planned, though some activities, such as risk assessment, biosecurity, risk communications, the national veterinary reserve, and emergency response plans spent less than half of what was planned, while others spent more than twice the planned amount. These significant variances represent an opportunity to improve planning and budgeting.

Governance and process

The evaluation found that although roles, responsibilities, and accountabilities within the CFIA are generally clear, there is an opportunity for improved management and coordination of the Initiative. Coordination was strong at the outset of the Initiative, when this was the responsibility of the AIWG. However, once implementation was well under way, other threats to animal health and food safety emerged, and coordination become the responsibility of the AHPOC. This was followed by a diminished level of coordination and intensity of focus accompanied by lower awareness of the Initiative among CFIA staff and managers.

On a positive note, the Agency has implemented some effective management practices. For example, the Agency systematically reviews its response to AI outbreaks and adjusts its activities according to lessons learned. Over time, this process of ongoing review and adjustment has produced numerous improvements to the Agency’s overall response to avian influenza.

With respect to the CFIA’s external partners–including other federal departments, the provinces/territories, regional laboratories, and industry–roles, responsibilities, and accountabilities are clear. These relationships appear to be effective and generally have been strengthened by the various frameworks and agreements that have been put in place.

Success

Overall, at the end of 2009, the CFIA is much better prepared than it was in 2006 to deal with an AI emergency. The Agency’s preparedness is not just theoretical. The proof is in the response to actual AI outbreaks, which has resulted in rapid containment of disease. However, it will be difficult to discern intermediate and long-term success of the Initiative if a more comprehensive collection of performance data is not undertaken.

Some of the key successes of the Initiative, along with some of the outstanding issues, are highlighted below.

• The infrastructure created as a result of AI funding has enhanced the CFIA’s animal health emergency response capacity. It has also enhanced the ability of the CFIA’s partners to respond to all animal health emergencies. Aspects of the CFIA’s approach to AI emergency response have been applied to H1N1 and, in future, may be used for other animal health diseases such as foot-and-mouth disease.
• The Initiative has produced a more comprehensive and harmonized response to avian influenza. Within the CFIA, numerous protocols and detailed procedures for response have better prepared the Agency to respond to AI outbreaks. Externally, the FADES agreements with the provinces, along with a variety of other agreements and frameworks, have strengthened relationships with federal, provincial, and industry stakeholders.
• The surveillance/detection network has been effective. Provincial laboratories, through passive surveillance, detected the index case in both the 2004 BC and 2007 Saskatchewan outbreaks.
• There appears to be an adequately trained, skilled, and equipped workforce to face AI outbreaks of the size experienced to date, and the Agency has successfully responded to several AI outbreaks in the past.
• The CFIA has adequate surge capacity in terms of external reserve staff, infrastructure, equipment, and laboratory capacity to handle AI outbreaks of the size experienced to date.

Cost effectiveness

Applicability of the Initiative’s infrastructure to other animal health emergencies suggests significant leveraging. In addition, the Initiative has leveraged public and private resources–among them veterinary schools, private veterinarians, provincial/university laboratories, academic institutions, and in-kind contributions from industry–for biosecurity measures. The value realized by the CFIA and the Government of Canada exceeds the dollar amount spent on the AI/PI Initiative.

Relevance

While there are other initiatives in Canada that address zoonotic disease and pandemic influenza, the AI/PI Initiative is unique in focusing on animal health. Furthermore, the broad objectives of the Initiative—achieving a better level of emergency preparedness and improving public safety—are consistent with current Agency and federal priorities, although the focus has shifted away from avian influenza to other, more immediate threats. Although AI has faded from the public consciousness, the threat remains. This suggests an ongoing need for preparedness initiatives at the national level. In light of this reality, there will always be a need to maintain the level of preparedness that has been achieved, and to regularly reassess the threat and the level of preparedness.

Recommendations

Recommendation 1. To address gaps in central oversight, the Operations Branch and the Policy and Programs Branch should take steps to improve the internal coordination of the AI/PI Initiative by, for example, ensuring data collection and expenditures are overseen by no more than one Agency office each and regular Initiative status reports are distributed to those responsible for its outputs.

Recommendation 2. The CFIA should review its performance measurement system for the AI/PI Initiative and present the results of the review to the Evaluation Committee at its September 2010 meeting. The following items should be addressed in the new approach:

1. review the Initiative’s performance measurement strategy to identify the most critical performance indicators, both existing and new, that can most realistically be collected, immediately and retroactively, eliminating unessential indicators and ensuring that tracking begins;
2. implement a system for reporting performance information, which could include development of standardized templates or reporting tools;
3. assign responsibility for performance reporting to specific positions within the Agency; and
4. ensure financial information is linked to Initiative outputs and outcomes.

Appendix A

Context for the AI/PI Initiative

First identified in Italy in 1878, avian influenza (AI), or “bird flu,” refers to a large group of contagious influenza viruses (type “A”) that primarily affect wild and domestic birds.¹⁷ Although most strains do not pose a serious threat of disease, certain strains have been known to cause widespread disease in birds, often leading to illness, death, and the possibility of infecting other species.

Influenza A viruses are classified into 16 H subtypes and 9 N subtypes, which are grouped into two categories—low pathogenic (LPAI) and high pathogenic (HPAI) forms—based on the severity of the illness associated with the particular strain. The highly pathogenic form spreads more rapidly, inducing sudden and severe illness in flocks of poultry, and has a mortality rate that can reach 90–100%, often within 48 hours.¹⁸

To date, virtually all highly pathogenic outbreaks have been attributed to certain strains of the H5 and H7 subtypes of the virus. In several cases, transmission of the AI virus to humans and other species has occurred as a result of close contact with infected birds or heavily contaminated environments.

Although not all viruses of these subtypes are known to cause the highly pathogenic form of the disease, some strains have demonstrated the tendency to mutate from low pathogenic strains to highly pathogenic forms while circulating within poultry populations. For example, the North American strains of H5N1 have thus far posed little harm to birds; however, the Asian strain of H5N1, which has been found in Asia, Africa, and Europe, has caused serious illness and death in domestic and wild birds and has also demonstrated some ability to infect mammals, including humans.¹⁹

Following a 1997 outbreak in Hong Kong, the incidence of the highly pathogenic AI has risen sharply. Although there have been several relatively small non-H5N1 outbreaks involving transmission to humans, as well as outbreaks among poultry with no transmission to humans, the most widespread and severe outbreaks affecting both birds and humans have involved H5N1.²⁰ In 2003, AI resurfaced in Hong Kong as two more human cases of H5N1 infection were reported in February. From mid-December 2003 through early February 2004, poultry outbreaks caused by the H5N1 virus were reported in eight Asian nations and in many other countries. Outbreaks are continuing with varying degrees of severity—despite the death or destruction of an estimated 150 million birds. The virus is now considered endemic in many parts of Indonesia and Vietnam and in some parts of Cambodia, China, Thailand, and possibly the Lao People’s Democratic Republic.²¹

There have been four outbreaks of avian influenza in Canada to date: three in British Columbia (2004, 2005 and 2009) and one in Saskatchewan (2007).

Impact of H5N1 and pandemic potential

The recent and ongoing outbreaks of highly pathogenic AI, which began in 1997 and ballooned in Southeast Asia in mid-2003, are the largest and most severe on record. According to the World Health Organization (WHO), the emergence of H5N1 is unprecedented; never before in the history of this disease have so many countries been simultaneously affected, resulting in the loss of so many birds.²² At the outset of the epidemic (between December 2003 and March 2004), more than 100 million birds either died of the disease or were culled in Asia. This is greater than the total number of poultry affected in the world’s previous five largest outbreaks combined.²³

Moreover, highly pathogenic AI of the H5N1 subtype has caused the most numerous and severe human infections of any of the AI viruses. Since late 2003, over 400 instances of H5N1 human infections have been confirmed in 15 countries in Asia, Africa, and Europe, with about 60 percent of those being fatal. According to the United Nations, estimates show that H5N1, a highly pathogenic AI, has already resulted in over US$20 billion in economic losses, and could cost the global economy around US$2 trillion if it becomes a pandemic.²⁴

The spread of H5N1 virus from person to person has been very rare, limited, and unsustained to this point. However, it constitutes a very serious public health concern as it is possible that the virus may adapt and mutate into a more transmissible strain. Many public health agencies and regulatory bodies have claimed that if H5N1 viruses were to have efficient and sustained transmission among humans, an influenza pandemic would likely result, with potentially high rates of illness and death worldwide.

International responses to AI

Several governments, international organizations, and health authorities from around the world have taken steps to strengthen AI/PI preparedness to reduce the potential for and minimize the costs of future outbreaks and/or pandemics. At the supranational level, the WHO has launched several initiatives designed to strengthen the monitoring of AI/PI risks and promote international collaboration in prevention. In 2002, the WHO adopted the Global Agenda for Influenza Surveillance and Control, with the chief objectives of strengthening surveillance, improving knowledge of the disease burden, increasing vaccine use, and accelerating pandemic preparedness.²⁵

Working in close collaboration with the WHO, the World Organisation for Animal Health (OIE) provides guidelines, advice, and standards to prevent, diagnose, and respond to outbreaks of AI. Similarly, the United Nations Food and Agriculture Organization (FAO) works alongside international partners to direct technical and resource assistance to help national governments align their prevention, control, and response efforts with OIE guidelines.²⁶

In August 2005, the WHO released a document that outlined recommended strategic actions for responding to the AI pandemic threat. This document provided a situation assessment that noted the following: the risk of a pandemic is great; the risk will persist; evolution of the threat cannot be predicted; the early warning system is weak; preventive intervention is possible, but untested; and reduction of morbidity and mortality during a pandemic will be impeded by inadequate medical supplies.²⁷

As a result of this assessment, the WHO recommended all countries ensure that strategic actions aim to strengthen national preparedness, reduce opportunities for a pandemic virus to emerge, improve the early warning system, delay initial international spread, and accelerate vaccine development.²⁸ This document preceded the formation of the International Partnership on Avian and Pandemic Influenza (IPAPI) at the United Nations General Assembly in September 2005, and marked one of the first collaborative efforts of member countries to work together on avian and pandemic influenza. Around the same time, the FAO and OIE jointly published the FAO-OIE Global Strategy for the Progressive Control of Highly Pathogenic Avian Influenza (HPAI), which was subsequently revised in 2008.

Aside from the strategic plans and recommendations, the WHO operates the Global Outbreak Alert and Response Network (GOARN), which serves as a technical collaboration focusing on rapid identification, confirmation, and response to outbreaks of international importance.²⁹ The organization has also adopted revised International Health Regulations, which came into force in 2007 and included key provisions for detecting, reporting, and responding to AI/PI. It is currently working on developing a new electronic system to track influenza A (H5) viruses.³⁰

In March 2006, Canada, Mexico, and the US adopted a cross-border regional plan under their newly established Security and Prosperity Partnership of North America (SPP) to address the threat of AI/PI. The plan complements national emergency management plans and builds on the core principles of IPAPI, international standards and guidelines of the OIE and the WHO (including the revised International Health Regulations), as well as rules and provisions of the World Trade Organization (WTO) and the North American Free Trade Agreement (NAFTA). The plan set out a framework for cooperation on AI/PI, which included a series of principles to guide collaboration.

Appendix B

Evaluation Matrix: Formative Evaluation of the Avian and Pandemic Influenza Preparedness Initiative

Relevance

Evaluation issues and questions	Data indicators	Data sources	Formative	Summative
1. What circumstances led to the development of the AI/PI Initiative? Is there still a need for the Initiative?	Documented rationale for Initiative Descriptions provided by key informants International norms and laws/regulations with respect to the control of avian flu Scientific information on the potential for a pandemic Recent incidents of avian flu outbreaks Review of scientific literature on best practices to control avian flu	Policy and program development documents Interviews (esp. CFIA staff and management) Tracking of avian flu outbreaks Review of scientific literature Review of practices in other countries (OECD, WHO)	X	X
2. Does the Initiative duplicate or complement any other federal, provincial, local, or industry initiatives?	Review of related initiatives in the CFIA and in related departments (AAFC, PHAC) as well as provincial agriculture and health ministries Descriptions of key intergovernmental relations provided by Initiative and Agency managers	Interviews (CFIA staff and management, provinces, industry) Document review (intradepartmental coordination and federal–provincial practices)	X	X
3. Are the objectives of the AI/PI Initiative still consistent with current Agency and federal priorities? How should the objectives of the Initiative be modified, if at all?	Perception of key stakeholders on the alignment of objectives Documentary evidence of the alignment of Initiative objectives to the CFIA and federal objectives of food safety and animal/human health Comparison of Initiative objectives to international norms and laws/regulations	Interviews (CFIA staff and management) Document review (policy and planning documents, MCs and TB submissions, descriptions, evaluations of programs in OECD countries, WHO reviews)	X	X

 

Design and delivery

Evaluation issues and questions	Data indicators	Data sources	Formative	Summative
4. Have the activities of the Initiative been implemented as planned? What challenges, if any, were encountered in implementation? Have these challenges been effectively addressed?	Performance reports, memos, and accounts of missed deadlines and re-profiling of planned activity Evidence of best practices or mitigation strategies to address challenges Perception of CFIA staff and management on Initiative progress	Document review (internal reports on progress) Interviews (CFIA staff and management)	X
5. Are the activities of the Initiative adequate to achieve its objectives? What, if any, adjustments could be made without affecting the likelihood of achieving the objectives?	Organization chart outlining the link of programs to outputs Operation and strategic plans describing the program logic Perception of CFIA management and government stakeholders regarding coherence of Initiative logic	Interviews (CFIA staff and management, government stakeholders) Document review (organization charts, human resourcing plans) File and database review (progress reports) Review of practices in other countries (OECD, WHO)	X
6. Are the financial and other resources committed to Initiative programs and activities sufficient to produce the required outputs?	Changes in Initiative and program budgets Perception of CFIA staff and management and government stakeholders of budget/resource adequacy Outputs produced	Document review (financial analyses) Interviews (CFIA staff and management, government stakeholders)	X	X

 

Governance

Evaluation issues and questions	Data indicators	Data sources	Formative	Summative
7. Is the governance structure of the Initiative appropriate? Within the Initiative and within the CFIA, are roles, responsibilities, and accountabilities clear? Are roles, responsibilities, and accountabilities clear with respect to partners such as Canada Border Services Agency, provinces, and regional laboratories?	Perception of CFIA and provincial managers and stakeholders Document review (operations guidelines/ practices, memoranda of understanding)	Document review of operations procedures Interviews (CFIA staff and management, external stakeholders) Review of practices in other countries (OECD, WHO)	X	X
8. Within the CFIA, are Initiative activities effectively coordinated among the Agency's branches, area offices, and regional offices?	Documentation of internal coordination Perceptions of CFIA staff and management on effectiveness of internal coordination	Document review Interviews (CFIA staff and management)	X	X
9. Are the partnerships with provinces and regional laboratories effective in delivering the programs under the AI/PI Initiative?	Documentation and literature detailing results of collaborative efforts, joint planning, post-event analyses, and operations practices Perception of CFIA and provincial managers and stakeholders	Document review (records of meetings and intergovernmental communications and reports, e.g., partnership agreements) Interviews (CFIA staff and management, external stakeholders)	X	X
10. What are the risks associated with the AI/PI Initiative? Does CFIA management regularly review and update risks?	Quality and accessibility of risk management strategies in AI/PI Initiative information, reporting materials, and documentation Indicators of management of risks Perception of CFIA staff and management and government stakeholders on Initiative risk management	Document review (internal memos and reports) Interviews (CFIA staff and management, government stakeholders) Review of practices in other countries (OECD, WHO)	X
11. Are emergency exercises conducted as planned? Are the recommendations arising from these exercises implemented? Why or why not?	Documentation of emergency exercises Evidence that recommendations were implemented Perception of CFIA staff and managers	Document review Interviews (CFIA staff and management)	X

 

Performance measurement and reporting

Evaluation issues and questions	Data indicators	Data sources	Formative	Summative
12. Has a performance measurement system been implemented? Does the performance measurement system collect adequate output and outcome information to report on the implementation and impact of the Initiative?	Documentation on performance measurement Evidence of regular performance reporting Perceptions of senior CFIA and other departmental management on adequacy of reporting	Documents (progress reports) Interviews (CFIA management)	X	X
13. What mechanisms are in place to report on performance to senior management? Does senior management receive information on performance in an effective, timely, and efficient way?	Evidence of regular performance reporting Perceptions of senior CFIA and other departmental management on adequacy of reporting	Documents (progress reports) Interviews (CFIA/CBSA management)	X	X

 

Success

Evaluation issues and questions	Data indicators	Data sources	Formative	Summative
14. Is there an adequately trained, skilled, and equipped workforce to face AI? Is staff training relevant and updated on a timely basis?	Evidence of increased training/skill levels of workforce Evidence of ongoing training Evidence that workforce is better equipped Perceptions of CFIA management, provinces, and other stakeholders Response to incidents	Interviews (CFIA staff and management, province, regional laboratories, other stakeholders) File and database review on outputs and outcomes realized, e.g., training strategy development	X	X
15. Is there adequate surge capacity in terms of external reserve staff, infrastructure, equipment, and laboratory capacity? Is the infrastructure developed by the Initiative maintained and kept up to date?	Evidence of improvements in infrastructure, equipment, and laboratory capacity Evidence of ongoing maintenance of infrastructure Perceptions of CFIA management, provinces, and other stakeholders Response to incidents	Interviews (CFIA staff and management, province, regional laboratories, other stakeholders) File and database review on outputs and outcomes realized, e.g., updated veterinary reserve and retiree lists and updated infrastructure assessments	X	X
16. Is the surveillance/ detection network effective? Are risks associated with importation mitigated?	Evidence of effectiveness of surveillance/ detection network Perceptions of CFIA management, provinces, and other stakeholders Response to incidents	Interviews (CFIA staff and management, province, regional laboratories, other stakeholders) File and database review on outputs and outcomes realized, e.g., emergency response plans and contracting strategies	X	X
17. Are there adequate diagnostic techniques and avian vaccines to face AI?	Documentation of diagnostic techniques and avian vaccines Perceptions of CFIA management, provinces, and other stakeholders Response to incidents	Interviews (CFIA staff and management, province, regional laboratories, other stakeholders) File and database review on outputs and outcomes realized, e.g., methods for assessing vaccine strain suitability	X	X
18. Are communications mechanisms in place to monitor public opinion and provide timely information to the public?	Evidence of public opinion monitoring mechanisms and communications mechanisms Perceptions of CFIA management, provinces, and other stakeholders Response to incidents	Interviews (CFIA staff and management, province, regional laboratories, other stakeholders) File and database review on outputs and outcomes realized Review of CFIA media output, including Web site review	X	X
19. If these outcomes have not been achieved, is material progress likely within the Initiative's time frame?	Progress on reaching objectives Perceptions of CFIA management, provinces, and other stakeholders Response to incidents	Interviews (CFIA staff and management, province, regional laboratories, other stakeholders) File and database review on outputs and outcomes realized	X	X
20. Have any unintended impacts, either positive or negative, resulted from AI/PI activities?	Indications of unintended impacts within AI/PI Initiative reporting and documentation Anecdotal data from all lines of evidence	Document review (internal memos, meeting minutes, etc.) Interviews (CFIA staff and management, provinces, regional laboratories, and other stakeholders) Review of practices in other countries (OECD, WHO)	X	X

 

Cost-effectiveness

Evaluation issues and questions	Data indicators	Data sources	Formative	Summative
21. Is the AI/PI Initiative optimizing outputs, given the available resources? To what extent is the infrastructure used in AI/PI activities used for other purposes within the CFIA and the federal government? To what extent has the Initiative leveraged other public and private resources?	Expert opinion from CFIA management and from funding departments Analysis of financial reporting materials (overview of current and historical funding distribution and allocations)	Interviews (CFIA staff and management) File and database review (financial)	X	X
22. Are the available resources appropriately allocated?	Perception of CFIA management Indications of funding reallocations within AI/PI Initiative information, reporting materials, and documentation	Interviews (CFIA management) Document review File and database review (progress reports)	X	X
23. Are there alternative approaches that could increase the efficiency or effectiveness of the Initiative?	Perception of CFIA staff and management and external stakeholders	Interviews (all stakeholders) Review of practices in other countries (OECD, WHO)	X	X

Appendix C

Formative Evaluation of the Avian Influenza and Pandemic Influenza Preparedness Initiative Interview Guide

This interview guide has been designed for a variety of interviewees. If you are not in a position to address some of the questions, please let me know and we will move on.

Background

1. In what capacity are you currently involved with the AI/PI Initiative? Please describe your current position, roles and responsibilities, length of time involved, etc.

Design and delivery

2. Have the activities of the Initiative been implemented as planned? What challenges, if any, were encountered in implementation? Have these challenges been effectively addressed?

3. In your view, are the activities adequate to achieve the objectives of the Initiative? Please explain. What adjustments, if any, could be made to the activities without affecting the likelihood of achieving the objectives?

4. Are the financial and other resources committed to the Initiative sufficient to produce the required outputs?

5. Are you aware of whether any recommendations, lessons learned, or best practices emerging from past evaluations or reports have been implemented? If so, which ones (e.g., lessons learned from past epidemics or outbreaks)?

6. Do you have any suggestions for improving the design or delivery of the Initiative?

Governance and process

7. In your opinion, is the governance structure for the Initiative appropriate? Are roles, responsibilities, and accountabilities clear:

• within the CFIA?
• with respect to other federal partners?
• with respect to the provinces?
• with respect to regional laboratories?

What, if anything, is unclear?

8. Within the CFIA, are Initiative activities effectively coordinated among the Agency’s branches, area offices, and regional offices? Why or why not?

9. Are the partnerships with provinces and regional laboratories effective at delivering programs under the Initiative? Why or why not?

10. What are the risks associated with the AI/PI Initiative? Are risks reassessed on a regular basis?

11. To what extent are emergency exercises conducted as planned? Are the recommendations arising from these exercises implemented? Why or why not?

12. Do you have any suggestions for improving the governance structure of the AI/PI Initiative?

Performance measurement and reporting

13. Has a performance measurement system been implemented? If so, does the system collect adequate information to report on the implementation and impact of the Initiative? What changes, if any, are required?

14. What mechanisms are in place within the CFIA and other federal partners to report on performance to senior management? [For senior managers] Are these reporting mechanisms effective, timely, and efficient? Are reports useful? What changes, if any, are required?

Success

This section asks whether the expected immediate outcomes of the Initiative have been achieved. Please comment only on those outcomes that are applicable to you and about which you are knowledgeable, providing evidence or concrete examples of impacts where possible. If you are aware of any supporting documents or data, please share them with us if possible.

15. Is there an adequately trained, skilled, and equipped workforce to face AI? Is staff training relevant and updated on a timely basis?

16. Is there adequate surge capacity in terms of external reserve staff, infrastructure, equipment, and laboratories? Is the infrastructure developed by the Initiative maintained and kept up to date?

17. Is the surveillance/detection network effective? Are risks associated with importation mitigated?

18. Are there adequate diagnostic techniques and avian vaccines to face AI?

19. Are communication mechanisms in place to monitor public opinion and provide timely information to the public?

20. If these outcomes have not been achieved, is progress likely within the Initiative’s time frame?

21. Have any unintended impacts, either positive or negative, resulted from the Initiative?

Cost-effectiveness/alternatives

22. How efficient is your branch/organization in its use of AI/PI resources? To what extent is infrastructure used for other purposes within the CFIA and the federal government? To what extent has the Initiative leveraged other public and private resources?

23. Are the available resources appropriately allocated? To what extent have changes in the level of funding affected implementation and progress toward expected outcomes?

24. In your opinion, are there other approaches that could increase the efficiency or effectiveness of the Initiative?

Relevance

25. Are the objectives still consistent with departmental and government-wide priorities? How should the objectives be modified, if at all?

26. Is there any duplication of other federal, provincial, local, or industry initiatives? How does the Initiative relate to those undertaken by Health Canada, the Public Health Agency of Canada, and the Canadian Institutes of Health Research?

27. In your view, is there still a need for the AI/PI Initiative? Why or why not?

Conclusion

28. Do you have any additional comments not covered in previous questions?

29. Can you recommend any documents and/or reports that provide information regarding the implementation of the AI/PI Initiative?

Thank you for your participation.

Appendix D

Supplementary Data Tables

Table 1 Status of implementation of AI/PI components and activities

Table 2 Recommendations from past outbreaks

Table 3 Areas of support and stakeholders in Alberta FADES agreement

Table 4 Pandemic influenza and emergency response activity of other federal departments

Table 5 AI/PI activities and responsibilities (as of June 2007)

Table 6 Major documents reviewed by the evaluation

Table 1 Status of implementation of components and activities comprising the CFIA's AI/PI Initiative

 Source: PHAC Departmental Performance Reports (Horizontal Initiatives) for 2006–07, 2007–08, and 2008–09; CFIA reports and documents, especially the Final Report of the Avian Influenza Working Group (June 2007) and the Avian Influenza Related Activities Tracking (October 2008); and key informant interviews.

Component	Status of implementation
Vaccines and antivirals	Vaccines: The CFIA conducted research on vaccination as a strategy against the spread of AI and created a policy on the effective use of vaccination. The policy allows for the vaccination of poultry for AI if an infection is: 1) detected after it has been widely disseminated in Canada; or 2) highly virulent and spreading faster than the CFIA can remove the infected flocks. The policy also describes the type of vaccine to be used in specific situations and the type of surveillance activities to be implemented after vaccination. A Canadian vaccine bank of 10 million doses of both H5 and H7 subtypes was initially established, but, in 2009, was donated to another country experiencing an AI outbreak because it was nearing the end of its sustainable shelf life. For various reasons, the CFIA has moved away from vaccination as a strategy to address AI and will no longer seek to stockpile a significant amount of vaccine. As an alternative, the CFIA has ensured a supply through a vaccine sharing agreement between Canada, the US, and Mexico. Antivirals: An antiviral stockpile consisting of 250,000 doses has been established as planned, and protocols to access and implement use of antivirals have been developed.
Surge capacity	Internal staffing reserve: In 2008–09, the CFIA continued to develop a human resources skills inventory database to track training and experience of CFIA staff. This serves as a centralized information source to determine appropriately trained individuals to respond to AI events. In addition, the Agency has identified retired staff who could return to active duty during an emergency, as required. External staffing reserve: To increase specialized capacity in the event of a large-scale emergency, the CFIA and the Canadian Veterinary Medical Association had recruited and trained a total of 204 reservists by October 31, 2009. Training sessions were held in fall 2007, winter 2007, fall 2008, winter 2008, and fall 2009; the next training session is expected to occur in fall 2010. The CFIA is currently seeking management approval to move from a developmental strategy to a maintenance one, involving refresher training and provision of online training opportunities. Equipment: Spending on specialized equipment was 13% of what was planned. A national stockpile inventory has been established to give ready access to necessary specialized equipment and supplies within hours of an emergency. Four Modified Atmospheric Chambers for the humane destruction of infected poultry are being maintained and four Mobile Telecommunications Vehicles have been purchased; however, the latter are not compatible with the way the CFIA actually responds to emergencies and three are currently for sale. Work is ongoing to expand the capability and capacity of the information management system—the Canadian Emergency Management Response System (CEMRS)—to deal with high volumes of samples collected during outbreaks. Laboratories: AI funding has enhanced the ability of the 10 provincial and university laboratories that are part of the Canadian Animal Health Surveillance Network (CAHSN) to support diagnostic surge capacity (the laboratories represent all 10 provinces). Quality assurance management systems, ISO accreditation training, and standard operating procedures have been implemented. To date, 10 laboratories have passed proficiency panels and are certified to do AIV RT-PCR testing, while six are certified to do AIV ELISA testing. In addition, there are currently 52 laboratory analysts in CAHSN laboratories across Canada trained and certified to perform AIV RT-PCR and AIV ELISA testing. Finally, three of the provincial laboratories in CAHSN have completed procedures and met the necessary biocontainment standard. The remaining laboratories have put procedures in place until they are certified so that they can carry out the testing.
Prevention and early warning	Wild bird surveillance: The CFIA has supported annual national wild bird surveillance of both dead and live birds since 2005. The survey is a joint initiative of CFIA, provincial/territorial governments, the Canadian Cooperative Wildlife Health Centre (CCWHC), the Avian Influenza Virus Laboratory Network, Environment Canada, and the Government of Iceland. Through the survey, Canada participates in Canada-USA-Mexico trilateral collaborations for continental surveillance for AI, and, collaboratively with the US Department of Agriculture, participates in analysis of the risks posed to commercial poultry by AI viruses in wild birds. Commercial bird surveillance: The Canadian Notifiable Avian Influenza Surveillance System (CanNAISS) has been developed and partially implemented in partnership with industry as a surveillance system for domestic birds. CanNAISS will have six components when fully implemented. In 2008–09, three components (passive surveillance, pre-slaughter serological surveillance, and voluntary enhanced notifiable avian influenza [NAI] surveillance) were active, and the remaining three components (parent flock serological surveillance, ad hoc serological surveys, and targeted surveillance – NAI) were in development. Biosecurity: In October 2009, the CFIA launched, on its Web site, the National Avian On-Farm Biosecurity Standard as the basis of a comprehensive voluntary program designed to provide applicable guidance for owners or managers in all poultry sectors in Canada. An accompanying General Producer Guide for the Standard is in development and is expected to be finalized in December 2009. The CFIA has also funded projects through the Avian Biosecurity Technology Development Fund (ABTDF), including seven projects in 2006–07 and five in 2007–08. The ABTDF was eliminated as a result of strategic review. With respect to importation, new import measures for live birds to prevent the introduction of avian influenza in domestic birds have been implemented for all countries other than the US (the import protocol for the US was developed in 2005–06). The new requirements are consistent with OIE guidelines and require pre-entry surveillance or testing and post-import testing during the import quarantine period. Intelligence gathering: The evaluation found no information on this activity.
Emergency preparedness	Emergency response teams: An AI National Emergency Response Team has been identified, as well as response teams in each of the CFIA’s four operational areas. The teams, which include veterinarians, executive management, and field staff, oversee the CFIA’s response and coordinate actions with federal, provincial, and municipal partners. Key personnel across the country have been familiarized with Incident Command Systems (ICS). Development of detailed procedures for response: Detailed procedures for response have been developed, including the CFIA National Emergency Response Plan; the Notifiable Avian Influenza Hazard Specific Plan; the Animal Health Functional Plan; the 24/48/72 Hour AI Response Guide; the Corporate Logistics Plan; the Human Resource Response Plan; Standard Operating Procedures for biocontainment, humane destruction, disposal (composting), alternate disposal (incineration, deep burial, rendering), and cleaning and decontamination; as well as Occupational Health and Safety Processes for Responders during an AI outbreak. AI scenarios and exercises: A variety of scenarios and exercises have been carried out involving the CFIA, other federal and international partners, provinces, and industry in various combinations. In total, 47 different exercises were identified on the CFIA’s “Readiness Exercise Plan for Avian Influenza.” Of the 47, the CFIA identified 21 as regional-level exercises, eight as provincial-level exercises, eight as national-level exercises, three as international-level exercises, and five as private sector-level exercises. Overall, through these exercises, all 10 provinces have had the opportunity to exercise emergency response. Although the CFIA was the lead organization for many exercises, others were led by provincial emergency management organizations or provincial ministries of agriculture, industry organizations, and others. Exercises involved simulations of AI outbreaks; updates/definitions of emergency structures and roles and responsibilities; ICS training; humane euthanasia; composting; and other relevant topics. More than half of the exercises (28) were conducted in 2006, 15 in 2007 and four in 2008. Activities related to partnerships and surge capacity are described under other components.
Critical science and regulation	Critical science: The CFIA has conducted research into developing rapid microarray-based tests; evaluating commercially available AI antigen detection kits; harmonizing AI testing methods with the US Department of Agriculture and Mexico; developing reagents for rapid tests; and evaluating potential new vaccine strains, humane euthanasia, and mobile incinerators. However, it is unclear which of these projects have been completed and which are ongoing. Planned research projects on aerosol and plume transmission were not carried out due to budget cuts. In addition, risk assessments were initiated or completed on AI risk issues, including those related to importation of live birds from AI-infected countries and animal products that might contain the AI virus, and AI disease spread modeling within the North American Animal Disease Spread Model was used to assist Agriculture and Agri-Food Canada to develop self-insurance for poultry producers in Ontario. The CFIA also participated in and completed a collaborative project with Agriculture and Agri-Food Canada, Alberta Agriculture and Rural Development, and Iowa State University to study the disposal of large ruminant carcasses via composting in a disease outbreak. Regulation: With provincial/territorial government, animal and public health experts, and industry, the CFIA explored regulatory options that could result in more effective control of disease spread, assessed compensation options for industry, and initiated the development of appropriate cost valuation models. These activities will continue in 2009–10. Spending on activities to support a strengthened economic and regulatory framework was 7% of what was planned.
Risk communications	Information for general public/public opinion tracking: The CFIA Web site provides extensive information to the general public, including basic information about avian influenza, implications for human health, reports on previous disease incidents, and information on AI-related restrictions on pet bird imports. The Web site also provides regular news releases and information bulletins on various topics, including AI. As public concern over AI has dissipated, the CFIA has stopped regular tracking of it, although occasionally there are questions pertaining to AI on the CFIA’s other public opinion surveys. Currently, proactive communications work is being done on traveller biosecurity and biosecurity for poultry operations, both of which are described below. Information for poultry operations: The CFIA Web site contains extensive biosecurity information for both commercial poultry operations and small flock owners, including the National Avian On-Farm Biosecurity Standard. The Web site also includes an animated video for small flock owners on prevention and detection of disease, as well as cleaning and disinfection tips. The CFIA has held information sessions for small flock owners in various provinces and has conducted opinion research on AI biosecurity awareness among targeted groups. Information for travellers: In collaboration with the Canada Border Services Agency and other partners, the CFIA delivered a traveller’s biosecurity advertising and outreach campaign (Be Aware and Declare) using airport posters, the CFIA Web site, and brochures in more than 20 languages.
Federal, provincial/ territorial, and international cooperation	Federal cooperation: The CFIA has developed several protocols and MOUs concerning emergency response with other federal partners, including an Umbrella MOU involving the CFIA, HC, and PHAC, including a roles and responsibilities framework; a Zoonotic Illness Outbreak Response Protocol involving the CFIA, HC, and PHAC, which addresses guiding principles and operating procedures for a coordinated response with these agencies to the threat of zoonotic disease outbreaks with national or international significance; a Protocol on Human Health Issues Related to Domestic AI Outbreak with PHAC; a Workplace Health and Safety MOU, Zoonotic Outbreak with HC; and the traveller’s biosecurity advertising and outreach campaign in collaboration with CBSA. Provincial cooperation: Foreign Animal Disease Emergency Support (FADES) agreements have been signed with all of the provinces, with the exception of Newfoundland and Labrador; these agreements spell out the respective roles and responsibilities of federal, provincial, and local agencies in the event of an animal health outbreak. The CFIA has also established and maintains AI communications and liaison networks in each area (Atlantic, Quebec, Ontario, Western). According to the June 2007 Final Report of the AIWG, area communications protocols and contact lists are kept current and made accessible to Area/Regional Management Teams; these networks enable each area to partner with stakeholders to support the CFIA’s efforts. Finally, the CFIA has partnered with provincial/university laboratories involved in the CAHSN network to enhance diagnostic surge capacity for AI. International cooperation: The CFIA participated in the development of the North American Plan for Avian and Pandemic Influenza (NPAPI). In addition, CFIA representatives met with counterparts from the US and Mexico to work on communications deliverables identified in the North American Plan, with the objective of aligning information sharing among the three countries and with the public during an AI outbreak. In April 2008, Canada, Mexico, and the US signed an agreement for sharing AI vaccines in the event of a highly pathogenic AI outbreak. CFIA staff was deployed to the OIE Central Bureau and Buenos Aires Regional Office to support, develop, and promote the use of science-based standards, risk assessment, capacity building, and training. Finally, the CFIA supported capacity development and infrastructure in developing countries through the Canadian chapter of Veterinarians Without Borders and the Canadian Veterinary Reserve.

Table 2: Recommendations from past outbreaks (all implemented or in progress)

BC 2004

Saskatchewan 2007

Develop and exercise Foreign Animal Disease Emergency Support (FADES) plans in all provinces Develop collaborative arrangements with HC and PHAC to increase federal capacity to respond to zoonotic disease outbreaks Review and formalize emergency response protocols Develop IT/IM capabilities to ensure more timely and efficient management and transmission of field and laboratory data Implement ICS structure during emergencies Clarify respective biosafety responsibilities of the CFIA and public health authorities in response to zoonotic disease outbreaks Involve poultry industry in development of biosecurity programs Develop Standard Operating Procedures to ensure HR, finance, IT/IM, and administrative support to Emergency Operations Centres

Implement training sessions to practise roles and responsibilities of the ICS Work with partners at regional level to coordinate inter-jurisdictional flows and triggers with other federal departments and agencies Clarify and strengthen MOUs regarding existing interdepartmental links / create new agreements when necessary Reconfirm official communications protocols with federal partners Increase training efforts to help ensure adequate succession planning for future outbreaks Review Emergency Operations Centre logistics procedures and ensure proper supply chain management pathways are followed and communicated at regional level Propose plans for increasing internal surge capacity / continue improving business continuity planning Dedicate resources to globally monitor AI strain modifications and ensure improvements to screening tests Make additional funding available for augmenting IT/IM systems, such as the Canadian Emergency Management Response System (CEMRS) and the Laboratory Sample Tracking System (LSTS), to increase functionality and response time

Note: The lessons learned review of the BC outbreak was prior to implementation of the AI/PI Initiative.

The Alberta FADES plan identifies eight areas of support to the CFIA and involves multiple federal/ provincial/local, and industry organizations in providing this support, as summarized below.

Table 3: Areas of support and stakeholders in Alberta FADES agreement

Areas of support	Federal agencies	Provincial agencies	Industry organizations
FAD prevention Economic impact and compensation Diagnosis, trace back, and epidemiology Eradication, destruction, disposal, cleaning, and disinfection Movement control Communications and public information Public health and food supply Geographical information system and database management	CFIA Public Safety and Emergency Preparedness Canada Agriculture and Agri-Food Canada Health Canada Public Health Agency of Canada Prairie Farm Rehabilitation Administration Canada Border Services Agency Communications Canada Public Works and Government Services Canada Industry Canada Human Resources and Skills Development Canada Environment Canada Parks Canada Indian and Northern Affairs Canada Western Economic Development National Defence – Canadian Forces	Alberta Agriculture and Rural Development Alberta Emergency Management Agency Alberta Environment Alberta Sustainable Resources Development Alberta Health and Wellness Alberta Solicitor General Alberta Infrastructure Alberta Transportation Alberta Employment and Immigration Alberta Aboriginal Relations Municipal governments	Livestock industry organizations Alberta Veterinary Medical Association Alberta Farm Animal Care Alberta Society for the Prevention of Cruelty to Animals Livestock Identification Services Western College of Veterinary Medicine Alberta Livestock Fairs and Exhibition Grounds Various financial institutions

Source: Alberta FADES Plan, May 27, 2008

Table 4: Pandemic influenza and emergency response activity of other federal departments

Department	Description
Public Health Agency of Canada	In the event of a pandemic, PHAC is the lead federal organization for coordinating the human health response through the Centre for Emergency Response and Preparedness (CERP). CERP’s responsibilities include, among other things, developing and maintaining national emergency response plans for PHAC and HC; monitoring outbreaks and global disease events; and assessing public health risks during emergencies. Within the Centre, the Office of Emergency Response Services is responsible for supporting emergency health and social services in the provinces/territories or abroad, which includes managing the National Emergency Stockpile System, administering the Quarantine Act, and overseeing the federal response to emergencies that have human health repercussions, including deployment of health emergency response teams. PHAC has also established an Incident Command System (ICS) for human health that runs parallel to the CFIA’s ICS for animal health; implemented a Canadian Pandemic Influenza Plan for the Health Sector; and developed a Tool Kit Pandemic Influenza Exercise for the Health and Emergency Social Services Sectors. The Pandemic Influenza Portal, which provides access to information from various Government of Canada departments and agencies (including the CFIA) on pandemic, avian, and seasonal influenza, is accessible through PHAC’s Web site. (Source: www.phac-aspc.gc.ca/cepr-cmiu/index-eng.php)
Health Canada	In the event of a pandemic, HC is responsible for providing emergency health care for First Nation communities on reserve, providing occupational health care for federal government employees, and approving new drugs and vaccines to treat Canadians and minimize the spread of disease in the event of an outbreak (i.e., regulatory preparedness). (Source: www.hc-sc.gc.ca/hc-ps/ed-ud/prepar/flu-pandem/index-eng.php)
Public Safety Canada	Public Safety Canada coordinates the overall federal government’s domestic response efforts. The Department provides support to government and key national players in responding to events of national significance. (Source: http://montebello2007.gc.ca/doc/pandemic_en.pdf) Public Safety Canada houses the Government Operations Centre, an advanced centre for monitoring and coordinating the federal response to an emergency. Public Safety also administers the Emergency Management Act and has developed an Emergency Management Framework for Canada, a National Disaster Mitigation Strategy, and a National Exercise Program. (Source: www.publicsafety.gc.ca/prg/em/index-eng.aspx)

Table 5: AI/PI activities and responsibilities (as of June 2007)

Activity	Lead	Timing
Policy, procedures, standard operating procedures, IT development, and disease modeling	Programs Finance, Administration and Information Technology (FAIT)	Ongoing
Developing A1 training material and trainers at field level Supporting the maintenance of Al staff inventory	Human Resources (HR) Programs Operations	Ongoing
Building, practicing, and maintaining field capacity Building Canadian veterinary reserve	Operations	Ongoing
Building and maintaining Al laboratory network Upgrading laboratories, quality assurance processes, and interoperability	Science FAIT (IMIT)	Ongoing
Research and development related to Al	Science	5 years
Developing, upgrading, and maintaining IMIT tools supporting Al preparedness and evolving to support other foreign animal disease (FAD) Procurement processes supporting Al preparedness are in place and are transferable to other FAD	FAIT	Ongoing
Supporting Al initiatives with necessary communications products	Public Affairs	Ongoing
Working with the international community in developing global strategies	All branches	Ongoing
Monitoring and reporting performance, spending related to Al funds A senior multi-branch oversight committee will monitor the Al performance indicators and provide direction as required CS leads corporate performance reporting (development performance reports [DPR] and related obligations) with all branches providing input	Audit, Evaluation and Risk Oversight (AERO), Corporate Services (CS), FAIT and branches that received Al funding	5 years
Participating in the interdepartmental governance (PHAC, CFIA, HC) of horizontal initiative (AI/PI funding)	Programs	5 years

Table 6: Major documents reviewed by the evaluation

• Assorted information from PHAC and HC related to avian and pandemic influenza (obtained through a PRA Internet search)
• AHPOC Avian Influenza-related Activities Tracking (updated October 2008)
• AHPOC membership list and meeting notes
• Avian Influenza Preparedness Oversight Committee (AIPOC) Record of Decisions March 3, 2008, to June 24, 2008
• Avian Influenza Working Group Project Plan Manual and Final Report of the Avian Influenza Working Group (June 2007)
• Canadian Notifiable Avian Influenza Surveillance System Report No. 2 (December 2008)
• CFIA Departmental Performance Reports for FY 2003/04, 04/05, 05/06, 06/07, and 08/09
• CFIA Web site
• FADES agreements for Alberta and British Columbia (obtained through a PRA Internet search)
• Financial information provided by FAIT
• Integrated Interdepartmental Risk Analysis
• Lessons Learned Report and MRAP – the CFIA's Response to the 2007 AI Outbreak in Saskatchewan
• Lessons Learned Review: The CFIA's Response to the 2004 Avian Influenza Outbreak in British Columbia
• Memorandum to Cabinet for the AI/PI Initiative (reviewed on site)
• National Avian On-Farm Biosecurity Standard
• North American Plan for Avian and Pandemic Influenza – Security and Prosperity Partnership of North America (August 2007)
• Notifiable Avian Influenza Hazard Specific Plan
• Pandemic influenza preparedness plans for most provinces/territories (obtained through a PRA Internet search)
• PHAC Departmental Performance Reports (Horizontal Initiatives) for FY 06/07, 07/08, and 08/09
• Readiness Exercise Plan for Avian Influenza
• Results-based Management and Accountability Framework for the AI/PI Initiative
• Spreadsheet – Progress on Key AI Activities/Deliverables (January-March 2008)
• Treasury Board submission for the AI/PI Initiative (reviewed on site)

---

¹Formative evaluations are typically conducted in the early stages of an initiative, assessing implementation effectiveness and focusing on improving performance.

²A detailed discussion of the context for the AI/PI Initiative can be found in Appendix A.

³All information in this section describing the AI/PI Initiative is taken from the Initiative’s Results-based Management Accountability Framework (RMAF).

⁴RMAF, p. 20.

⁵In addition, key informants reported that a small number of activities that were implemented as planned were not readily compatible with Agency practices or capabilities. Examples given were the purchase of mobile communications vehicles and hand-held global positioning system (GPS) devices, neither of which were fully integrated into the Agency's normal operating protocols.

⁶The CFIA describes its use of the ICS in its Animal Health Functional Plan, accessed from the CFIA Web site on January 8, 2010, www.inspection.gc.ca/english/anima/ heasan/man/ahfppfsa/ahfppfsa_5_1e.shtml.

⁷CFIA. Integrated Interdepartmental Risk Analysis. Version dated May 16, 2006.

⁸Lessons Learned Report: The CFIA’s Response to the 2007 Avian Influenza Outbreak in Saskatchewan, www.inspection.gc.ca/english/agen/eval/avflu2007/avflu2007e.shtml.

⁹Alberta FADES. May 27, 2008, p. 9.

¹⁰See Appendix D for an example of areas of support and stakeholders involved in a FADES agreement.

¹¹See the CFIA Web site: http://www.inspection.gc.ca/english/anima/heasan/ disemala/avflu/2004sum/revexae.shtml.

¹² There are four CFIA areas (Atlantic, Quebec, Ontario, and Western) and 18 regions. Information provided by the CFIA indicates that all four areas have participated in emergency exercises, as have 17 regions.

¹³Final Report of the AIWG (June 2007).

¹⁴There are many examples of inconsistencies. For example, "avian biosecurity on farms" is listed under "prevention and early warning" in the RMAF, but under "surge capacity" for the purpose of financial reporting. Activities absent from the RMAF but listed under financial reporting include real property, enhanced enforcement measures, AI-enhanced management capability, and performance and evaluation. Surge capacity consists of four activities in the RMAF, but seven activities are listed under this component in the financial reporting. The RMAF refers to "internal staffing reserve" while financial reporting refers to "field surge capacity."

¹⁵See Table 1, Appendix D, for further details.

¹⁶See the CFIA’s Reports on Previous Disease Incidents at http://www.inspection.gc.ca/english/anima/ disemala/avflu/reprape.shtml.

¹⁷Ligon, L.B. (2005). Avian Influenza Virus H5N1: A Review of Its History and Information Regarding Its Potential to Cause the Next Pandemic. Seminars in Pediatric Infectious Disease 16, 326–335.

¹⁸http://www.cdc.gov/flu/avian/gen-info/facts.htm

¹⁹http://www.inspection.gc.ca/english/anima/heasan/ disemala/avflu/bacdoc/appdeme.shtml

²⁰http://www.cdc.gov/flu/avian/outbreaks/past.htm

²¹http://www.who.int/csr/disease/avian_influenza/avian_faqs/en/index.html

²²http://www.who.int/csr/disease/avian_influenza/avian_faqs/en/index.html

²³http://www.who.int/csr/don/2004_03_02/en/index.html

²⁴UN Report Oct 08 – A Strategic Framework for Reducing Risks of Infectious Diseases at the Animal–Human–Ecosystems Interface.

²⁵de Jong, M.D., and Hien, T.T. (2006). Avian Influenza A (H5N1). Journal of Clinical Virology, 35, 2–13.

²⁶SPP: http://www.spp-psp.gc.ca/eic/site/spp-psp.nsf/vwapj/ pandemic-influenza.pdf/$FILE/pandemic-influenza.pdf

²⁷http://www.who.int/csr/disease/avian_influenza/avian_faqs/en/index.html

²⁸http://www.who.int/csr/disease/avian_influenza/avian_faqs/en/index.html

²⁹http://www.who.int/csr/outbreaknetwork/en/

³⁰Ong, A., Kindhauser, M., Smith, I., and Chan, M. (2008). A Global Perspective on Avian Influenza. Annals Academy of Medicine, 37(6) 477–481.