Working Conference on Strengthening the Evaluation of Real World Drug Safety and Effectiveness: Summary Report

Prepared by One World Inc. (OWI)
For Health Canada
November 2005

Help on accessing alternative formats, such as PDF, MP3 and WAV files, can be obtained in the alternate format help section.

(116K)

Table of Contents

Executive Summary

1. Introduction
2. Context

2.1 Introductory Presentations
2.2 Discussion
2.3 Pre-conference Survey

3. Problem Analysis

3.1 Small-group Discussion of Problem Statements
3.2 Discussion

4. Solution Analysis

4.1 Presentation: Complementary Approaches
4.2 Small Group Dialogues on Approaches
4.3 Concluding Presentations

Appendix 1: Conference Agenda

Executive Summary

The Working Conference on Strengthening the Evaluation of Real World Drug Safety and Effectiveness brought together more than 100 stakeholders, including members from government, industry, academic research, and groups representing health care providers and patients, to explore how to harness and improve Canada's capacity to monitor and evaluate the "real world" (post-market) safety and effectiveness of drug products, and incorporate this information into decision-making processes. The conference was one consultation opportunity in the development of the National Pharmaceuticals Strategy (NPS). It included presentations and facilitated small-group sessions in which diverse groups of participants worked on clarifying the problems to be addressed and elements of possible solutions. Facilitation and note-taking was provided by one World Inc.

Initial presentations summarized the content and progress of the NPS and the case for action on evaluation of real world safety and effectiveness. Presenters discussed the need to improve such evaluation, as well as what can be learned from pharmacosurveillance initiatives in jurisdictions outside Canada, including the experience of Australia.

In facilitated small groups, conference participants discussed problem statements dealing with safety (adverse drug reaction information from spontaneous reporting, and data for pharmacosurveillance activities); effectiveness (evaluation of therapeutic effectiveness); the evaluation function (organization of information-gathering and analysis activities); priority-setting (criteria and process for selecting drugs to be evaluated); and organization (system structure and integrity). In these discussions, they identified additional aspects of the issues, including:

• problems relating to the current adverse drug reaction (ADR) reporting system;
• limitations of databases for pharmacosurveillance activities;
• the need for a holistic definition of real world effectiveness and a vision of the mandate for a structured process for effectiveness evaluation;
• concerns about the collection, analysis and dissemination of information and data;
• factors that need to be taken into consideration in the development of a standardized process or criteria for selecting drugs to be evaluated; and
• the need for clarity in defining the mandates, authorities and roles related to any overall structure for evaluating real world drug safety and effectiveness, including mechanisms for communication and dissemination.

Participants then worked in four facilitated small groups to discuss two complementary approaches for strengthening the evaluation of real world drug safety and effectiveness. These approaches were not mutually exclusive, and were presented not as models, but as concepts to stimulate discussion. Through dialogue, each group developed points of common ground regarding elements of a desirable approach to evaluation of real world drug safety and effectiveness. These were presented to a plenary session. Some key points noted included:

• the need to change the health care "culture", to emphasize collaboration/communication with all stakeholders, transparency and feedback;
• the need to improve linkages between data sources and systems that already exist, to make use of a full range of sources of data (including spontaneous reports, electronic records, administrative data, health survey data, registries, data from compulsory "Phase V" post-market monitoring, etc.). Suggestions were made regarding improving data collection systems (e.g., improving the spontaneous ADR reporting system by making the system easier to use, providing incentives to report, establishing local centres of ADR reporting);
• the importance of ownership/commitment to priorities and collaboration on priority-setting by the full range of stakeholders (including patients and health care providers);
• support for local creation and sharing of data, as well as feeding of this data into a national system or network with national standards so that information is comparable;
• the importance of building or improving capacity in academic environments; and
• the importance of building on what already exists and learning from existing models such as the Canadian Coordinating Office for Health Technology Assessment (CCOHTA) and the Canadian Patient Safety Institute. Each of the four groups envisaged a model for governance of the evaluation system. These models had some commonalities in that each included some sort of national/central governing body (council, board, forum, multi-stakeholder advisory committee) with cross-cutting coordination/extensive involvement, commitment and collaboration by research centres, regulatory bodies, patients, health care providers, industry and governments.

The conference ended with a multi-stakeholder panel discussion and presentations, in which participants from a range of sectors summarized what they had heard and challenges for the future. The issues addressed by the conference were complex and challenging, and were approached by participants in a spirit of collaboration. Recommendations arising from the conference will be used to develop a way forward on the issue of evaluation of real world safety and effectiveness, in preparation for a report due to Ministers of Health by June 2006.

1. Introduction

The Working Conference on Strengthening the Evaluation of Real World Drug Safety and Effectiveness took place in Ottawa, Ontario on September 14 and 15, 2005. The conference brought together a variety of stakeholders, including members from government, industry, academic research, and groups representing health care providers and patients, to explore how to harness and improve Canada's capacity to monitor and evaluate the "real world" (post-market) safety and effectiveness of drug products, and incorporate this information into decision-making processes, so that patients experience better health outcomes and fewer adverse effects.

The conference was one consultation opportunity in the development of the National Pharmaceuticals Strategy (NPS), an initiative called for by Canada's First Ministers in September 2004 as part of the 10-year plan to strengthen health care. The NPS is a federal-provincial-territorial (FPT) initiative designed to create a policy framework for an integrated, comprehensive and collaborative approach to the availability and use of pharmaceuticals in Canada. An important element of the NPS is the concept of stronger monitoring and evaluation of real world drug safety and effectiveness.

The NPS highlights the need for coordination and cooperation among governments and stakeholders to effectively manage all stages of a drug's life cycle. Much of the conference consisted of facilitated small-group sessions in which diverse groups of participants worked on clarifying the problems to be addressed and elements of the approach to be taken to improve the evaluation of real world drug safety and effectiveness. (See the conference agenda in Appendix 1).

The issues addressed by the conference were complex and challenging, and were approached by participants in a spirit of collaboration. Recommendations arising from the conference will be used to develop a way forward on the issue of evaluation of real world safety and effectiveness, in preparation for a report due to Ministers of Health by June 2006.

2. Context

2.1 Introductory Presentations

Opening Remarks from the Co-Chairs
Dr. David Clapin (Associate Director General, Marketed Health Products Directorate, Health Canada) and Susan Paetkau (Director, Drug Programs Branch, Ontario Ministry of Health and Long-term Care)

Dr. Clapin and Ms. Paetkau welcomed participants to the conference. They noted that the challenge for the conference was to analyze the problems and begin a discussion about solutions, and that opportunities for dialogue would continue after the conference.

National Pharmaceuticals Strategy: Progress and Next Steps
Dr. Penny Ballem (Deputy Minister, British Columbia Ministry of Health) and Ian Shugart (Senior Assistant Deputy Minister, Health Policy Branch, Health Canada)

In September 2004, Canada's First Ministers agreed on a 10-Year Plan to Strengthen Health Care, which includes a National Pharmaceuticals Strategy (NPS). The NPS is a collaborative, integrated and comprehensive approach to pharmaceuticals in Canada. It is a multi-year strategy that is national in scope and responsive to regional needs. It is an opportunity for governments to work together with input from health care providers, researchers, policy-makers, industry, patients and the public.

There are nine key elements to the NPS: one of these elements is evaluation of real world drug safety and effectiveness. A Ministerial Task Force has been established and five areas of priority have been identified: one of those areas of priority is real world drug safety and effectiveness. Throughout Fall 2005 and Winter 2006, there will be ongoing consultation on specific priority areas. Real world drug safety and effectiveness is a key part of the overall quality and safety agenda, and a full range of stakeholders need to be involved in helping to develop the best approach.

Case for Action: Why is this an issue now? What lessons have we learned so far? Federal, provincial and international views}
Abby Hoffman, Executive Coordinator - Pharmaceuticals Management Strategies, Health Policy Branch, Health Canada

Drug safety is a fundamental aspect of federal regulatory responsibility. Pharmaceuticals are an increasingly important form of health care. As utilization goes up, and with new, more expensive drugs, system sustainability is a critical challenge. There are increasing public concerns about safety, therapeutic benefit, access and affordability. Expanding access to drugs will depend on demonstrating that drugs are safe, have therapeutic benefits and are cost-effective relative to other interventions.

Safety and effectiveness are two sides of the same coin. When a new drug is introduced, we need to know whether it is safe and lives up to its billing of therapeutic benefit; how it is prescribed and used, for which indications and with what effects; changes in the size and characteristics of the patient population; and whether, in what circumstances and with what consequences, new drugs displace old ones. Evidence is needed from the entire spectrum of the product life cycle, from pre-market to post-market.

There are many different perspectives; transparency and communication are important. Challenges include dealing with system disconnects, generating knowledge in ways which increase the likelihood of uptake (influencing policy and practice), and looking at old products or new products.

Inappropriate Prescribing
Dr. Ed Hunt, Medical Consultant, Newfoundland and Labrador Department of Health and Community Services

There are many drivers to inappropriate prescribing; changing aggressive marketing by pharmaceutical manufacturers could probably create the greatest benefit in terms of safety, efficiency and effectiveness. The role of ethics committees needs to be considered with respect to marketing disguised as research. In Newfoundland and Labrador, legislation has been introduced to establish a provincial ethics committee ─ a first in Canada. There are risks of approved research at the community level: when the research is finished, patients are left abandoned, and many cannot afford to buy the drugs. There are new advocacy organizations that seem to be linked to a specific product or manufacturer. Opportunistic marketing is an issue.

What is needed: A national source of unbiased information based on evidence (e.g., COMPUS), an effective jurisdictional dissemination system at the point of contact; regulated research; and regulated promotional activity.

Learning from Drug Plan and Drug Regulatory Pharmacosurveillance Initiatives Outside Canada
Michael Paterson, Senior Research Coordinator, Institute for Clinical Evaluative Sciences

A review was conducted of novel drug plan and drug regulatory pharmacosurveillance initiatives in the United States, United Kingdom and selected other jurisdictions. The review was based on print material, and little information was available on funding, governance and effectiveness.

Drug regulatory initiatives included data mining; active adverse drug reaction (ADR) surveillance; independent, multidisciplinary evaluation of suspected ADRs; and formal pharmacoepidemiology studies. Drug/health plan initiatives include pharmacoepidemiology studies; registries; and conditional coverage agreements. There is much to be learned from existing pharmacosurveillance frameworks, and potential for international collaboration.

Real World Safety and Effectiveness
Dr. Richard Hill, Deputy Director, Adverse Drug Reactions Unit, Therapeutic Goods Administration, Australia

Australia uses a common formulary but there is a lack of global data linkages, although there are some small projects. The National Medicines Policy, similar to Canada's NPS, was endorsed in 1999. It has four components: timely and affordable access to medicines; appropriate quality, safety and efficacy of medicines; quality use of medicines (QUM); and a responsible and viable pharmaceutical industry. QUM means: selection of the best management option; appropriate choice of medicines; safe and effective (real world) use at an individual or population level.

QUM principles are: strategic research required at all levels; continuous feedback to improve practice; routine datasets should be established, including a patient-linked pharmacoepidemiological database (this has yet to be done). The QUM process includes: existing evidence or sponsored research; evidence for effective interventions; implementation of trials; facilitation of implementation; evaluation of outcomes; and feedback. National Prescribing Service Australia is independent and government-funded; it targets a wide range of efforts to health care professionals and consumers. These include campaigns to control over-use and over-prescribing of antibiotics and provision of evidence-based information about drugs to physicians and pharmacists.

2.2 Discussion

In a question and answer session, a participant noted that governments are having positive collaborations with the pharmaceutical industry in Newfoundland. Dr. Hunt agreed, and referred to work being done to involve pharmacists effectively with primary health care teams. Another participant noted that Canada's NPS is analogous to the national immunization strategy and that there might be opportunities for sharing of experience between the strategies.

There was a question about whether the NPS would include looking at regulation of pharmaceutical promotion as one means to address the issue of inappropriate prescribing. Ms. Hoffman noted that this is an open question and that the issue needs to be dealt with. What mix of regulation and policy would be appropriate is not clear, but there are policy issues involved.

2.3 Pre-conference Survey

A survey had been carried out prior to the conference by Sintera Inc. on behalf of Health Canada. The results were not presented at the conference due to time constraints, but are to be included in the web materials posted subsequent to the conference. The qualitative survey had been distributed via e-mail to 100 people; 42 completed surveys were returned. A range of participants responded, with the greatest number of responses coming from universities and research organizations.

3. Problem Analysis

3.1 Small-group Discussion of Problem Statements

Jacquie Dale, the plenary facilitator, introduced the task and asked conference participants to break into small groups to discuss six problem areas described in the conference Dialogue Guide. The groups considered the problem statements presented in the guide; they discussed whether there was agreement with the statements and, if not, how the statements should be modified.

The discussions followed a "World Café" format. There were 12 tables (two for each problem area) where up to 10 participants could discuss a problem area. There were two rounds of discussion, so each participant had a chance to discuss two problem areas; participants self-selected based on their interests. Discussions at the tables were facilitated by One World Inc. (OWI) facilitators and volunteer facilitators from Health Canada's FACT network, and notes were taken by OWI personnel and volunteers from the FACT network.

In a plenary session following the World Café discussions, facilitators presented the common themes that had emerged from the discussions of each problem area. These are summarized below.

Problem Area 1: Safety: Adverse drug reaction information from spontaneous reporting

Problem Statement: The utility of ADR reports in decision-making about the real world use of drugs is limited by significant under-reporting, as well as incomplete reporting.

In general, in the four groups that discussed this, there was agreement that under-reporting and incomplete reporting are issues. However, other reasons and factors that reduce the utility of ADR reports in decision-making were noted. The discussions dealt with the spontaneous reporting system administered through Health Canada's Canadian Adverse Drug Reaction Monitoring Program (CADRMP). A number of limitations of the ADR system and reporting process were noted. These included a lack of communication to consumers and physicians about ADR reporting. There was also considerable discussion of the general burden of reporting (time required to report, questions of privacy, confidentiality, liability, etc.).

Other limitations noted were: a lack of resources to report; a lack of clarity about the definition of an ADR (e.g., the difference between an event and a reaction); and a lack of awareness about the need to report, what and when to report, and how to go about reporting.

It was noted that there is insufficient feedback to those who do report regarding what results from ADR reports. This can create a disincentive to report. It was also noted that there is an apparent lack of capacity to process the data that is collected, and concerns were noted about overloading the system if reporting is increased.

Problem Area 2: Safety: Data for pharmacosurveillance activities

Problem Statement: The fact that existing databases in Canada contain incomplete data on use of a drug, or the population using it, limits their use for calculating ADR incidence or in pharmacoepidemiology studies.

The four groups that discussed the statement agreed that it is difficult to calculate incidence, but would reword parts of the statement. There was discussion of gaps in data collection and the lack of partnerships or linkages.

A number of key points were reported that had been noted in the various groups' discussions (but which were not necessarily common to all groups). It was noted that what exists now in terms of databases is not being fully utilized. There is a lack of compliance with the voluntary reporting template, so that data obtained are incomplete. A continuum of pre- and post-market reporting and probationary licensing could be beneficial (i.e., link post-approval adverse reactions with pre-market experience).

There is no national infrastructure for data (although there are some provincial linkages). An inventory of available databases is needed. Different groups (industry, researchers, clinicians, patients) have different information needs. ADRs are not linked to any other database. Disease-based population registries could be useful. There is a need for FPT political will for disclosure, to link data nationally and standardize data.

Surveillance of the use of drug samples is lacking. Concomitant medications are often not addressed in current reporting, especially the uses of over-the-counter and natural health products. It was noted that ADR incidence rate is not the only goal: this is too restricted. There is a need to follow up on patient outcomes. There is a lack of feedback/follow-up to people who are reporting, re. what is happening to the information being collected and why information is being collected.

Problem Area 3: Effectiveness: Evaluation of Therapeutic Effectiveness

Problem Statement: There is no structured process for investigating real world drug effectiveness to assist in the ongoing determination that the therapeutic benefits of a drug in real world use outweigh its risks.

The four groups that discussed this problem statement agreed that there are not structured processes in place: there are some elements, but no comprehensive, structured process. Other key themes that emerged included the need for a definition of effectiveness. There is a need for a holistic approach to the definition of effectiveness that takes into account the diverse perspectives (patient, physician, regulator, etc.) It is important to remember that real world effectiveness involves individuals (patients).

There is also a need to define "real world". There was some agreement that it means post-market, but also the concept should be broader in its approach and reflect the pre and post-market continuum. There is a need for a vision of what the mandate of a structured process should be. There is a need for coordination since multiple groups are involved in pieces of the process.

It was noted that there is insufficient appropriate investigation of real world effectiveness. There is little or no incentive to investigate real world effectiveness. Industry regulation may have an impact on the uptake of evidence in decision-making.

Problem Area 4: Evaluation Function: Organization of information-gathering and analysis activities

Problem Statement: Existing capacities in the generation and evaluation of real world safety and effectiveness evidence are not organized in a way that best meets the needs of decision-makers.

Three groups discussed this problem area. The groups essentially saw the problem statement as incomplete. In particular, the word "decision-makers" as defined in the description accompanying the problem statement (regulators, drug plan administrators and health professionals) was seen as too narrow in scope: consumers/patients should be included. The various needs of decision-makers need to be explored, as different decision-makers have different information needs. The term "organized" was seen as too narrow: the problem also includes long-term planning and sustainability.

Other points raised were that considerable information/data is collected, but there were questions about whether it was the right information/data. Data are not being linked, and there is no comprehensive or complementary approach to collecting data and no overall or long-term plan for optimal use of the data. To consider only safety and effectiveness was seen as too narrow: patient data such as socioeconomic data should be included.

It was noted that the statement was missing analysis and dissemination, including knowledge translation. There is a need for independent capacity: researchers and decision-makers need to work together to prioritize and put in place long-term, well-resourced and sustainable approaches and determine the type of evidence that needs to be generated. The information collected needs to ultimately go to a known and trusted component of the system, a source of information that is available to, and known by, all concerned.

One of the groups proposed new wording for the problem statement: "Existing capacities in the infrastructure, expertise, generation, evaluation, integration and translation of real world safety and effectiveness evidence are insufficient and not organized in a way that meets the needs of all decision-makers."

Problem Area 5: Priority-setting: Criteria and process for selecting drugs to be evaluated

Problem Statement: There is no standardized process or criteria for selecting individual drug products or drug classes for real world safety and effectiveness evaluation.

Two groups discussed this problem area. The status quo of examining effectiveness on a case-by-case basis was seen as a problem, as were the current drivers, which tend to be medical catastrophe and/or the media. There is difficulty in the very notion of standardized approaches when there is no single body to guide the review process.

Factors were noted that need to be taken into consideration: the varied perspectives of the parties involved (health care providers, governments/payers etc.) need to be taken into account; standardization needs to be redefined to be more reflective of the real world context ("universally accepted" is a better descriptor); there is a distinction between cost and therapeutic effectiveness, and at the same time there are linkages between the two.

The groups agreed that much more coordination and collaboration are needed, some content already exists for evaluation criteria and some international standards and/or guidelines are available and may be adaptable to the Canadian context.

One group proposed rewording the statement to say: "There is no coordinated and collaborative approach, nor universally accepted guidelines for selecting individual drug products or drug classes for real world safety and effectiveness evaluation."

Problem Area 6: Organization: System structure and integrity

Problem Statement: In Canada, there is no overall structure for evaluating real world drug safety and effectiveness that is sufficiently coherent, organized and integrated to adequately support better health outcomes and fewer adverse events.

Four groups discussed this problem area. There was general agreement that there is no overall structure and that what exists is not cohesive, well-linked or integrated; gaps exist. However, caveats and modifications to the problem statement were also noted. For example, one group noted that a new structure is not necessarily required. There are already many elements in place, many strengths to build on, and existing resources in some (but not all) jurisdictions.

Many of the groups felt the need for greater clarity in the statement. There is a need for clarity in the definitions of safety and effectiveness, and the links and relationships between the two. What is meant by "structure"? There is also a need for a clear understanding of mandates and authorities related to a structure.

Other items were noted as insufficiently reflected in the statement. There is a need for clarity of roles but also integration across roles. Learn from other countries. The roles and involvement of the public and private sectors need to be addressed, and opportunities for sharing of data and research. Transparency and accountability are issues. Evaluation in the real world should be integrated with the rest of the system, particularly pre-market. Greater linkages are required, especially across jurisdictions. In the development of a system, roles and mandates need to be understood, committed to and endorsed. Health Canada is presently focused primarily on safety but there is a need for a clear understanding of the department's role related to effectiveness. Health Canada could and should take a lead role federally, but should ensure the collaboration of all parties.

There was considerable discussion of the importance of a greatly improved mechanism for communication and dissemination; one group saw this as a separate problem area. There is a great need for an improved system for acquiring and disseminating information, determining what evidence is needed, etc. One group noted a caveat that an effective organizational structure could contribute to, but not ensure on its own, better health outcomes and fewer adverse events.

3.2 Discussion

Several points were made in a plenary session following the reporting of the groups' discussions. A participant noted that the majority of adverse drug reactions are reported by pharmacists. Another participant noted that the issue was broader than adverse drug reactions, and included pharmacosurveillance.

A participant noted that perhaps there should be a separate problem statement dealing with issues of transparency, institutional conflict of interest etc. that are broad, non-technical issues. Confidentiality and security issues are important, and there is a need to build infrastructures that are responsive to ethical as well as legislative requirements.

Another participant noted that risk/benefit analysis needs to take into account that risk/benefit is defined differently depending on populations, diseases and disabilities. Another noted that the process for evaluating real world drug safety and effectiveness should be ongoing, transparent, centralized and responsive to needs such as the sensitivities of patients.

A participant clarified that in discussing the spontaneous reporting system, it had been recognized that there should be systemic monitoring as well; the two are not mutually exclusive. Another noted that one group had suggested adding a point to Problem Statement 1 about inadequate relevant information on the reports themselves.

4. Solution Analysis

4.1 Presentation: Complementary Approaches

Dr. Bruce Carleton (Children's & Women's Health Centre of British Columbia)
and Dr. Supriya Sharma (Health Policy Branch, Health Canada)

Two complementary approaches for strengthening the evaluation of real world drug safety and effectiveness were presented. (The two approaches are described in detail in the Dialogue Guide.) The approaches are organized according to seven general feature areas. It was noted that the two approaches are not mutually exclusive, and that the approaches were intended not as models, but as a means of stimulating discussion.

In a question and answer session, a participant asked if there had been discussion between the federal, provincial and territorial governments on the feasibility of a centralized model. Dr. Carleton noted there had not been such discussion because an actual model has not yet been developed. The working conference is intended to get the views of many stakeholders, not just governments, and with that input begin to build a model. Another participant questioned if a decentralized approach would mean devolving safety assessment to the local level; Dr. Carleton replied that this was not the intention.

4.2 Small Group Dialogues on Approaches

4.2.1 Initial Dialogues

Participants were divided into four groups and spent several hours, with the assistance of a facilitator, discussing the two complementary approaches. The original intention of the conference had been to use a fairly structured dialogue process in which the groups would work through what they considered the positive and negative features of each approach, in the seven general feature areas. However, several of the small groups did not find this to be the best way to consider the issues and instead discussed issues in a less structured fashion, before focusing on areas they considered to be key and developing ideas/recommendations for those areas.

On the morning of September 15, in a plenary session, the groups' facilitators reported the main ideas that had emerged. Ideas that emerged from more than one group included the importance of:

• a hybrid approach building on local strengths, structures, databases, networks etc., with a national structure for coordination
• connecting internationally as well as nationally and locally
• collaboration and cooperation (one group referred to "a culture of collaboration")
• patients' involvement/focusing on the right outcomes for the public
• coordination/integration of data sets, improvement of access to data
• an incremental approach, building on what exists and is feasible.

4.2.2 Final Small-group Dialogues

Participants returned to their four facilitated small groups for several hours in the morning of September 15 to discuss and begin to build elements of a desired model. In a plenary session in the afternoon, members of the small groups presented their groups' common ground. After each group's presentation, plenary participants were asked to respond on wireless keypads to questions about the perceived importance of the features presented. Participants' responses were collected, analyzed and presented back to the plenary session by Sharpe Decisions Inc. using a real-time measurement system.

4.2.2.1 Group 1 ("Red" group)

Perspectives on Change

1. Changing the Culture

There needs to be more collaboration/communication with all stakeholders including patients/the public (this can be done in the short-term). Tools need to be developed to support a culture of collaboration. In the long term, what is needed is a new vision of health care in Canada, based on a culture of collaboration. Changing the culture will require flexible expectations; social marketing could be part of a mechanism.

Transparency and feedback are important. There is a need for incremental reporting. Interprofessional education is needed. This should build on what exists already and involve leadership with national professional bodies. Current privacy legislation creates barriers to sharing health information.

2. Coherent Legislation (Long-term)

In the short term, there should be an examination of what legislated change is required to support a new model (e.g., looking at privacy issues between Federal/Provincial/Territorial authority to have the data generated). Development and implementation of any needed legislation would be long-term.

3. Better Linkages

Better linkages are needed between what already exists.

Priority-setting

Concentrate on a few priorities agreed upon by provinces and territories (using the Canadian Coordinating Office for Health Technology Assessment (CCOHTA) as an example). Explore the feasibility of data collection and sharing at the local level. There needs to be ownership/commitment to the priorities, and collaboration on priority-setting by patients/health care providers, industry and government. Priorities need to address stressors at the provincial level (i.e., sustainability issues).

Governance

A national "council" with comprehensive membership should be responsible for overseeing the collection of data. (The term "council" was used to convey the idea of a national entity with comprehensive membership, but a specific entity was not defined). Funds could be provided by the federal government or through multi-party funding. The council would disseminate information on various research studies to the public.

Post-market research would be compulsory, i.e., "Phase V" studies under the Notice of compliance (NOC)/Notice of Compliance with Conditions (NOCc) regulatory framework. Post-market research would be registered and approved; it would be funded by industry.

Performance measurement would be based on patient outcomes.

Managing Safety and Effectiveness Data

Data sources would include public and private sources (drugs and medical services), "Phase V" studies, ADRs, in-hospital data (drugs, medical services, imaging, lab), and drugs and people not covered by plans. Data would be created locally but fed into a national "network" with national standards so that information is comparable. The national "council" would have responsibility for access/custody of the data.

Discussion

In a question and answer session, it was clarified that legislative change is not absolutely required, but the potential need for it should be explored. It was noted that privacy legislation can be an impediment for researchers, and when databases are crossing borders there should be an element of legal protection for the providers of data.

Responses to Real-time Questions

Plenary participants responded, using a real-time measurement system, to eight questions about the group's presentation. Support was greatest for the following factors.

Perspectives on Change:

• 68 percent thought "improved linkages for what is already in place" were "very" important
• 67 percent thought it was "very" important to change to a culture of greater collaboration and communication with all stakeholders, including patients and the public.

Priority-setting:

• 51 percent thought that "collaboration on priority-setting by patients and health care providers, industry and governments" was "very" important.

4.2.2.2 Group 2 ("Blue" group)

Governance

The group used the analogy of an "umbrella" to explain its approach to governance. Research centres, regulatory bodies, patients, health care providers and industry would be in the spokes and fabric of the umbrella. At the top of the rod, where the spokes connect, would be a National Pharmaceuticals Safety and Effectiveness Board. The Board would be responsible for agenda-setting. At the handle would be F/P/T governments. The spokes lead to the Board, but organizations in the fabric also would interact with each other in many ways.

Capacity-Building

The group saw a need to build or improve capacity-building in academic environments. Such environments are the national resource to provide what is needed to fulfill the mission of the National Pharmaceuticals Strategy.

Enhance Data Sources

Types of data that would be sources for this model include the following:

• Spontaneous reports: enhance patient and health care provider generation of spontaneous ADR reports. Make it easier (e.g., provide a 1-800 number with a knowledgeable person to collect high-quality data); provide assistance in clinics and pharmacies. This, rather than a mandatory approach, will improve adverse event reporting.
• Electronic records will be a valuable resource (e.g., such records contain information on why the medication was used).
• Administrative data
• Health survey data
• Registries (disease registries, patient registries)

It is important to harmonize and work with what exists today; a minimum data set with collaborative links would be a good starting point. While the ideal may be to completely harmonize and synchronize, start with what exists now.

There is a need for people who know how to do the analysis, who understand the context in which data are generated and the clinical context in the analysis. It is not necessary to have unanimous consent to analyze a question: if only three provinces want to look at a question, proceed with what exists and do it well.

Discussion

In a question and answer session, it was noted that the group had not discussed how the "umbrella" system would be resourced.

Responses to Real-time Questions

Plenary participants responded, using a real-time measurement system, to four questions about the group's presentation. Support was greatest for the following factors and concepts:

• 64 percent thought it was "very" important to have a spectrum of complementary data sources
• 53 percent thought it was "very" important to work towards harmonization of data, including a common software infrastructure.

4.2.2.3 Group 3 ("Yellow" group)

Governance

The system should build on what we have. The infrastructure should involve cross-cutting coordination but central leadership of some kind is also needed. The coordination structure should be based on a good Canadian model (not necessarily from the health field): sharing common values, key interests and clear intent. There needs to be leadership and a visionary focal point; this requires commitment from all players to accept this leadership.

There are a number of barriers to such a system. There is no common set of values at present. Jurisdiction, risks in the current ADR system (e.g., liability) and the fact that we are not all talking the same language (e.g., there are different protocols) are barriers. There is a feeling that people on the ground are not given credit for the work they do. The drug safety paradigm is not well defined.

The group's proposal, for the short term, is to establish a pan-Canadian forum to discuss how to enable maximum participation from all. F/P/T governments, researchers, industry, health care practitioners and consumers/patients should all be represented at this forum. The Canadian Patient Safety Institute is an example of a model. Steps involved are: develop a business case for the process; initiate the process; and evaluate.

Data Utilization

For safety and effectiveness, provincial drug plan databases and private health plan data are important. Linking of databases is key. Subpopulations within databases (e.g., over-65s) may be "canaries in the coal mine", likely to first show problems with a drug. The electronic health record is an important tool. Drug sample data and teaching hospital databases should be used.

Health Outcomes

For health outcomes, similar databases were noted. There are issues around privacy, ethics and timing/timeliness. It is also important to consider the ethics of not using data when it exists and could be used to improve patient outcomes.

ADRs

Quality and quantity of ADRs need to be improved. Positive incentives or reimbursement to motivate front-line professionals to report should be considered (e.g., ADR reports to count towards professional development credits). Other "types" of ADR (e.g., errors and incidents) need to be considered. Reporting should be facilitated through dedicated resources, improved feedback, linkage to Continuing Medical Education/Continuing Professional Development "points", and tools.

Discussion

Responses to Real-time Questions

Plenary participants responded, using a real-time measurement system, to five questions about the group's presentation. Support was greatest for the following factors and concepts:

• 72 percent thought it was "very" important to increase the quality of ADR reporting (only 43 percent thought it was "very" important to increase the quantity)
• 58 percent thought linkages between various databases (private and public) would be "very" favourable.

4.2.2.4 Group 4 ("Green" group)

The group developed a vision of a hybrid model, with a detailed matrix. One axis of the matrix dealt with safety (ADR reports, pharmacosurveillance data), effectiveness and evaluation, and priority-setting and organization. The other axis dealt with organizing principles, perspectives on change, the priority-setting framework, and system governance. Principles and short-term and long-term actions were identified.

Safety

Collaboration and relationship-building are key to overcoming the "fear factor" and lack of knowledge about ADR reporting. Local centres of ADR reporting could be established to encourage reporting, improve reporting and ascertainment of real ADRs versus unsafe use, and provide feedback. Behaviour change is based on trust and is only going to occur in local settings. No regulatory change is needed. Demonstration projects would be needed in the short term.

Effectiveness and Evaluation

There are problems in the evaluation of therapeutic effectiveness and in the organization of information-gathering and analysis. It is important to have a strong governance structure to enable input, credibility and buy-in for the entire process. Build on existing models: borrow from work already done. No regulatory change is needed; demonstration projects can be done.

In the short term, the governance structure of CCOHTA could be a model (not responsible to any organization, owned by all jurisdictions, advisory committee). The UK Bio-bank is another structure (Board of Directors with an independent ethics advisory committee; intelligent accountability - explicit criteria with implicit application). In the short term, regulatory change can take place at the Ministerial level; in the long term, legislative change would be needed (i.e., to provide Health Canada with the authority to ensure requirements are being met). There needs to be further small-group discussions about governance to build on what has been discussed at this conference.

Priority-setting and Governance

The basic premise is that drugs will be more fully integrated into the health system. The overall safety and effectiveness of drugs needs to be looked at within the context of other treatment options. A culture of evaluation is needed for other treatment options that are considered as alternatives to drugs.

A multi-stakeholder (patients, providers, governments, industry etc.) advisory committee is needed for priority-setting and governance. It should be established in the short term, as it will propose what the overall decision-making body will do and what form it will take. The environment within which it works needs to be "safe", with meaningful deliberation (a balance between transparency and intellectual property).

A decision-making body is needed that has authority around issues of, for example, post-market surveillance. It would evolve from the multi-stakeholder committee. It must have stability (funding) and authority. Its location needs further discussion. One of its main roles would be a focus on the need for a continuum between pre-market and post-market evaluation (e.g., safety and effectiveness signals could be established earlier in the process).

Discussion

A participant noted that agencies such as CCOHTA can take years to be legislated and get up and running. Also, there is the issue of accountability for a collaborative structure. The response from the group was that in the short term, the multi-stakeholder committee would help to establish the governance structure.

Responses to Real-time Questions

Plenary participants responded, using a real-time measurement system, to four questions about the group's presentation. Support was greatest for the following factors and concepts:

• 61 percent thought it is "very" important to integrate drugs more fully into the health care system
• 56 percent thought it was "very" important to build on existing models for effectiveness and evaluation
• 50 percent thought a model that incorporates a decision body with authority around post-marketing surveillance was "very" important.

4.3 Concluding Presentations

4.3.1 Panel Discussion: Feasibility of Proposed Solutions - A Critical Look

Moderator: Pierre-Gerlier Forest, Chief Scientist, Health Canada

The panellists gave their perspectives on: what they had heard and seen in the last two days; do we have answers to the questions; the feasibility of solutions; do we have the capacity and resources to address the issues discussed? Key points from their presentations follow.

Kathy Kovacs Burns, Chair, Best Medicines Coalition

Solutions principles heard during the conference were: transparency; accountability; integration; shared values; knowledge transfer and dissemination; education and awareness; a comprehensive system respectful of all; immediate health outcomes; inclusion; cooperation and collaboration; building on what already exists that is good; and sustainability. A broad scope is needed: a continuum from pre- to post-market, including patients, physicians and others in gathering data at the local level and ensuring it gets to the national level; and different data sources. Capacity-building is key.

A change in culture and mindset is needed, at both the local and national levels. Challenges remain: to look at what is short-term and long-term,

where funding is available for the short-term, what form the next steps of consultation and process should take, continuing to bring together patients, practitioners and decision-makers at various levels. There is a challenge in looking at how legislation and regulations will fit in, in the longer term, and in dealing with privacy and ethical issues.

Dr. Rav Kumar, Vice President of Pharmaceutical Research and Development and Regulatory Affairs, GlaxoSmithKline

There seems to be more agreement than disagreement among stakeholders on concepts. The focus on the patient was positive. It was positive that ideas put forward involved a multitude of stakeholders, a strong theme of collaboration, sharing of information and greater integration of drugs into the health system.

It is important not to create a large bureaucracy that takes a long time to put in place and ultimately stands in the way of access to important drugs by patients. It is important to remember that Canada is only two percent of the global pharmaceutical market, so there is much data elsewhere. The solution needs to be workable, and collaboration and stakeholder involvement will help to ensure that. Industry would like to be part of the solution being developed.

Dr. Steve Morgan, Research Lead, Pharmaceutical Policy Program, University of British Columbia

The process has been positive and we need to move quickly on the consensus that seems to exist. It is important to push for legislative frameworks that serve the needs of Canadians. Regulation of industry marketing practices is fundamentally important. We need to build on administrative databases and patient registries, and find better ways to use electronic medical records and electronic prescribing. This issue is sufficiently important to merit specific federal funding.

Dr. Jeffrey Tyberg, Chief, Emergency Medicine, Toronto East General Hospital

The recognition of clinical patient safety as a major issue is positive. It is important to recognize the front-end challenge of engaging patients, health care providers, specialty groups, hospitals and clinics: 90 percent of the iceberg is doctors talking to patients. Engaging the tens of thousands of practitioners in this process will not be easy, but it is very important to do so.

Dr. Stephanie Young, Clinical Pharmacy Specialist, Cardiac Care Program, Health Care Corporation of St. John's/Eastern Health

A hybrid model, building on what we can, is most likely to succeed. The system needs to be less fragmented and less complicated for front-line people. Knowledge transfer and dissemination are key and we need to look at best practice models. The role of industry needs to be defined at the micro and macro levels. Resources will be needed: the major barrier to ADR reporting is time. Staff skill development, incentives and professional education are all important. The system needs to be transparent and user-friendly for front-line providers, and they need to have input into the parts of the system that are key for them. Data should be accessible to researchers. All parties need to be open to new models and collaboration.

Discussion

Dr. Forest recapped the main themes: long term/short term; data administration issues; how to provide a voice for patients and practitioners; and models and strategies.

During a question and answer session, there was discussion of funding for the system, and the extent to which funding should be provided by the federal government, provincial governments and/or industry. There was discussion of whether privacy legislation is a barrier to research; differing views were expressed. The issue of regulation of pharmaceutical industry marketing was discussed in more detail, with varying views being expressed. There was discussion of the difference between adverse events and errors, and the need to distinguish between them.

4.3.2 Reflections on Conference Deliberations

Dr. Ed Hunt, Medical Consultant, Newfoundland and Labrador Department of Health and Community Services

Dr. Hunt reviewed what had been presented and discussed at the conference. There was some convergence on possible solutions. There was a message of not reinventing the wheel (build on current capacity, i.e., strengthen regulatory and legislative agendas). Moving from the abstract to the concrete, in terms of governance for non-regulatory agendas, there was support for a hybrid model - with credibility, arms-length but yet responsive to jurisdictions (CCOHTA is a possible model). Linking the solutions analysis more closely to the problem analysis could be effective.

Abby Hoffman, Executive Coordinator - Pharmaceuticals Management Strategies, Health Policy Branch, Health Canada

There is unequivocal consensus on the need for evaluation of real world safety and effectiveness. Developing new capacity, mobilizing existing capacity, and linking decision-makers, users and analytical capacity are important. Safety and effectiveness are related; they share some of the same evidence base, but each has some independent data and evidence streams. Administrative data are important, but we need other evidence (head-to-head trials, cost-effectiveness).

There is a need to ensure capacity to understand influences on behaviour and to develop interventions and methods to change behaviour in desired directions (incentives, policies, methods). Seamless access to, and linking of, evidence across the product life is needed.

Dilemmas remain around issues such as: where does change in regulatory standards (standards for and conditions of licensure, safety, efficacy/effectiveness, clinical trials, post-market requirements etc.) fit?
How should partnership and multiple stakeholder interests be reconciled with leadership issues, and what are the next steps? The NPS provides an opportunity and it is time to move forward.

4.3.3 Closing Remarks and Evaluation

Dr. David Clapin, Associate Director General, Marketed Health Products Directorate, Health Canada, and Susan Paetkau, Director, Drug Programs Branch, Ontario Ministry of Health and Long-term Care

Dr. Clapin thanked participants and noted that the conference had helped the working group for the real world safety and effectiveness initiative understand the complexities of the initiative and the views of stakeholders. The challenge now is to weave together the threads and come up with a way forward. He noted that he would continue to be available for follow-up discussions. Ms. Paetkau thanked participants and noted that the proceedings of the conference would be compiled and disseminated, and the results shared with the Ministerial Task Force.

A total of 45 conference participants completed evaluation forms. There was general agreement (93 percent "strongly" or "somewhat" agreed) that the conference had allowed for an interesting exchange between participants. Fifty-three percent "strongly" or "somewhat" agreed that the process had enabled progress to be made on many of the issues to be resolved with respect to strengthening the evaluation of real world drug safety and effectiveness. The conference was an important first step in a longer process to develop a system for evaluation of real world drug safety and effectiveness.

Appendix 1: Conference Agenda

Agenda
Working Conference on Strengthening the Evaluation of Real World Drug Safety and Effectiveness
September 13 - 15, 2005, Ottawa Marriott

Room locations:

• Plenary sessions will take place in the Victoria North room on Floor 2
• Breakout rooms for group work and group dialogue:
  • Albion Salon - Lower level - (red dots on participant name tags)
  • Alta Vista Salon - Floor 2 - (blue dots on participant name tags)
  • Dalhousie Salon - Floor 3 - (green dots on participant name tags)
  • Mackenzie Salon - Floor 27/28 - September 14 only (yellow dots on participant name tags)
  • O'Connor Salon - Floor 2 - September 15 only (yellow dots on participant name tags)

Tuesday evening, September 13, 2005
7:00 - 9:00pm Informal reception and early registration
Appetizers and cash bar
Sussex Salon
Floor 27/28,
Marriott Hotel

Wednesday, September 14, 2005
Victoria South

7:30 Full breakfast and registration

8:30 Opening remarks from the co-chairs
Victoria North

David Clapin, Associate Director General, Marketed Health Products Directorate, Health Canada, and Susan Paetkau, Director, Drug Programs Branch, Ontario Ministry of Health and Long-term Care

8:35 Overview of the National Pharmaceuticals Strategy
Dr. Penny Ballem, Deputy Minister, British Columbia Ministry of Health, and Ian Shugart, Senior Assistant Deputy Minister, Health Policy Branch, Health Canada

8:55 Explanation of the deliberative dialogue process
Jacquie Dale, Senior Conference Facilitator, One World Inc.

Problem Definition and Analysis

9:05 Case for action: Why is this an issue now? What lessons have we learned so far? Federal, provincial, and international views
Abby Hoffman, Executive Coordinator - Pharmaceuticals Management Strategies, Health Policy Branch, Health Canada; Ed Hunt, Medical Consultant, Newfoundland and Labrador Department of Health and Community Services; Michael Paterson, Senior Research Coordinator, Institute for Clinical Evaluative Sciences; and Richard Hill, Deputy Director, Adverse Drug Reactions Unit, Therapeutic Goods Administration, Australia

10:00 Question Period

10:15 BREAK
Victoria North Foyer
10:30 Results of the pre-conference survey
Louise Travill, Sintera Inc.

10:45 Introduction to the Dialogue Guide and thematic issue areas in real world drug safety and effectiveness
Jacquie Dale, Senior Conference Facilitator, One World Inc.

11:00 Problem Analysis Round 1
Each table of participants considers one of several proposed thematic issues in the evaluation and monitoring of real world drug safety and effectiveness

11:40 Problem Analysis Round 2
Each table considers a second proposed thematic issue

12:20 LUNCH
Victoria South
Over lunch, table hosts meet to synthesize the discussions on each thematic issue. Intent is to find points of convergence and divergence on how the problem is perceived

1:15 Reports on the Problem Analysis Rounds: Is there a common understanding of the problem?
Table hosts for each thematic issue present key findings

Building Common Ground

2:00 Introduction of complementary concepts for strengthening the evaluation of real world drug safety and effectiveness, and explanation of group work
Bruce Carleton, Director, Pharmaceutical Outcomes Program, BC Children's and Women's Health Centre; Supriya Sharma, Senior Policy Advisor, Health Policy Branch, Health Canada; and Jacquie Dale, Senior Conference Facilitator, One World Inc.

2:40 BREAK
Victoria North Foyer

3:00 Group dialogues
(Room locations as assigned - see note on page 1)
Work groups consisting of participants from all stakeholder areas explore the strengths and weaknesses of the complementary concepts

5:00 Adjournment

September 14 Evening Program

Private dinner for conference participants
The Canadian War Museum

Shuttle service will be provided:

6:30 Cash bar opens
The Mess

7:00
The Mess

8:30 Private one-hour tour of the museum

Thursday, September 15, 2005

(Note: Participants with yellow dots on name tags will move to the O'Connor Salon, Floor 2, for group work)

8:00 Full breakfast and late registration
Victoria South

Building A Solution

9:00 Synthesis of Day 1
Victoria North
Key points and findings from Day 1 - conference co-chairs and facilitators

9:30 Group work
(Room locations as assigned - see note on page 1)

10:30 Break
Victoria North Foyer

10:45 Group work continues
(Room locations as assigned - see note on page 1)

11:30 Cross-group work on feature areas
(Room locations assigned by facilitators)

12:15 LUNCH
Victoria South

1:15 Plenary presentations, and real-time measurement
Victoria North
Real-time polling of participants by Sharpe Decisions Inc.

2:15 Break

Moving Forward

2:30 Panel Discussion: Feasibility of Proposed Solutions - A Critical Look
Moderator: Pierre-Gerlier Forest, Chief Scientist, Health Canada
Panelists: Kathy Kovacs Burns, Chair, Best Medicines Coalition; Dr. Rav Kumar, Vice President of Pharmaceutical Research and Development and Regulatory Affairs, GlaxoSmithKline; Dr. Steve Morgan, Research Lead, Pharmaceutical Policy Program, University of British Columbia; Dr. Jeffrey Tyberg, Chief, Emergency Medicine, Toronto East General Hospital; Dr. Stephanie Young, Clinical Pharmacy Specialist, Cardiac Care Program, Health Care Corporation of St. John's/Eastern Health

3:30 Reflections on conference deliberations
Federal and provincial views on areas of convergence and divergence, gaps, and new ideas revealed during the conference
Abby Hoffman, Executive Coordinator - Pharmaceuticals Management Strategies, Health Policy Branch, Health Canada, and Ed Hunt, Medical Consultant, Newfoundland and Labrador Department of Health and Community Services

3:50 Closing remarks
David Clapin, Associate Director General, Marketed Health Products Directorate, Health Canada, and Susan Paetkau, Director, Drug Programs Branch, Ontario Ministry of Health and Long-term Care

4:00 Conference ends