Health Canada Research Ethics Board Researcher Survey 2004 - Summary Report

Prepared by Praxis Research
February 2, 2004

1. Purpose
2. Research Approach
3. Results
4. Summary of results
5. Conclusion

1.0 Purpose

In 2002, Health Canada established an independent research ethics board responsible for reviewing all Health Canada research involving human subjects. As this is a newly established board, the Research Ethics Board (REB) Secretariat is seeking to assess the efficiency and effectiveness of the Research Ethics Board and the approval process. Praxis Research was retained to conduct an independent assessment.

Board members and researchers were surveyed about their experiences with the research ethics approval process. This report outlines the research approach and results for the researchers' survey. A separate report has been prepared which summarizes the results of the Board survey.

2.0 Research Approach

2.1 Survey Design

A survey was designed which asked researchers to report about their experiences in the following areas:

• background information,
• preparing the application - documentation and process,
• preparing the application - the REB Secretariat,
• review by the Research Ethics Board,
• orientation sessions,
• perceived value of ethics review, and
• overall satisfaction with the review process.

The survey was comprised of a combination of closed and open ended questions.

2.2 Sample and Response Rate

The Health Canada REB Secretariat provided Praxis Research with a contact list of 42 researchers. Two researchers were removed from the list as they were duplicate project contacts. The final sample size was 40 researchers who had submitted applications for ethics approval in the past year. Thirty-two of the 40 researchers completed the survey resulting in a response rate of 80%.

2.3 Survey Implementation

The survey was administered online. The Health Canada REB Secretariat provided Praxis Research with a list of email addresses for the researchers. An introductory email was sent to them by the Secretariat followed by an email from Praxis Research which provided a link to the survey and an individual password. The purpose of the password was to ensure the confidentiality of responses and to secure access to the responses.

Participants who were not able to complete the survey at one time were able to reenter the survey using their password and complete it at a later date. The survey was administered online from mid December 2003 until mid January 2004.

3.0 Results

Data were analyzed using SPSS (Statistical Package for the Social Sciences). Frequencies are provided for the closed ended questions. Responses to open ended questions are presented as themes that emerged or lists of suggestions provided by the researchers. The results are presented according to the main sections of the survey.

3.1 Background Information about Participants

Participants were asked to provide the current review status of their project. Seventy-five percent of researchers indicated that approval had been granted, 16% had approval pending, 6% had completed their research and submitted a termination form, and 3% had their research re-approved for an additional year. None of the researchers indicated that they were awaiting annual re-approval.

Thirty-eight percent of the respondents indicated that their initial application was approved as submitted, whereas 62% were approved with conditions.

The main research classifications were "research undertaken in collaboration or partnership with Health Canada" and "research carried out on Health Canada premises".

Research Classification

The researchers provided the following research classifications:

• 44% in collaboration with Health Canada
• 25% on Health Canada premises
• 13% were contract research
• 6% were intramurals
• 6% were funded by Health Canada
• 6% in collaboration with scientist at Canadian universities and partnerships with provincial/territorial/universities and Health Canada.

Awareness of the REB

The majority of researchers (58%) became aware of the REB through communication from senior management or other colleagues, 16% from presentation by Secretariat, 10% by training/orientation session, 7% by broadcast news and 10% by

• previous employment with Health Canada and submissions to the REB,
• Funding requirement, and
• Collaborators.

3.2 Preparing the Application - Documentation and Process

Researchers were asked to rate their satisfaction with the clarity of five aspects of preparing the application. The results are presented below.

Satisfaction with the clarity of

• printed resources, 0.0% very dissatisfied, 3.1% dissatisfied, 28.1% neutral, 65.6% satisfied, and 3.1% very satisfied.
• type of review required, 9.4% very dissatisfied, 12.5% dissatisfied, 12.5% neutral, 56.3% satisfied and 9.4% very satisfied.
• five main components, 0.0% very dissatisfied, 12.5% dissatisfied, 21.9% neutral, 62.5% satisfied and 3.1% very satisfied.
• steps involved in process, 3.1% very dissatisfied, 18.8% dissatisfied, 15.6% neutral, 53.1% satisfied and 9.4% very satisfied.
• required forms to complete, 0.0% very dissatisfied, 18.8% dissatisfied, 31.3% neutral, 50.0% satisfied and 0.0% very satisfied.

Highest satisfaction ratings were for the clarity of printed resources, type of review required and five main components of the application. Clarity about the type of review required had the greatest range of responses. Approximately one fifth of the researchers were dissatisfied with the clarity about steps involved in the process, the type of review required and the required forms to complete.

A series of questions were also asked about the time it took to obtain information and documents from the Secretariat. The results are presented below:

Satisfaction

Satisfaction with the time it took to obtain:

• application forms once requested, 3.3% dissatisfied, 10% neutral, 36.7% satisfied, 50% very satisfied.
• reply to questions about application, 3.4% dissatisfied, 6.9% neutral, 31.0% satisfied, 58.6% very satisfied.
• notification of additional requirements, 4.3% dissatisfied, 8.7% neutral, 39.1% satisfied, 47.8% very satisfied.
• supporting documents requested, 4.3% dissatisfied, 17.4% neutral, 34.8% satisfied, 43.5% very satisfied.

The results presented in the above paragraph show that 78% or more of the researchers were "satisfied" or "very satisfied" with the time it took to obtain application forms once requested, to receive a reply to questions about the application, to obtain notification of additional requirements, and to receive supporting documents. None of the researchers were "very dissatisfied" and only one researcher was "dissatisfied" with the time it took to receive various information and documents from the Secretariat.

3.3 Preparing the Application - The REB Secretariat

The researchers answered extremely favourably to three questions about the REB Secretariat. All of the researchers indicated that the REB contact person was accessible, 94% reported that the contact person was helpful with answering questions, and 96% indicated that the Secretariat accommodated requests for time sensitive reviews.

Researchers were asked to comment about their experiences with the REB Secretariat. Most of the researchers who commented expressed extreme satisfaction with their experience particularly with respect to communication, helpfulness, responsiveness and turn around time. It should be noted that two researchers expressed that there was a lack of communication between 1) the REB and the Secretariat regarding concerns about adhering to the REB process and 2) Health Canada and other agencies regarding the need for an additional review by Health Canada.

3.4 The Research Ethics Board Review

Several questions were asked about presenting in front of the REB. Eighty-one percent found the opportunity to appear in front of the REB in person or via teleconference helpful. Ninety-seven percent indicated that they had adequate time to discuss their application at the meeting.

Half of the participants provided comments about their experiences appearing in front of the REB in person or via teleconference. Many expressed their satisfaction with the experience and provided favourable comments such as "enjoyed the opportunity", "appreciated the questions", "useful to clear up misunderstandings", and "encouraging and constructive comments". Several researchers raised concerns about the presentation process. For example, two researchers commented that the appearance was not necessary given the nature of the projects and one researcher found the concerns raised by the Board to be extremely unrealistic. Three researchers did not find the experience to be positive, commenting that it was an "intimidating setting", "some members of the committee were not very friendly", and there appeared to be "not a lot of respect for social science work" from some Board members.

Researchers were asked to identify how long it took to obtain ethics approval from the REB from the time of the application to the time the decision was communicated by the Board. Twenty-four participants answered this question. The results shown in the following paragraph indicate that the response time varied across researchers with some receiving approval very quickly (ex.: a few days or 1-2 weeks) while others waited over 6 weeks.

Length of time it took to obtain ethics approval:

• 1% said 1 week
• 5% said 1-2 weeks
• 6% said 3-4 weeks
• 3% said 5-6 weeks
• 3% said over 6 weeks
• 5% could not remember
• 1% the review still pending.

While response times varied, almost all of the researchers (97%) indicated that the REB communicated its decision in a clear and timely manner and 89% reported that the REB accommodated time sensitive reviews.

Researchers were asked to rate their satisfaction with the overall timing/length of the review process. The results are presented below:

• very dissatisfied, 6%
• dissatisfied, 6%
• neutral, 19%
• satisfied, 56%
• very satisfied, 13%

Sixty-nine percent of researchers were "satisfied" or "very satisfied" with the timing/length of the review process. This rating is lower than all of the individual measures of timing which were assessed. This finding suggests that while the researchers were very satisfied with the timing of specific stages of the process, they were less satisfied with the time entailed by the overall process (ex.: from start to finish).

Several researchers commented about the need to speed up the process. Comments provided in other sections of the survey also indicated that the process was a) very time consuming when approval had already been granted by another organization and b) that a full review may not be necessary for certain types of research.

3.5 Activities carried out by the Secretariat/Orientation

Only 19% said they had attended an REB orientation session and 22% indicated they had attended a REB Secretariat short presentation. Those who commented, expressed that these sessions were of value.

The following suggestions were provided regarding activities the Secretariat could undertake to assist Health Canada Researchers with research ethics issues:

• Vary the procedure for different types of research projects
• Prepare thorough guidelines about which activities fall under the mandate for submission to the REB
• Prepare guidelines for specific research activities such as age of consent
• Continue to provide opportunities to learn about and discuss different research approaches and related ethical issues
• Provide an advisor to assist with the preparation of applications
• Remove review requirement if already received approval from another organization
• Continue to market the activities of the Secretariat beyond the scientific community (ex.: qualitative research)
• Provide information (ex.: on website) about typical ethical issues, examples of recent reviews and explanations of why they were successful or required revision

3.6 Perceived Value of Ethics Review

The following shows the frequencies for six statements pertaining to the value of the Health Canada research ethics review process,

• is necessary to publish my research, 6.3% strongly disagree, 9.4% disagree, 9.4% neutral, 34.4% agree, and 40.6% strongly agree.
• provides credibility to my research, 9.4% strongly disagree, 0.0% disagree, 18.8% neutral, 40.6% agree, and 31.3% strongly agree.
• provides protection to human subjects, 9.4% strongly disagree, 3.1% disagree, 12.5% neutral, 46.9% agree, and 28.1% strongly agree.
• provided an independent review, 9.4% strongly disagree, 3.1% disagree, 12.5% neutral, 50.0% agree and 25.0% strongly agree.
• provides integrity to my research, 12.5% strongly disagree, 0.0% disagree, 25.0% neutral, 28.1% agree, and 34.4% strongly agree.
• raised my level of awareness about ethical issues, 9.7% strongly disagree, 6.5% disagree, 41.9% neutral, 25.8% agree, and 16.1% strongly agree.

Seventy-five percent of the researchers "agreed" or "strongly agreed" that the approval process is necessary to publish research, provides protection to human subjects and provided an independent review. Agreement ratings were the lowest for the statement, "raised my level of awareness about ethical issues." This variable also had the highest percentage of neutral responses. Researchers who disagreed with these statements, "strongly disagreed" with the statements about providing credibility, integrity, protection to human subjects, and independent review.

Perceived ov erall value

Researchers were also asked to rate the overall value of the research ethics review process on their research. The results range from 1 'no value' to 5 'a great deal of value':

• 13%
• 13%
• 9%
• 47%
• 19%

As indicated above, 66% of researchers indicated that the review process has some or a great deal of perceived value. Twenty-six percent indicated that the process had little or no overall value. Researchers who rated the value as three or less were asked to provide their thoughts about how the value of the review process could be improved.

The main theme that emerged was there is no added value in obtaining a second approval when the research has already been approved by another institution or board. Obtaining an additional approval from Health Canada was described as "frustrating", "inefficient", and simply a "formality".

3.7 Final Thoughts About the HC Research Ethics Review Process

At the end of the survey, researchers were asked to indicate their overall satisfaction with the review process and to comment about opportunities for improvements to the research ethics review process.

Overall satisfaction with the review process is

• 3% very dissatisfied
• 9% dissatisfied
• 22% neutral
• 50% satisfied
• 16% very satisfied

As indicated above, the majority of the researchers (66%) were "satisfied" or "very satisfied" with the review process. This rating is consistent with the results for some individual variables (e.g., clarity of documents, overall length/timing of the review process, overall perceived value variables) but is lower than the ratings provided for other indicators (e.g., experience with the REB Secretariat, time it took to obtain information and documents, some perceived value measures).

Researchers identified the following opportunities for improvements to the research ethics review process:

• Streamline the approval process to avoid duplicate reviews
• Ensure that Health Canada staff are aware of when a review is required
• Limit comments to ethical not scientific issues
• Offer an expedited review process for low risk research
• Enhance communication by providing more information about the role of the REB, the process, what to expect, frequently asked questions etc.

4.0 Summary of Results

In this section, the results of the survey are summarized according to: 1) areas with extremely high satisfaction and agreement ratings, 2) areas with generally high satisfaction and agreement ratings, and 3) areas that may require further discussion or action.

4.1 High Satisfaction and Agreement Ratings

Seventy percent or more of the researchers were satisfied/very satisfied or agreed/strongly agreed with the following areas:

• the timing of stages involved in the process (ex.: to obtain a reply to questions, application forms, notification of additional requirements, supporting documents; time to present application at the Board meeting; and timely communication of review decision)
• accommodating time sensitive reviews
• service from the Secretariat (ex.: accessible, helpful, responsive and efficient)
• interaction with the Board (ex.: opportunity and experience of appearing in front of the Board, clear communication of review decision)
• the perceived value of the approval process in terms of: necessary to publish, protection to subjects, providing an independent review, and providing credibility

4.2 Generally Satisfied or in Agreement

Between 50% and 69% of the researchers were satisfied/very satisfied or agreed/strongly agreed with the following areas:

• the clarity of printed resources, type of review required, five main components of the application, steps involved in the process and required forms to complete
• the approval process providing integrity to research
• overall perceived value of the review process
• overall satisfaction with the review process
• overall timing/length of the review process

4.3 Opportunities for Further Discussion or Action

Comments and dissatisfaction/disagreement ratings revealed the following opportunities for further discussion or action:

• enhance the clarity of steps involved in the process, the type of review required and the required forms to complete
• examine whether appearing in front of the Board should be required for all projects
• assess the unique value of receiving Health Canada REB approval and in particular, the value of having an additional review for projects that have already received approval from a partner agency/organization
• explore ways of streamlining or speeding up the overall timing of the process
• (ex.: expedited review, different reviews for different types of research projects)
• monitor the Board's style of interaction with presenters
• determine ways to increase attendance levels for orientations and short presentations
• review suggestions for further activities and areas for improvement identified by the researchers

5.0 Conclusion

The purpose of this research was to assess the efficiency and effectiveness of Health Canada's research ethics review process. The results demonstrate that the review process is particularly efficient and effective in the areas of 1) the time it takes to receive information, documents and approval notification; 2) services provided by the REB Secretariat to researchers; 3) communication between the REB and researchers; and 4) adding specific value to the research project.

While the majority of researchers were satisfied/in agreement, ratings indicate that the effectiveness and efficiency of the process may be further enhanced by 1) providing additional clarification about documents and the process; 2) enhancing perceptions of the overall value of receiving Health Canada REB approval, particularly if it is an additional approval; and 3) exploring ways to speed up or streamline the approval process.

Further opportunities for discussion or action include 1) increasing attendance figures for the orientation sessions and short presentations by the REB Secretariat, 2) exploring researchers' suggestions for activities that could be undertaken by the Secretariat to assist Health Canada researchers with research ethics issues, and 3) reviewing researchers' suggestions for improvements to the ethics review process.