New Substances Program: Notifier Survey

Final Report

April 5, 2004

Introduction

Executive Summary

Detailed Analysis

Profile of Organizations

Contact with New Substances Program

Notifier Satisfaction

Overall Satisfaction

Service Delivery

Information Access

Information Needs

Accessing the New Substances Program

Payment Process

Service Improvements

Survey Methodology

Introduction

The New Substances Notification Regulations (NSNR) were developed under the Canadian Environmental Protection Act, 1999 (CEPA 1999), and are an integral part of the federal government's national pollution prevention strategy. The Regulations are intended to ensure that no new substance is imported into or manufactured in Canada without a formal review of its potential risks to human health and the environment. Environment Canada evaluates the potential risks of new substances to the environment, and Health Canada evaluates potential risks to human health.

In 1999, a multistakeholder consultation process was undertaken with the New Substances Program (NSP) in order to identify possible areas of improvement for the Program. A variety of recommendations were made by stakeholders, including increasing the transparency of the Program and improving the level of service. Environment Canada wished to conduct more in-depth research with its stakeholders about these recommendations, specifically among "notifiers" of new substances and commissioned Decima Research to conduct this survey. The objective of this study is to measure notifiers ¹' levels of satisfaction with the New Substances Program. More specifically, the objectives of this study are to:

• Measure client satisfaction with the service delivery of the New Substances Program;
• Obtain information that can be benchmarked - to the extent possible - using the Common Measurement Tool (CMT); and
• Identify areas of improvement to the New Substances Program.

A total of 99 notifiers completed the survey online between March 1 and 16, 2004, which represents a response rate of 53 percent. A more detailed description of the methodology used to complete this research is presented at the end of this report.

This report begins with an executive summary highlighting the key findings and conclusions, followed by a detailed analysis of the survey findings. Appended to this report are the survey questionnaire, and a set of detailed "banner tables" which presents the results of all questions by key sub-groups. The detailed analysis section of this report denotes these tables by question number (e.g. Q.1) for easy reference.

Executive Summary

Profile of organizations

Most of the participating organizations are located in either Ontario or the United States. They are also most likely to report either a chemical or polymer.

Respondents are more likely to be from larger organizations: those with annual revenues of at least $40 million.

Contact with the NSP

Most organizations have contacted the Program within the past 12 months, and a majority have also made a submission in this period.

A sizeable minority (41%) of respondants have submitted at least 20 substances since the inception of the Program, but a large majority (84%) report submitting ten or fewer substances within the past year.

About two thirds have contacted the NSP within the past year, and have contacted the Program no more than ten times.

A majority of notifiers use the telephone as their primary means of communicating with the Program, but only half would prefer to use this method for future contacts. Others would prefer to use the Internet or e-mail, and they would also like to have the ability to submit their notifications online.

Notifiers initially learned about the Program from a variety of sources from within the industry (such as an industry association or mail out), and look to the web site, colleagues, and other professional development activities for additional information about the Program.

Notifier Satisfaction

Notifiers are fairly satisfied with Environment Canada's delivery of the NSP, and are slightly less satisfied with Health Canada.

Participants give high marks to the NSP staff for a variety of features including their courteousness and respectfulness, but place more importance on the protection of their Confidential Business Information.

Participants identify a need to improve the responsiveness of the scientific assessment staff at both Environment Canada and Health Canada, as well as a need to improve the Infoline.

Notifiers are reasonably satisfied with the accessibility of the Program, particularly in terms of being aware that submissions should be sent through Environment Canada, but place greater importance on improving communication about the NSP.

They identify the web site as an area for improvement, in terms of the ease of navigating and finding information.

Notifiers are also reasonably satisfied that their information needs are being met by the Program, especially in terms of bilingual service. They point to the consistency and clarity of the information and documents as areas for improvement.

Participants are satisfied with the available methods of payment for the Program. They would also prefer to pay for the Program using a credit card.

Service Improvements

The most important service improvements for notifiers are: reducing the waiting time for problem statement letters, and final assessment and acknowledgement letters; and the ability to track an NSN online.

Detailed Analysis

Profile of Organizations

Participants were asked to provide information about their organization, including the province or country in which they are located, the estimated annual revenues of their company, and the type of substances notified.

Half of the participant organizations are located in Ontario, and a majority have annual revenues over $26 million. Participants are most likely to report either a chemical or a polymer.

One-quarter (25%) are in the United States. The remainder are located in either Quebec (15%), western Canada (8%), or the Atlantic region. (Q.35)

Respondents were asked to estimate their annual sales and half (50%) report sales of greater than $40 million. Another one in ten (10%) notifiers sales between $26 and $40 million. The remainder report sales between $13 and $26 million (8%), or below $13 million (21%). One in ten did not report the annual sales of their organization. (Q.33)

Eighty two percent (82%) of the survey respondents notified a chemical and 70% have notified a polymer. Few reported notifying micro-organisms (8%), biochemicals (5%), biopolymers (2%), or organisms (1%). The organizations reporting polymers are most likely to be in Ontario (84%). (Q.1)

Type of Substance Notified to the New Substances Program

Contact with the New Substances Program

Notifiers were asked a series of questions about their level of contact with the Program. These include the number of substances they have reported, as well as the frequency with which they contact the Infoline. Participants were also asked how they first learned about the Program, and from where they have received information.

Most participants have notified a substance within the past year, and a majority have made fewer than ten notifications in the past 12 months.

A majority of notifiers have submitted a new substance notification (NSN) within the past year. Four in ten (40%) have done so within the past three months, and an additional three in ten (29%) have submitted a notification within the past four to twelve months. Just over one quarter (28%) of submissions to the Program were made over one year ago. (Q.2)

To gauge their level of Program use, notifiers were asked to estimate how many substances they have notified since the establishment of the Regulations, as well as the number submitted within the past 12 months. In the past 12 months, 84% notified ten substances or less. By contrast, 47% have reported ten or less since the establishment of the regulations, while 41% have notified more than 20. (Q.3, Q.4)

Number of Notifications to the NSP

Organizations that have submitted a notification within the past three months were more likely to have reported at least 50 substances (83%), and be located in Ontario (55%).

Smaller companies were more likely to have notified fewer substances. Organizations with reported annual sales of less than $13 million were more likely than others to report no more than ten substances.

Organizations in the United States were most likely to have reported only one substance in the past year.

Notifiers were asked when they last contacted the NSN Infoline, and a majority of them have contacted it recently. Four in ten (41%) have contacted the Infoline within the past 3 months, and an additional one-quarter (24%) have made contact within the past four to twelve months. Similar proportions report contacting the Infoline more than one year ago (17%), or that they have not contacted it at all (14%). (Q.5)

When Notifiers Last Contacted the NSN Infoline

Those who have not made a submission to the New Substances Program within the past year were less likely to have contacted the Infoline (29%).

Those who have contacted the Infoline in the past year were asked to estimate how many times they have done so, and a majority of notifiers have made contact ten times or fewer. This is the case for those who have made contact in the past three months (86%) or in the past four to twelve months (77%), while the remainder contacted the Infoline more frequently. (Q.6, Q.7)

Number of Times Contacted the NSN Infoline

A majority of notifiers communicate by telephone for information about the NSP, and about half chose the phone as their preferred method. Nonetheless, more than half would like to submit their notifications online.

Participants were asked which communication tool they currently use for information about the NSP, and a majority (57%) use the telephone. One in five (20%) use the Internet, while fewer use e-mail (9%), posted mail (7%), or fax (2%) for information about the NSP. (Q.10)

Notifiers were also asked what their preferred method of communication would be, and about half (49%) would prefer to use the telephone. Others would prefer to communicate by e-mail (27%) or Internet (22%), and only one percent would choose communication by fax. (Q.11)

Primary and Preferred Communication Tool

Notifiers were also asked how they would like to submit notifications in the future, and a majority would prefer to do so online. Over half (57%) would like to submit their notifications over the Internet using a secure system, while one in three (32%) prefer a paper version. The remainder would prefer to submit notifications by fax (3%), or were unsure (4%) of their preferred method. (Q.12)

Preferred Method for Submitting a New Substances Notification

Notifiers have learned about the Program from a variety of sources, and obtain information from multiple sources. Very few participants have contacted the office for Compliance Promotion.

Participants were asked where they first heard about the New Substances Program, and from which subsequent sources they have received information. Similar percentage of notifiers report initially learning about the Program from an industry association (18%), a mail out (17%), or from a web site (14%). Fewer became aware of the Program from professional development events, such as a workshop (8%), or a presentation or exhibit at a conference (4%). Few said they have learned about the Program through work (4%) or from on-site visits (2%). About one in six (15%) cannot recall where they first learned about the Program. (Q.20)

Notifiers also received other information about the Program from a variety of sources. Over half (52%) have used the website, and slightly fewer have received information through workshops (43%) or colleagues (41%). Participants have received information from industry sector contacts (36%), mail outs (35%), or at information seesions (28%), or at an information sessions (28%) or conference (28%). (Q.23)

Notifiers who have recently made a submission to the Program are more likely to obtain additional information about the Program through professional development activities. These include workshops (70%), colleagues (55%), an information session (53%), or presentations and exhibits at conferences (48%). Furthermore, larger companies (those with annual sales above $40 million) and those who have reported a large number of substances (more than 50) were more likely to use the Internet for additional information.

Few notifiers have contacted a regional office for Compliance Promotion information, with less than one in ten (7%) having done so. (Q.21)

Those who have contacted the Compliance Promotion office (n=7) were asked which office they contacted. Two notifiers reported contacting each of the Atlantic region and British Columbia offices, while one contacted the office in Quebec. Two could not recall which office they had contacted. (Q.22)

All notifiers were asked if the information they received was adequate to determine if they were subject to the New Substances Notification Regulations, and a majority agree it was. Six in ten (59%) agreed that they received complete information, while one in three (35%) said they received partial information (35%). Only one percent of participants said they did not receive enough information to determine whether they were subject to the Regulations. (Q.24)

Adequacy of Information to Determine Requirement of Submitting a Notification

Organizations who have notified a large number of substances (83%), and large organizations (65%) were more likely than others to agree that they received complete information about the Regulations.

Notifier Satisfaction

Overall Satisfaction

Notifiers were asked to rate their level of satisfaction with the overall quality of the client service delivery from Environment Canada's Notification and Technical section, as well as the Technical section at Health Canada. Participants were also asked to rate their overall satisfaction with Environment Canada's Infoline.

Notifiers are somewhat satisfied with the Notification and Technical sections at Environment Canada, and slightly less satisfied with the Technical section at Health Canada.

Notifiers are moderately satisfied with the service delivery of the New Substances Program. They are the most satisfied with the service delivery of Environment Canada's Notification section (3.8), followed by their Technical section, and Infoline (both 3.7). Participants express the least satisfaction with the Technical section at Health Canada (3.5).

Satisfaction with Overall Quality of Service Delivery

Service Delivery

Using questions from the Government of Canada's Common Measurement Tool, notifiers were asked to rate their level of agreement and importance with the service delivery of the New Substances Program.

Notifiers are most positive about the client service staff, and place the greatest importance on the confidentiality of their business information.

Notifiers were asked to give an overall score for their satisfaction with the "customer" services provided by the New Substances Program, and most are satisfied. The mean score given by notifiers is 4.0, and only three percent rated their overall satisfaction a "2" or lower. (Q.9)

Notifiers were asked to rate their agreement with a number of statements about the service they received from the NSP, using a scale from "1" (strongly disagree) to "5" (strongly agree). All statements are such that higher agreement represents higher satisfaction. (Q.8)

As the data in the following table indicate, notifiers are most satisfied with the NSP staff, in terms of their courteousness, respectfulness, knowledge, competency, and their listening skills.

Notifiers are the least satisfied with the scientific assessment staff Health Canada, and their ability to reach an Infoline agent without difficulty.

Participants were also asked to rate the importance of each service delivery attribute using a scale from "1" (not at all important) to "5" (very important). These results are also presented in the table. (Q.8)

All service delivery attributes are deemed important by notifiers, with scores of four and above. In particular, they rate the protection of their confidential business information (4.7) and the knowledge and competence of the NSP staff (4.6) as the most important aspects of the service delivery.

In order to identify any areas of improvement to the service delivery, a "quadrant analysis" was performed whereby the satisfaction and importance scores were plotted as points on a pair of axes. This analysis demonstrates what is important to notifiers, but is not fully meeting their expectations.

In the following grid, the mean satisfaction scores are plotted along the horizontal axis and the mean importance scores are plotted along the vertical axis. Items in the upper right portion of the quadrant represent features of the service delivery that are both important and being met. Those in the lower right portion are services that notifiers are satisfied with, but are also less important. Scores in the lower left portion are services that notifiers deem less important, and also have lower levels of satisfaction. The scores in the upper left hand quadrant are for features that notifiers deem most important, and are less satisfied with. These scores represent gaps in service delivery: attributes that are important, but are not being fully met.

Quadrant Analysis: Importance and Agreement with Features of Service Delivery

The quadrant analysis identifies several potential areas for service delivery improvement, where each attribute received importance scores of at least 4.0, but scored less than 4.0 for satisfaction:

• The responsiveness of the scientific assessment staff at Health Canada;
• The responsiveness of the scientific assessment staff at Environment Canada;
• The ability to get through to an Infoline agent without difficulty;
• The ability to access the Infoline after a reasonable number of contacts; and
• Clearly identifying what to do if there were questions about the Program.

Information Access

Notifiers were asked to rate their overall satisfaction with Environment Canada's and Health Canada's communications, as well as the degree of importance and agreement with the communications of the Program.

Notifiers are positive about the clarity of submitting new substances to Environment Canada. They also place a high degree of importance on the information and usability of the website.

Participants are reasonably satisfied with the overall accessibility of the Program, giving it a mean score of 3.8. Only three participants rate the accessibility of the Program a "2" or lower. (Q.14)

Notifiers were also asked to rate their level of agreement with statements about communicating with the NSP, using a scale from "1" (strongly disagree) to "5" (strongly agree). Higher scores of agreement represent higher satisfaction with the Program. (Q.13)

As the data in the following table indicate, notifiers are most satisfied with the clarity of submitting notifications through Environment Canada (mean score of 4.3).

Notifiers also give positive ratings about the different ways to access the Program, including fax, telephone, Internet, and e-mail (all 3.7).

Participants are the least satisfied with the automated phone system (3.2). Participants were also asked to rate the importance of each characteristic of communications about the NSP, using a scale from "1" (not at all important) to "5" (very important). These results are also presented in the table. (Q.13)

Communication about the NSP is the most important to notifiers. They find it important that the website have the information they require (4.4), that the information is easy to find (4.3), and that the Program be accessible by Internet (4.2). They also agree that it is important to have access to the Program by telephone (4.2).

Notifiers place the least amount of importance on the appearance of the web site (3.1).

In order to identify any aspects of improvement to the accessibility of the information, a "quadrant analysis" was performed whereby the satisfaction and importance scores were plotted as points on a pair of axes. This analysis demonstrates what is important to notifiers, but is not fully meeting their expectations.

Again, items in the upper right portion of the quadrant represent features of communications that are both important and being met. Those in the lower right portion are services that notifiers are satisfied with, but also find less important. Scores in the lower left portion are services that notifiers deem less important, and also have lower levels of satisfaction with. Finally, the attributes in the upper left hand quadrant are those that notifiers deem most important, and also express lower satisfaction with. These scores represent gaps in accessibility: attributes that are important, but are not being fully met.

Quadrant Analysis: Agreement and Importance with Communications

Based on the quadrant analysis, the following areas for improvement to the communications for the NSP can be identified, each receiving importance scores of at least 4.0, and satisfaction scores below 4.0:

• Ensuring the web site has the required information;
• Ensuring the information is easy to find on the web site;
• Making the Program easily accessible over the Internet; and
• Making the Program easily accessible by telephone.

Information Needs

Notifiers were asked if their information needs are being met, and what problems, if any, they experienced while attempting to access the Program.

Participants are most satisfied that the Program is available in both languages, and they view all of the information items important.

Participants were asked to rate their overall satisfaction with both Environment Canada's and Health Canada's communications about the NSP, and they are more satisfied with Environment Canada. Notifiers give Environment Canada a mean score of 3.9 for NSP communications, whereas Health Canada receives a rating of 3.4. (Q.16, Q.17)

Notifiers were also asked to rate their level of agreement with statements about their information needs for the NSP, using a scale from "1" (strongly disagree) to "5" (strongly agree). Higher levels of agreement represent higher levels of satisfaction with the Program. (Q.15)

Notifiers are most satisfied about the bilingual aspect of the Program, giving a mean score of 4.6 for having the option of either English or French for the Program.

Notifiers are also positive about the information they have received to aid them in the notification process. This includes having their questions answered, receiving accurate information, and receiving the information they need about the regulatory requirements, as well as the clarity of the language.

Notifiers are the least satisfied with the information given by the Infoline with respect to clarifying who to speak to at Health Canada about technical questions (3.1).

Participants were also asked to rate the importance of these information items, using a scale from "1" (not at all important) to "5" (very important). These results are also presented in the table.

Notifiers agreed that almost all of their information needs are important, giving importance ratings between 4.4 and 4.6 for almost all of their needs. Slightly less importance is placed on the Infoline's responsibility to inform notifiers about the correct contact person at Health Canada, and about the availability of the Program in both English and French.

In order to identify any aspects of improvement to the information needs, a "quadrant analysis" was performed whereby the satisfaction and importance scores were plotted as points on a pair of axes. This analysis demonstrates what is important to notifiers, but is not fully meeting their expectations.

The scores in the upper left hand quadrant are the ones that notifiers deem most important, but are less satisfied with - these scores represent gaps in information needs: items that are important, but are not being fully met.

Quadrant Analysis: Agreement and Importance of Information Needs

Based on the quadrant analysis, several areas of improvement to the information needs for the NSP can be identified. These include:

• The clarity of the information;
• The clarity of the documents;
• The consistency of the information and advice;
• Documents and other information were easy to understand;
• Being well-informed about the procedures for submitting an NSN;
• The accuracy of the regulatory information requirements; and
• The clarity of the Infoline

Accessing the New Substances Program

A majority of notifiers say they have experienced difficulties accessing the Program, mainly seeking out the correct person to direct their questions.

A majority of participants experienced difficulties while attempting to access the NSP. One-quarter were unsure where to look or who to ask for information (26%), and the same proportion report that the telephone lines were busy (24%). One in ten cite problems with the automatic telephone answering system (13%), receiving conflicting information from different sources (13%), or they were bounced to different people (10%). Others were concerned about the confidentiality of their business information (7%). A few other problems were experienced by fewer participants. One third (33%) have not experienced any problems while attempting to access the Program. (Q.18)

Difficulties Encountered While Attempting to Access the NSP

Organizations in different regions experienced different problems while attempting to access the NSP. Those in Quebec were more likely to be unsure of where to look for information (47%), while those in Ontario were more likely to encounter busy signals (33%). In contrast, those in the United States were more likely to report not having any problems while accessing the NSP (52%). This was also the case for those who have not made a submission within the past year (46%).

Notifiers were given the opportunity to provide additional suggestions to improve the current Guidelines for Notification and Testing of New Substances: Chemicals and Polymers, as well as Organisms, and few took this opportunity to do so. Suggestions for improvements for Chemicals and Polymers include improving the information (15%) and improving the updates (6%). Others suggest easing the regulations (8%) and allowing joint submissions to the Program (6%). Seven in ten (69%) did not provide comments or suggestions for improvements to the Guidelines for the Notification and Testing of New Substances for Chemicals and Polymers. (Q.19a)

Of the few who provided comments for the improvement of the Guidelines for the Notification and Testing of New Substances: Organisms, most parallel the ones made for Chemical and Poylmers. Notifiers suggest improving the information (4%) as well as easing the regulations (2%) for Organisms. Ninety one percent (91%) did not provide any suggestions for improving the Guidelines for the Notifications and Testing of New Substances: Organisms. (Q.19b)

Payment Process

Notifiers were asked about the methods of payment they currently use, and which ones they would prefer to use as payment for the Program.

Half would like to use a credit card to pay the Program fee. Notifiers are satisfied that the payment method is secure.

Notifiers would prefer to use a credit card as a method of payment for the NSP. Half (50%) of the participants indicate they would prefer to use a credit card, whereas just over one in ten (13%) would like to use a certified cheque. Few (3%) indicate that they would prefer a money order and one third (33%) do not have a preferred method of payment. (Q.25)

The desired method of payment differs among notifiers. Participants who have notified more than 50 substances (71%), from larger organizations (61%), and those who have made a submission within the last three months (65%) are more likely than others to prefer to pay for the Program by credit card.

Preferred Method of Payment

Notifiers are somewhat satisfied with the current payment process. When asked to rate their overall level of satisfaction with the payment process, notifiers report a mean score of 3.2. One in six (15%) rate their satisfaction either a "1" or a "2". (Q.27)

More frequent notifiers are less likely to be satisfied with the payment process - those who have reported more than 50 substances give a mean score of 2.8.

Notifiers were asked to rate their level of agreement with statements about the methods of payment for the NSP, using a scale from "1" (strongly disagree) to "5" (strongly agree). Higher levels of agreement represent higher levels of satisfaction with the Program. (Q.27)

The degree of satisfaction for various aspects of the payment process differs. Notifiers are most apt to be satisfied with the security of the method of payment (4.0) and are less satisfied with the ease of determining the fee and convenience of the payment methods. Participants are the least satisfied with the comfort level with providing their credit card information through the mail for a government service. (Q.26)

Notifiers place a high degree of importance on all aspects of the payment process, but particularly on their comfort level of sending their credit card information through the mail to pay for a government service (4.4). Importance is also placed on the security of the payment method and the ease of calculating the applicable fee. Notifiers place the least importance on the convenience of various methods of payment.

Service Improvements

Notifiers place a high degree of importance on the reduction of waiting time for correspondence and decisions from Environment Canada.

Participants were asked to rate the importance of a series of potential service improvements using a scale ranging from "1" (not at all important) to "5" (very important). (Q.32)

Notifiers place the most importance on improvements related to the time associated with the service delivery. These include reducing the waiting time for problem statement letters, for final assessment letters, and for acknowledgement letters.

Importance of Potential Service Delivery Improvements

Others also place importance on reducing the time for an acknowledgement of requests and complaints. They would like Environment Canada to reduce the time taken to acknowledge requests to ten days, and the time to acknowledge and deal with complaints to 14 days. They also place importance on improving the response time of voicemail left on the Infoline to within 24 hours.

Participants also cite enhancements to the website as an important improvement to the service delivery of the NSP. They would like the ability to track the status of an NSN online, and also have the capacity to submit a notification through a secure online system.

Improvements related to the website are moderately important to notifiers. These include improving the ease of finding information on the site and improving both the technical and regulatory information. Others would also like a better search engine and an improvement to responses by e-mail.

Importance of Potential Service Delivery Improvements

Participants also rate the clarity and accuracy of the forms as moderately important. More specifically, they would like to the form to be simplified, and include clearer instructions.

Notifiers also place moderate importance on the information related to the Program, particularly improving the accuracy and consistency of the information provided, as well as making it easier to access.

Notifiers place the least importance on improving the competency and courteousness of the client service staff.

Survey Methodology

Questionnaire Design

The questionnaire for this study was designed by senior Decima consultants in consultation with the project manager from Environment Canada. The questionnaire incorporated items from the Common Measurement Tool (CMT), a standardized set of questions developed by the Government of Canada to measure importance of and satisfaction with the delivery of its services, although many of these were adapted for the New Substances Program (NSp) survey. The questionnaire was translated into French by Decima's in-house translator.

Sample Design

The sample for this study was designed to complete a census of all organizations in Canada and the United States who have submitted a notification to Environment Canada. That is, an effort was made for this study to contact all Canadian and American companies that have submitted a notification.

Environment Canada provided Decima Research with a database of the eligible organizations. Those located outside of Canada and the United States were excluded from the research.

Survey Administration

In order to maximize the limited sample, a three-step approach was employed to recruit respondents:
All potential respondents were initially invited to participate in the study over the telephone and were asked for their e-mail address; respondents were then e-mailed a password and a link to the survey; respondents then completed the survey using an online form. In addition, two reminders were distributed by e-mail to prompt response: The first on March 8 and another on March 15.

Completion Results

A total of 426 valid records were provided by Environment Canada, of which 210 agreed to participate in the study and were e-mailed a password and the link to the survey. Of the 210 distributed, 22 e-mails were returned, reducing the number of potential respondents to 188. A total of 99 surveys were completed, yielding a response rate of 53%.

The table below presents the number of completions by region compared to the number of organizations invited to participate in the study from each region.

Sample Disposition

The final disposition of all contacts to complete this study is presented in the following table.

¹The term "notifiers" and respondent is used interchangebly in this report to specify those that participated in the survey.

²Based on total of numbers attempted (n=426)