Evaluation of the Institute for Chemical Process and Environmental Technology (ICPET)

Report Summary

The evaluation of the Institute for Chemical Process and Environmental Technology (ICPET) of the National Research Council (NRC) was carried out over the period September, 2002, through May, 2003, by the Policy, Planning and Assessment Directorate (PP&A) of NRC Corporate Services. It was conducted at the request of NRC's Senior Executive Committee, as well as in accordance with NRC's evaluation cycle and Treasury Board Secretariat policies. The study was overseen by an Evaluation Advisory Committee consisting of representatives from PP&A, NRC senior management, industry, and Industry Canada's Audit and Evaluation Branch. ICPET also held ex-officio membership.

The overall "bottom line" conclusion from the evaluation is as follows:

ICPET's activities are rated as excellent:

• the Institute's research has a high degree of scientific merit;
  
  
• the Institute's research is relevant to the needs of industry; and
  
  
• the work of the Institute has resulted – or is expected to result in the future – in economic benefits.

The evaluation identified opportunities for ICPET to increase its visibility and impact, both within NRC and with the external sectors it serves. In order to do this:

• the Institute should consider narrowing its research focus (and this has organizational and operational implications that may require alternative resource allocation strategies); and
  
  
• the Institute should pursue ongoing and increased outreach and linkage activities with industry and universities.

The specific issues that were addressed in the evaluation and the main findings related to each of these are summarized below.

Relevance – the extent to which ICPET's activities and competencies have been aligned with the needs of the chemical process intensive manufacturing sector, as well as government policies in the areas of innovation and sustainable development.

Findings:

• ICPET has the capabilities and expertise to address many of the current and evolving priority research needs of industry.
  
  
• ICPET's competencies and activities have been highly focused on meeting industry needs.
  
  
• ICPET has made many contributions to environmental protection. However, the Institute's Sustainable Technology Office (in particular its liaison function) has not been highly relevant to date.

Success – the extent to which ICPET has contributed to increased industrial competitiveness and to the strengthening of the Canadian innovation system in manufacturing.

Findings:

• For most of ICPET's clients, the services they have received from ICPET have contributed, or are expected to contribute in the future, to tangible benefits such as the development of new or improved products and the development of new or improved manufacturing processes.
  
  
• Almost half of these clients have already experienced economic benefits. Reduced costs of the organization's R&D is the major category of economic benefits to date. However, the evaluation also documented some significant examples of reduced manufacturing costs and increased revenues.
  
  
• ICPET has consistently been one of NRC's top performing NRC institutes with regard to amount of licensing revenue.
  
  
• In the areas of training of highly qualified personnel and strengthening of the fuel cells innovation system, ICPET has played a significant role in support of the innovation system. However, there remain opportunities for ICPET to increase its involvement in other areas of innovation support including outreach to industry, information dissemination to industry (other than direct clients), and development of industry linkages (including incubation) and partnerships.

Scientific Excellence – the extent to which ICPET's research is leading edge and has high scientific merit.

Findings:

• ICPET has a high level of scientific output (when examined in the context of other NRC institutes), as measured by the number of publications and technical reports.
  
  
• For the Institute overall, the degree of scientific excellence of the research carried out is high, with opportunities among some groups to further strengthen the quality of the research carried out.

Organizational Responsiveness – the extent to which ICPET has been able to effectively reallocate resources and competencies, reorient its business, and restructure itself to deal with changing industry and government/NRC priorities.

Findings:

• There has been much management activity over ICPET's history to organize and reorganize the Institute in response to industry needs, and this activity has been successful in maintaining the relevance and impact of the research program.
  
  
• There are further organizational and operational measures that could be taken that would enable ICPET to achieve greater visibility and impact.
  
  
• Despite the existence of a management strategy that supports the identification of new invention for protection and exploitation, ICPET has not identified and developed significant opportunities to support the formation of new companies (spin-offs or spin-ins). The Institute's IP and commercialization strategy is based on technology transfer to existing companies.
  
  
• The Sustainable Technology Office, in its liaison function, has lacked sufficient direction and focus, and has experienced limited resources and critical mass.

Future Directions – opportunities for ICPET to better meet the evolving needs of the manufacturing sector, as well as to support evolving government and NRC policies in the areas of innovation and sustainable development.

Findings:

• The main avenues for improvement are organizational and operational changes – in particular, a clearer Institute vision and a narrower research focus.
  
  
• There needs to be a higher degree of integration of the research activities within the Institute.
  
  
• There are opportunities to increase emphasis on outreach activities and relationship building – both with industry and industry networks, as well as with universities.

The evaluation study contains seven recommendations designed to enable the Institute to sustain and improve its performance. A summary of the recommendations, and the Institutes' response to these, are as follows:

1. (a). Recommendation: In its current strategic planning exercise ICPET should attempt to develop a clearer vision for the Institute's research program, one that has a narrower and more manageable focus than the current program.
   
   Response: Through its strategic planning, the Institute will create a clear vision in conjunction with goals and strategies tied to a performance framework.
   
   

1. (b). Recommendation: ICPET should seek ways to increase the degree of integration of the research activities within the Institute.
   
   Response: Integration will be accomplished in part as research activities are developed around selected research themes identified in the strategic plan.
   
   

2. Recommendation: ICPET should seek ways to increase interactions and build stronger ties with industry.
   
   Response: Industry awareness of ICPET will be increased through several mechanisms.
   
   
3. Recommendation: ICPET should increase its interactions and collaborations with universities.
   
   Response: ICPET will continue to increase its linkages with universities, as identified in last year's Planning Outlook.
   
   
4. Recommendation: ICPET should place emphasis on ensuring that it maintains an appropriately balanced research portfolio between long-term strategic research, near-term collaborative research, and applied research.
   
   Response: The Institute's intent in its planning to focus on specific themes, with applications arising from the themes, will provide a mechanism for effectively managing the research portfolio.
   
   
5. Recommendation: ICPET should seek ways to increase its visibility and raise its profile, both outside of and within NRC.
   
   Response: As described in the response to Recommendation 1(a), improving the Institute's focus will facilitate its efforts to increase its profile and recognition.
   
   
6. Recommendation: In considering appropriate mechanisms for transferring and commercializing its technology, the Institute should try to achieve a better balance between traditional licensing and the formation of spin-off or spin-in enterprises.
   
   Response: In the strategic planning done thus far, ICPET has identified "building commercialization capacity" as a potential strategic theme to be pursued.
   
   
7. Recommendation: The Sustainable Technology Office should concentrate in the future on developing its capabilities in sustainability analysis. In addition, a review of the STO's wider functions should be undertaken.
   
   Response: Sustainable technologies and systems are expected to be central to the Institute's future, and the development of tools to assess sustainability aspects of technologies will be factored into ICPET's strategic planning process. A wider STO role, such as facilitation, support, and promotion of sustainable technologies beyond ICPET will require a clear NRC mandate and resource framework.