Mid-Term Evaluation Study of
Industry Canada’s
Sustainable Development Strategy, 2000-2003

Audit and Evaluation Branch
Industry Canada

July 3, 2003

Executive Summary

Sustainable development (SD) has been integrated into both the Department of Industry Canada Act and the Auditor General Act. Industry Canada’s next strategy must be tabled in Parliament by the end of 2003. This evaluation reviews the mid-term results achieved by SDS II during the period from January 2001 to the present.

This mid-term evaluation is but one of the activities towards developing the department’s next Sustainable Development Strategy (SDS III). This report provides a review of the results achieved to date by the department through the implementation of SDS II, what aspects have changed since SDS I, and the lessons learned from SDS II. This evaluation also identifies implementation questions that need to be addressed for the next phase of sustainable development initiatives by Industry Canada. An update on the accomplishments of the previous SDS I is also included in the study report.

Objectives of SDS II

In its second Sustainable Development Strategy (SDS II), Industry Canada was committed to promoting sustainable development as part of its mandate to create the foundation for a more productive, competitive, knowledge-based economy.

Industry Canada’s SDS II consisted of three objectives that focus on eco-efficiency, environmental technologies, and decision making:

• Eco-efficiency: Enhance the capacity of Canadians, industries and firms to develop and use eco-efficient practices, tools, technologies and products that contribute to increased productivity and environmental performance.



• Environmental technologies: Facilitate the development and diffusion of environmental and enabling technologies that produce long-term economic and environmental benefits.



• Decision-making: Improve the integration of sustainable development objectives into the decision-making and delivery of departmental policies, plans, programs and operations.

Evaluation Questions

The specific focus of the mid-term evaluation was on the following key research questions:

• How relevant are the SDS II objectives and priorities to Industry Canada, and to Industry Canada’s stakeholders’ needs?



• Is SDS II consistent with the department’s mandate? How do the SDS II action items relate to IC’s strategic objectives?



• What is the relevance of IC’s SDS II initiatives for the government-wide sustainable development objectives?



• How successful has SDS II been in achieving its objectives (including productivity through eco-efficiency, environmental technologies, and decision making objectives)?



• Have the intended near-term sustainable development impacts of SDS II been achieved? What were the unintended impacts from SDS II, if any?



• To what extent has SDS II achieved sustainable development results relating to the department’s strategic objectives (i.e., innovation, connectedness, marketplace, investment, and trade)?



• What are the lessons learned, based on factors that might have facilitated and/or impeded the implementation of SDS II, which could be useful to SDS III?

Findings

Update on Achievements of SDS I

• Most officials interviewed from several branches of IC felt that they had achieved the SDS I targets set out for their various SD-related projects. Generally, they also felt that the department was heading in the right direction in terms of integrating SD into operations, and in addressing SD related issues.



• The final status of SDS I action items, is as follows: 12 action items were completed according to plan, 15 were completed with add-ons and/or rolled into SDS II for follow ups, and 1 was discontinued.



• Fostering a marketplace climate — Most interviewees thought that IC made good progress from 1998 to 2000 in addressing its marketplace climate objective — i.e., marketplace rules and services; reasoned advocacy to shape SD policy; and consumer choice and the marketplace. Since 2000 the department is generally seen as heading in the right direction, but may have lagged in the advocacy and consumer elements of this objective.



• Enhancing the ability of Canadian firms to develop and use innovative technologies — This can be described as a forte of Industry Canada. Many SD initiatives that are likely to have a more direct impact on innovative technologies (particularly at the early development stages) are in place. The ability of the department to measure the commercial potential of innovative technologies supported by IC, however, has not progressed sufficiently. It is therefore near impossible to attribute long-term results to IC’s activities contributing to innovative technologies, beyond anecdotal evidence.



• Encouraging trade and investment — The update on this particular objective of SDS I is that Industry Canada has made progress in continuing to support the Canadian government’s efforts to encourage the export of Canadian knowledge, products, practices and technologies that further SD objectives. However, the general view is that this progress is slow, notwithstanding some very successful initiatives, including Sustainable Cities Initiative, Canada’s participation in the World Summit on Sustainable Development, and Trade Team Canada Environment activities.



• Continuing to improve the capacity of Industry Canada — SDS I was seen to be relevant and consistent with the departmental mandate and goals. As such SDS I was able to establish a strong foundation for SDS II, to implement action items focused on entrenching SD within the department. The capacity of IC to manage and deliver departmental policies, programs and operations that contribute to SD, by all measures, seems to have been quite successful. However, where SDS I appears to have fallen short is in fostering a clear image of the long-term outcomes that the strategy is meant to achieve. This same challenge was carried over into SDS II.

Relevance of SDS II

• All Industry Canada’s SD initiatives appear relevant to the department and to government-wide SD goals, as described in A Guide to Green Government and in various Speeches from the Throne (1999, 2001, 2002) and the government’s Budget statements (e.g., most recent Budget 2003), and as expressed in the Leaders Forum on SD in 2000 (e.g., “productivity through eco-efficiency” is a co-lead with Natural Resources Canada and Environment Canada).



• To date, the department continues to make the strongest link with sustainable development through its innovation and marketplace objectives. However, the department’s contribution to SD has become relatively more diversified in scope across the department and across IC’s other strategic objectives, compared to SDS I.



• The Commissioner of the Environment and Sustainable Development, in her 2002 report to the House of Commons, stated that: “The government has yet to provide a clear picture of what a sustainable Canada would look like 20 years from now.” In the absence of such a government-wide vision, it makes it difficult for departments such as Industry Canada to develop long-term SD goals. Nonetheless, IC has identified its long-term, as well as near-term, intended results that are relevant within a broad government framework, and linked these to specific SDS II action items. Notwithstanding, there is a lack of clarity about the path towards achieving the long-term goals. Most of the SDS II action items are actually short or near-term in nature, with a three-year time span, since the Minister of Industry and the department are required to update the strategy every three years. Yet long-term thinking for the next generation of Canadians is the essence of SD.

Results of SDS II: Productivity Through Eco-Efficiency

• Nineteen action items fall under Industry Canada’s productivity through eco-efficiency objective. Two action items are related to “capacity building in R&D and skills”. Twelve action items fall under “applying tools in the marketplace”, while five action items relate to “measuring success”.



• IC has undertaken a broad range of activities in order to address the productivity through eco-efficiency objective. Out of 19 action items in this area, the department has fulfilled or exceeded requirements connected to 13 items, and is making progress towards completing 3. Another 3 action items are reported to be in a planning or early implementation phase and may need to be rolled into SDS III.



• Key mid-term results related to the eco-efficiency objective are listed below. Other results are discussed in the report.
  
  
  • Three new relevant Networks of Centres of Excellence were announced and have been making progress in designing and implementing SD related research programs.
  
  
  
  • A multi-stakeholder steering committee at Canadian Standards Association was mandated to expand the use of environmental standards and eco-efficiency tools to SMEs.
  
  
  
  • Various online self-assessment tools for SD performance were completed and are being used.
  
  
  
  • An online registration tool and database was developed to enable companies to register and update their climate change technology showcasing information.
  
  
  
  • Two reports were completed on environmental information for consumers.
  
  
  
  • A biotechnology web site was set up to promote awareness of applications of biotechnology for SD.
  
  
  
  • Several reports were completed and widely circulated on corporate social responsibility and on corporate sustainability reporting.

Results of SDS II: Environmental Technologies

• Nineteen action items fall under Industry Canada’s environmental technologies objective. Six action items are related to “promoting technology innovation”. Seven action items fall under “encouraging new approaches”, and six action items relate to “working together through partnerships”.



• IC has undertaken a broad range of activities in order to address the environmental technologies objective. Out of 19 action items, the department has fulfilled or exceeded requirements connected to 8 items, and is making progress towards completing 8. Another 3 action items are reported to be in an early implementation phase and may need to be rolled into SDS III.



• Key mid-term results related to the environmental technologies objective are listed below. Other results are discussed in the report.
  
  
  • Technology Partnerships Canada (TPC) has invested considerably in SD oriented projects.
  
  
  
  • IC worked with NRCan and EC to get the Sustainable Development Technology Fund up and running.
  
  
  
  • Industry Canada continues to support the Canada Foundation for Innovation, which also funds SD-related projects.
  
  
  
  • Several technology roadmaps have been completed which have led to collaborative actions by many industry stakeholders.
  
  
  
  • Several reports on international business development competitiveness were completed and posted on the web.
  
  
  
  • Canadian firms are benefiting from the development of international markets from Trade Team Canada Environment.
  
  
  
  • The Sustainable Cities Initiative successfully evolved from a pilot project to a $9 million program covering 17 cities.
  
  
  
  • Three technology roadmaps were launched for climate change.
  
  
  
  • Two studies on fuel cell technology were completed in support of this industry.
  
  
  
  • A vision was advanced for a bioproducts and biobased economy in Canada.
  
  
  
  • Partnerships with leading industry firms and science-based departments and agencies were established for developing an innovation strategy and action plan for bioproducts and bioprocesses.
  
  
  
  • A Canadian environmental solutions web site was launched.

Results of SDS II: Decision Making

• Twenty action items fall under Industry Canada’s “integration of SD into decision making” objective. Ten action items are related to “improving planning practices”. Six action items fall under “enhancing implementation of sustainable development”, while three action items relate to “strengthening consideration of SD in evaluation”.



• IC has undertaken a broad range of activities in order to address the integration of SD into decision making. Out of 20 action items, the department has fulfilled or exceeded requirements connected to 13 items, and is making progress towards completing 5. Another 2 action items are reported to be in a planning or early implementation phase and may need to be rolled into SDS III.



• Key mid-term results related to the decision making objective are listed below. Other results are discussed in the report.



• Elevation in the quality of discussion of SD and environmental impact issues at the Senior Policy Committee of IC.



• Improved Strategic Environmental Assessments (SEAs) were implemented in numerous submissions and Memoranda to Cabinet.



• Project environmental assessments were improved at IC through training, improved networking with other departments, and sharing best practices.



• IC has been proactive in advancing integration of social, economic and environmental elements of SD in several national and international fora.



• SD was successfully integrated into the Report on Plans and Priorities (RPP).



• Three ADM champions were appointed for outreach to industry, greening operations, and SDS implementation and monitoring.



• A robust eco-efficiency web site was launched.



• The department continues to move forward on greening its operations.



• Several training and awareness initiatives on SD were delivered to IC staff.



• SD considerations have been included in RMAFs and in evaluation studies, and an SDS II evaluation framework was completed.

Lessons Learned and Recommendations

The following lessons learned from the SDS II experience can help the department build on and improve the process for SDS III, so that government requirements can be met and SD can continue to become an integral component of departmental culture. Recommendations associated with the lessons learned are also presented.

Making progress — Industry Canada has considerably progressed since SDS I in advancing its sustainable development agenda. SDS II had 58 SD action items, compared to SDS I’s 28. This in itself suggests an increase in SD activity in the department. However, it also means that there is a requirement to consolidate the various initiatives underway around key objectives of the strategy. While SDS II represents progress in establishing a strategic “top-down” view for SD at IC, the process is still seen by some as a fragmented “bottom-up” collection of projects/action items. A balance between the “top-down” and “bottom-up” perspectives would be useful, not only in terms of how these fit together on paper (i.e., in the strategic document itself), but also in the implementation process and in the reporting on results. Recommendation: Industry Canada should consolidate the various action items that emerge for SDS III; into no more than 10 key; outcome areas that are associated with the objectives of the strategy. Implementation and reporting on results should be structured around these key outcome areas. While SDS II; had 9 key; outcome areas, the implementation and reporting structure of the strategy was focused on the 58 action; items, and not around the 9 key; outcome areas.

Evaluation framework — While SDS II; included a set of performance indicators associated with many of the SD action items, the challenge of measuring results of SD initiatives, in relation to the overall long-term objectives of the strategy, is still not sufficiently addressed. It should be noted that all other departments of government similarly face the same measurement challenges. The SDS I; mid-term evaluation study recommended that the department carry out an evaluation framework to inform the process for measuring results – and to help develop evaluation indicators. This was not done until late in the implementation phase of SDS II.; Recommendation: An SDS III; evaluation framework study, consistent with guidelines of Treasury Board Secretariat, should be undertaken concurrently with the planning process for developing the next strategy. This will contribute to addressing the issue of appropriate indicators for near-term and long-term analysis of results.

Decision making — While Industry Canada has successfully integrated SD into its decision making process, a focus on integration continues to be necessary into and throughout the next three-year period. While the quality of discussion and expertise about SD has increased in the department since the first generation of SD strategies, the challenges are ongoing and the need to remain vigilant is still present. Recommendation: Integration of SD in the decision making process remains an important priority for Industry Canada, to maintain a high profile and a focus on this endeavour. SDS III should retain Decision Making as one of its strategic objectives.

Scope and flexibility of the strategy — SDS II;, as a strategic process and implementation framework, did not capture all SD-related work underway in the department. Some SD related initiatives and opportunities emerged after the strategy was implemented (e.g., activities of the Manufacturing Industries Branch regarding “lean manufacturing”). Recommendation: While flexibility was demonstrated in the implementation of SDS II;, in that it was possible to add new action items to the original 57, the department should review the plan on an annual basis and adjust actions and deliverables as required, to meet key outcomes and objectives of the strategy.

Monitoring and reporting — Compared to SDS I;, SDS II monitoring and reporting have solicited praise, and hardly any complaints during the consultation process for this study. However, improvements are needed for capturing changes and additions to original plans and action items, and for tracking outcomes. Recommendation: For SDS III;, individual SD project leaders should consider compiling performance information consistent with the RMAF framework of Treasury Board Secretariat, on an ongoing basis as part of the SD monitoring and reporting system. In this respect, guidance from TBS and/or the office of the Commissioner of the Environment and Sustainable Development would be welcome. Nonetheless, the department needs to develop its own measurement system for SDS III.;

Resources — IC managers and staff consider the lack of funding as a significant constraint to SD implementation, generally resulting in a cautious approach in committing to relevant projects, with some exceptions. Recommendation: The strategic planning process for SDS III; should explore the potential of allocating funds for projects under an SDS III; appropriation framework.

Internal partnerships — The department has become smarter (more effective) in collaborating and partnering with other government departments, provinces, municipalities, and private industry, non-profit organizations and associations – for SD-related activities in general. However, from the consultation process for this study, there are perceived opportunities to improve internal partnerships, within the department, for SD initiatives. The Industry Canada SDS group could encourage internal partnerships with/between branches within the department. These internal partnerships could create synergies in expertise and knowledge and bring about cooperation towards more effective delivery of SD outcomes and objectives. Intradepartmental working groups could be used more effectively to capitalize on these synergies. Recommendation: During the planning process for SDS III, the department should consider how to engender intradepartmental cooperation towards achieving the desired SD outcomes.

Delivery instruments — The department has made good use of the diverse tools available to it to achieve SDS II; results. However, in the next SD strategy, the challenge for IC will be to select the appropriate instruments that best achieve the intended outcomes of the strategy, in a suitable timeframe that is consistent to a government-wide schedule for achieving results. This challenge can be mitigated only to the extent that consensus emerges on a timeframe for results, with an accompanying clarity of vision expressed at a government-wide level, as well as within IC. Recommendation: Regardless of this challenge, it is important for the department to assess and select the most effective tools at its disposal that best achieve intended results, in a timely fashion.

Results — SDS I, SDS II, and SDS III activities are expected to yield societal results in the long-term (e.g., 5-10 years hence, and beyond). However, there is a need to start planning early for a full-scale evaluation (in 2006-2007), to measure the cumulative impacts of SDS I, SDS II and SDS III. Recommendation: To address the requirement of the Commissioner of the Environment and Sustainable Development, for a cumulative review in 2007 of ten years of SD and SDS monitoring work, Industry Canada should prepare to present a comprehensive evaluation of the cumulative results of its SDS strategies.

Parallel strategies — While SDS II has been incorporated in the department’s overall strategic framework (as expressed in Industry Canada’s Making a Difference document), the profile of SD within IC’s Innovation Strategy document – Achieving Excellence – is less evident. The Innovation Strategy perhaps as a necessity seems to exist as a separate framework for action, but does not in itself explicitly link-up with SDS II. Recommendation: To engender a more robust role for SD within the department, it would be useful if the next strategy (SDS III) would be substantially more referenced within parallel strategies of the department — such as the Innovation Strategy.

---

Final Report (PDF - 497KB - 53 pages)

Management Response (PDF - 112KB - 4 pages)

Note: to read the PDF version, you need Adobe Acrobat Reader on your system. If the Adobe download site is not accessible to you, you can download Acrobat Reader from an accessible page. If the accessibility of PDF is a concern, you can have the file converted to HTML or ASCII text by using one of the access services provide by Adobe.