Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

Formative Evaluation of the Sectoral Partnership Initiative

Purpose

The evaluation of the Sectoral Partnership Initiative (SPI) is being conducted as a multi-year evaluation process. There are three consecutive stages to this evaluation: 1) formative; 2) ongoing monitoring and data collection; and summative.

Since most of the sector councils were in the early stages of their operational phase, a formative evaluation was initiated. There were two main objectives for the formative stage of the SPI evaluation. The first was to obtain information which could address various evaluation issues, dealing primarily with rationale and delivery issues. The second was to refine the evaluation issues and to develop a methodology for the later stages of the SPI evaluation. The methodological issues for the later stages of the evaluation are dealt with in a separate document and will not be discussed here.

Background

The five-year (1993/94 to 1997/98) SPI, announced by the Minister of Finance in the Fall 1992 Economic Statement, is a program designed to facilitate the development of a training culture and increase private sector investment in training, in which the federal government will assist the private sector in developing sectoral partnerships for coordinating human resource management. The SPI now consolidates four functions within one Directorate of Human Resources Development Canada (HRDC), that is the Human Resources Partnerships Directorate: Sectoral Partnerships Delivery Division; Standards, Planning and Analysis Division; Sector Studies Division; and Occupational and Career Development Division. The purpose of the SPI is to challenge industry to work together in a new human resource planning system that will lead to the development of human resources strategies. The SPI is only involved in upside adjustment situations.

A number of important objectives have been set for the Sectoral Partnership Initiative. SPI is designed to encourage the development of new self-sufficient partnerships (sector councils) devoted to long-term human resource planning and development. The key partners in this process include business, labour, government and educational bodies. The SPI, through the sector councils, intends to make important achievements in the human resources area, including:

• building a training culture within individual firms;
• raising awareness of human resource development issues;
• mobilizing workers and employers to take responsibility for training;
• developing and implementing national occupational and skill set standards; and
• leveraging greater training effort by the private sector.

Each sector council is not expected to meet all of SPI's objectives. In addition, the Initiative is evolving and these original objectives are currently being supplemented, with greater emphasis now being given to creating systemic changes in human resources (HR) practices (e.g., establishing national industry training standards and programs with educational institutions).

When a sector becomes a participant in the SPI, the first step in the development of a sectoral agreement is generally a sector study. Following the sector study, the sector may enter into a developmental phase which lasts up to one year, during which a business plan for the sector council is prepared. If the business plan is approved by HRDC, the council is instituted and enters a three year operational phase.

During the three years of the operational phase, the council receives funding on a declining cost-shared basis from HRDC through an operational contribution agreement. The funds provided are intended to cover the administrative costs of running the council. At the end of the three years of the operational phase, the council is expected to be financially self-sufficient in terms of its core operational costs.

In addition to receiving funds to defray the costs of instituting and operating the council, many sectors enter into additional agreements with HRDC to fund specific human resource development initiatives. These agreements include Skills Upgrading projects, Occupational/Skill Standards projects and Youth Initiatives.

As of the fall of 1995, there were 18 operational sector councils and three sector councils at the developmental stage. An important feature of sector council delivery is the joint labour-management design and delivery of programs and services. The programs and services offered by the sector councils represent a continuum of occupational standards development and training implementation.

The diversity in the sector councils is reflected in the nature of their programs and services, organizational structures (e.g., regional organization versus centralized structures), the partnerships developed, and the types of workers served. Some sector councils have focused on the needs of large and small companies while others have focused on the needs of small companies or large companies only. This is sometimes dictated by the nature of the industry. Some industries are comprised almost exclusively of large companies while others are characterized by smaller firms.

There are differences in the types of workers served among the sector councils. While most sector councils focus on the broader workforce within their sector, some sector councils target specific occupational groups. These sector councils are typically cross-sectoral councils, structured around a specific occupation or type of employment, or even a particular segment of the labour market.

The original budget for the SPI, fiscal years 1993/94 to 1997/98, was $250 million, however, after various budget reductions the five-year budget as of November 14, 1995 stood at $135 million (O&M and contribution program dollars combined). Total expenditures (contribution dollars) under SPI were approximately $42.7 million at that time.

Methodology

This formative evaluation was largely based on a case study approach, each of eleven selected sector councils serving as a single case. However, other lines of evidence were also used, including:

• key informant interviews;
• document and file reviews;
• discussion groups with council staff and members of the board;
• surveys and focus groups with employers and employees; and
• analysis of administrative data.

Problems encountered by one sector council may not have been encountered by others, and rarely by all sector councils, making generalizations difficult. The emphasis in this report is on learning from these observations rather than a quantitative counting of problems or benefits.

It is also important to recognize that the sector councils are in different stages of their operational phase. Some sector councils are still in the process of developing their products and services, others are just beginning the 'marketing' phase to encourage firms and workers to participate, while others have had participants in their programs and services for more than a year. The different stages of implementation for the sector councils profoundly affect the results that could be observed during the formative stage of the evaluation. It should also be noted that the case studies reflect the situation for a sector council at the time the case study was completed. Since the case studies were conducted at various points in time during the formative evaluation, the information may not be completely up-to-date in every case.

Key Findings and Conclusions

Rationale

Relevance to Sectoral Needs

The rationale for the SPI is still valid and relevant to sectoral HR issues. The flexible, national, labour-management partnership approach is considered effective by both the private and public sector. The joint labour-management approach is also supported by existing research on HR practices. The flexible approach, which allows each sector to define its HR needs and the appropriate actions to be taken, implies that not all of SPI's objectives or priorities may be addressed or even relevant in each sector council. Overall, however, there is a good match between the sector council needs and government priorities for human resources development practices.

Adapting To Change

Generally, the sector councils are very willing to respond to new government initiatives. Not all sector councils have been able to respond to changing government priorities and the new initiatives launched under the SPI. Introducing new initiatives can interfere with meeting original plans and timetables of sector councils. The sector councils emphasised the need for the government to be flexible, allowing each sector council to respond to new government priorities according to each sector's needs and resources. SPI management stated that the intention of SPI is only to pursue new initiatives when there is a match between the sector council's needs and SPI.

Match Between SPI and Sector Council Objectives

There appears to be a good match between the objectives of the sectoral agreements and the SPI. Except for the process and delivery results, however, the linkages between the goals and objectives of SPI and the sector councils are not specified in the agreements. There should be a statement of intent regarding employability and firm level impacts in the agreements to clarify expectations for meeting SPI's broader objectives.

Public Support

SPI resonates well with the public agenda of a non-politicized approach to human resource programs and practices. Surveys of the general public reveal that public support for lifelong learning and the partnership approach is high. The public are less concerned about who is responsible for training and more concerned about getting something done.

Delivery Results

The SPI Process

There is some frustration with the administration of the SPI, however, sector councils' opinions about SPI management and staff are very positive. SPI representatives were viewed as very efficient, professional and helpful. Sector councils would like to see even more services provided by SPI representatives, including information on lessons learned and a simplified methodology for updating HR studies.

From the standpoint of sector councils, there are a number of possible changes in the administration of the SPI that would either benefit the sector councils or improve the process, including:

• providing a clear understanding of SPI policies and procedures, particularly at the developmental stage
• providing a clear statement of SPI objectives and expectations for each sector council in the agreements
• providing a precise statement of the self-sufficiency requirement in the sector council agreements
• in some instances, developing a transition period between the HR study and the developmental agreement to formalize partnerships, prepare potential strategies etc., early in the process; an
• developing a better match between the financial cash flow and expenditures of the sector councils and the (currently declining) payment schedule, although core self-sufficiency must be attained.

SPI management noted that the lack of guidelines and definitions stemmed from the fact that the program itself was evolving and it was a learning situation for the sector councils and program management. SPI management also intends to keep SPI flexible, even if it means some degree of ambiguity persists.

Design and Delivery

There is a wide diversity and innovation of programs and delivery mechanisms as a result of the sectoral partnership approach. It is unlikely that a single traditional government program could have yielded such diversity and innovation. This diversity reflects the different perceived needs within each sector. The diversity will provide an opportunity to explore how different types of programs and services meet the needs of labour and management in a variety of labour markets. This diversity could also lead to duplication of effort and it is important for the sector councils to maintain their lines of communication and for SPI to provide information on the lessons learned.

One of the most important characteristics of the delivery is the joint labour-management design and delivery of the programs and services. The joint labour-management process has worked exceptionally well in many sectors considering the different objectives of these stakeholders and the often long history of an adversarial relationship in their labour-management negotiations. In some cases, however, differences between labour and management continue to hamper progress.

Significant progress was found in product development and firm-level take-up. These results vary substantially by sector councils, although progress consistently lags behind original plans. The sectoral partnership approach is a complex process and should not be viewed as a quick fix.

One area of improvement could be in the marketing efforts for the sector councils' products and services. Sometimes so much effort is required to develop the products and services that, excluding the initial needs assessment, not much effort is expended on developing marketing plans and strategies until after the products are near completion. Leaving a large gap in the marketing activities, between the time the needs assessment is completed and the product development, can create another lag in the take-up of programs and services in the sector. HRDC is currently exploring HRDC's and sectors' needs regarding communications strategies.

Equity Issues

Few sector councils have implemented any employment equity initiatives despite the requirement with the sector council agreements to address employment equity issues. Although more equity initiatives are also being developed in conjunction with the Youth Initiatives, more resources should be devoted to identifying opportunities to incorporate employment equity initiatives during program design. In order to increase employment equity activity undertaken by the national sectoral organizations, an employment equity committee has been formed within the Human Resources Partnerships Directorate at HRDC.

Systemic Change

There is evidence of systemic change in HR practices as a result of the SPI initiative. The joint participation of labour and management in the sectoral initiatives is a profound shift in the way in which HR practices have been developed and implemented in the past. In some cases, educational institutions have become more responsive to the needs of the sectors, changing long standing approaches to the design of courses and the awarding of credits. Many of the stakeholders believe that jurisdictional boundaries are being broken down because programs and services delivered by sector councils are industry driven and are not designed by any level of government.

Self-Sufficiency

There is a strong rationale for promoting self-sufficiency among the sector councils. Some sector councils have achieved core operational self-sufficiency and the evidence suggests that most sector councils could achieve core operational self-sufficiency, although most will not meet their original cash flow projections. Several factors contribute to the deviations from the original cash flow projections for the sector councils, including delays in implementing programs and services and responding to new government initiatives. The ability to design and implement products and services and the ability to become self-sufficient are two different measures of success. If a sector council has had difficulty designing and implementing programs and services, this is not a self-sufficiency issue, it is a delivery or relevance issue.