Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

The purpose of this report is to present the findings of the research conducted for the formative evaluation of the Employment Benefits and Support Measures (EBSM).

1.1 Context and Background

The Employment Insurance (EI) Act, or Bill C-12, came into effect on July 1, 1996, replacing the Unemployment Insurance (UI) Act and the National Training Act. The EI legislation represents a restructuring of the old UI system. The new Act provides both income support and active measures designed to assist unemployed Canadians return to work as quickly as possible. Part I of the Act deals with changes to the income benefits and Part II outlines the employment benefits and support measures that are available to clients. Access to personalized interventions is available to individuals who meet selection criteria identified by the local Human Resource Centres of Canada (HRCCs) and are prepared to participate.

To be eligible for employment benefits and support measures, unemployed workers must be receiving EI benefits, have received UI benefits in the last three years, or have received maternity or parental benefits in the past five years. The benefits are targeted to those who need extra assistance and are prepared to make a personal commitment to a back-to-work action plan.

1.2 Main EBSM Evaluated

The five main EBSM on which the formative evaluation focused are described below.

• Targeted Wage Subsidies (TWS): TWS are designed to encourage employers to hire workers they would otherwise not consider. A wage subsidy can be provided to employers who assist clients, including those who face particular disadvantages (e.g., persons with disabilities, visible minorities, and individuals without recent job experience), in finding a job and gaining work experience. The expectation is that employers will keep the employees hired using a TWS once the intervention is over.

• Self-Employment (SE): The objective of the Self-Employment intervention is to help individuals with sound business ideas to start their own business by offering income support, mentoring/coaching and technical help. Clients who receive access to the Self-Employment benefits while they are active insurance benefits claimants continue to receive the benefits for the rest of their claim. Reachback clients (former claimants) receive income support under Part II. Unlike the old Self-Employment Assistance program, the benefits to be paid to clients are not fixed (e.g., assistance is not fixed at 52 weeks).

• Training Purchases and Skills Loans and Grants (SL&G;): Federal government purchases of training (either direct or indirect) is being phased out so that by 1999, the federal government is no longer involved in the direct purchase of training. During this period, with provincial and territorial agreements, HRCCs and partner agencies may continue to purchase training courses for their clients, but the amount purchased will decline each year as the new plan of Skills Loans and Grants is phased in. Because training is a provincial responsibility, skills loans and grants will be offered only in those provinces where there is an agreement with the relevant provincial government. One new important concept that has been integrated into SL&G; is Negotiated Financial Assistance (NFA). NFA is a process whereby HRCCs can negotiate their and their clients' mutual financial participation in paying for training-related costs.

• Job Creation Partnerships (JCP): Special projects may be developed in partnership with HRCCs, provinces, the private sector and community groups. These projects are intended to create incremental and meaningful work opportunities for clients with activities that help develop the community and the local economy. JCP can be used wherever there is a development need and an opportunity to give unemployed workers the chance to gain some work experience that can lead to long term employment. JCP is similar to UI Section 25. However, JCP focuses on skills development as opposed to skills maintenance, which was the purpose of Section 25. Under JCP, there is an expectation that there will be a job available to the clients at the end of their participation. Furthermore, private sector involvement is more desired than it was in the past.

• Employment Assistance Services (EAS): Employment Assistance Services include a variety of services accessible to clients either through a case management process or on a self-serve basis. Clients can be assisted, by being introduced to labour market information tools such as the labour exchange services (the job banks, the Electronic Labour Exchange and CanWorkNet). They can also be assisted to find volunteer work opportunities, participate in job finding clubs, and be referred to one of the Employment Benefits or to any other intervention deemed to be appropriate and needed to meet the action plan. The composition and mix of services provided in any one area will depend on a variety of factors, and could be delivered by the HRCC, or some other organization under a signed agreement with the HRCC.

The evaluation also examined Local Labour Market Partnerships (LLMP) and the Transitional Jobs Fund (TJF), but not in as much detail as the EBSM interventions described above. LLMP is the vehicle through which Regions and local offices can experiment with approaches to improve the functioning of their labour markets and address local labour force priorities. The TJF provides financial support for activities that will promote sustainable economic activity and, in turn, the creation of sustainable employment.

1.3 Evaluation Objectives and Scope

A formative evaluation of EBSM was carried out in mid-1997. A formative evaluation is typically conducted toward the beginning of a program to assess how it is doing and identify areas for modifications early on. The broad objectives of this formative evaluation were the following:

• compare estimates of the two primary results indicators (how many clients are working and unpaid benefits for EI claimants who return to employment as a result of their EBSM intervention) as calculated from the information within the system and available from the survey;

• provide feedback to managers and policy makers on design, delivery and client experiences;

• estimate preliminary impacts attributable to participation in EBSM at the individual and national levels;

• produce reliable information on "what lessons can be drawn"; and

• assess administrative systems used by Human Resources Development Canada (HRDC) in terms of their ability to meet evaluation data requirements.

This formative evaluation focused on design, implementation and delivery issues as well as the early experiences of clients.

The formative evaluation was conducted using the following methodologies:

• Review of documents and administrative data: The overall purpose of the review of documents was to provide a context for the evaluation and further our understanding of EBSM. Throughout the conduct of the evaluation, we reviewed documents that were identified, either by others, or ourselves, as relevant to the conduct of the assignment.

  In addition, the review of administrative data has been used to assess baseline data requirements, determine the validity of the estimates of the key outcome measures and provide some profile information on clients. The systems and administrative data were also reviewed and analyzed in terms of the requirements for the summative evaluation. Detailed information on the results of the systems review is presented in a separate report.

• Interviews with key informants: The interviews with key informants were conducted with representatives who have played an important role in the design, implementation and delivery of interventions. Interviewees included members of EBSM committees, National Headquarters (NHQ) management and staff, regional management and staff, local HRCC management and staff, local level program deliverers and community representatives. A total of 75 interviews were conducted.

• Official Language interviews: The provision of employment-related services to clients in the official language of their choice is an issue of special interest. This issue relates to the availability of, and satisfaction with, information, interventions and case management services in both official languages provided by the HRCC or third-party partners. The communities in which official language interviews were conducted were identified by reviewing a list of communities provided by HRDC which identified the HRCCs required to provide services in both official languages. Interviews were conducted with HRCC staff, community partners and third-party service deliverers. There were also three interviews conducted with representatives from minority language organizations.

• Focus groups with clients: The purpose of the focus groups was to obtain participants' perceptions, knowledge and reaction to the process, design and implementation of EBSM. In addition, the focus groups obtained a qualitative assessment of the impacts of EBSM on individuals. A total of 49 focus groups, representing twelve communities across Canada and two official language minority communities, were conducted. The communities were selected to represent different types of labour markets (e.g., urban, smaller town, seasonal economies) and an adequate pool of potential focus group participants.

• Official Language focus groups: Two communities were selected to represent a minority community in terms of official languages. Sherbrooke, Québec was selected as an English-speaking minority community and Sturgeon Falls, Ontario was selected as a French-speaking minority community. Where these communities did not offer some of the interventions or that there were very few clients who had participated in particular interventions because of limited demand, dyad or triad interviews were conducted.

• Case study research: Maximizing lessons learned was a key consideration in the selection of a diversified sample of cases, which highlighted both successful and failed projects. Information pertaining to potential cases was collected and a final selection of ten cases related to the various EBSM interventions was made. In addition, two cases focusing on the delivery of an EBSM intervention targeting official language issues were also identified. Four core activities were undertaken to research each case study, namely: a review of relevant files and documents (to the extent possible); interviews with responsible HRCC staff (e.g., program officer); interviews with project participants; and interviews with host organizations/providers/ deliverers. The majority of interviews were conducted in person, and site visits were made to third-party deliverers and community organizations, as well as case study participant work sites where possible.

• Surveys of participants and non-participants: The Price Waterhouse National Survey Centre conducted the telephone survey of 3,101 EBSM program participants from July 28 to September 10, 1997. A survey of 4,010 program non-participants was conducted from September 3 to October 5, 1997. For the purposes of this evaluation, non-participants were defined as clients who had not participated, or were not participating, in an EBSM intervention. However, these non-participants may have used self-serve tools such as job banks, Labour Market Information (LMI) and resource centres.

A Methodology Report detailing our approach to the formative evaluation was submitted on May 30, 1997.

It should be mentioned that the formative evaluation was taking place at a time when Labour Market Development Agreements (LMDAs) with the provinces/territories had yet to be implemented. Labour Market Development Agreements are agreements whereby provinces could become responsible for the design and delivery of employment benefits and support measures. Following the signing of the LMDAs, Canada and the provinces will jointly develop an evaluation plan that will determine the need for and timing of formative and summative evaluations in each province.

Generally, the negotiation of these agreements did not have a major impact on the conduct of this formative evaluation, except in New Brunswick where federal staff had been transferred to the province. The consultants were asked to not conduct any interviews with local level staff or focus groups with clients in this province given the sensitivities that surrounded the recent changes.

The summative evaluations on these programs will focus on the longer-term impacts of the interventions. For example, the summative evaluations may address the incremental impacts of EBSM and the achievement of targets in terms of employment and unpaid benefits. The summative evaluations will need to integrate data and information from provinces that will likely have different approaches to the delivery of active measures to EI clients.

1.4 Presentation of Report

The remainder of this report has been organized around the following six chapters:

• Socio-Demographic Profiles (based on administrative data and survey results; Chapter 2);

• Implementation (Chapter 3);

• Impacts and Effects on Individuals (Chapter 4);

• Primary Results Measurement (Chapter 5);

• Impacts on Communities (Chapter 6); and

• Conclusions (Chapter 7).

Each of the chapters devoted to findings presents the main findings related to the evaluation issues examined. The findings on each of the interventions evaluated are presented in appendices to this report, bound separately.