Evaluation of Employment Benefits and Support Measures in Ontario - September 2000

Evaluation of Employment Benefits and Support Measures in Ontario

View Whole Report

• Labour Market Development Agreements
• Background
• Key Findings
• Lessons Learned
• Conclusion

Evaluation of Employment Benefits and Support Measures in Ontario

During 1998 and 1999, a formative evaluation was conducted by Human Resources Development Canada (HRDC) of its Employment Benefits and Support Measures (EBSM) provided in the Province of Ontario. EBSMs are funded under Part II of the Employment Insurance (EI) Act. This formative evaluation is an element of HRDC's broader EI Monitoring and Assessment process that is intended to provide ongoing information to Parliament and HRDC management as to the progress of services provided and their impacts. The focus of the formative evaluation was to review issues associated with delivery and early short-term impacts and outcomes two years after the introduction of the EBSMs in Ontario.

The Employment Benefits and Support Measures (EBSMs) are HRDC's primary labour market developmental initiatives. They comprise a range of programs and activities. The objectives of EBSMs are to assist clients to find and maintain employment and to reduce their dependence on insurance benefits and other income transfers. The term found work and maintain employment also encompasses self-employment. There are two types of unemployed clients that are eligible for the full range of EBSMs: 1. active EI claimants, i.e., those who are currently receiving EI benefits; 2. previous EI claimants (referred to as reachback clients). Reachbacks comprise two groups of recent EI claimants, i.e., an individual who received EI benefits within the last three years but who is currently unemployed and not in receipt of EI, or an individual who received maternity/paternity benefits within the last five years and is now seeking to re-enter the workforce. The following five EBSMs were evaluated: Targeted Wage Subsidy, Job Creation Partnerships, Training, Self-Employment Assistance and Employment Assistance Services.

This brief presents highlights from the formative evaluation report. In addition, a set of lessons learned is also presented. A more complete analysis is presented in the EBSM Evaluation Overview Report. The Overview Report also contains HRDC's Management Response to the evaluation's key findings. The last page of this Brief explains how to obtain a copy of the Overview Report.

The formative evaluation examined a number issues that can be organized into 4 broad categories: Participant profile, Consistency of EBSMs (administrative issues), Intervention and Service Activity, and Client Outcomes. The key evaluation findings are presented below.

Participant Profile

1. The take-up of EBSM spans the key demographic groups, although visible minority and disabled clients are under represented compared to their share of the unemployed. The demographic profile of clients varied between the EBSM components. The average proportion of active EI claimants and Reachback was 80/20 percent. It was found that there is an absence of data on the Reachback population to gauge the use of EBSMs by this client group.

2. There was measured progress in EI claimants accessing Employment Benefits (Targeted Wage Subsidy [TWS], Self-Employment Assistance [SEA], Job Creation Partnerships [JCP], and Training). For active EI clients, over the period July 1996 to March 1998, there was a decline in the lapsed time between the establishment of the EI benefit period and commencement of an EBSM intervention.

EBSM Consistency

3. HRDC Management and staff perceive that planning and community consultations are a growing factor in EBSM delivery. There is, however, a wide variation in approach to and quality of business planning between HRCCs that may limit effective programming. There is an indication that clients in HRCCs with better quality business plans, experience a higher rate of post-EBSM employment.

4. The focus on individual responsibility is consistent with client's view of the relevance of EBSMs. The majority of clients understand and accept the rationale of individual responsibility. In addition, clients believe that developing an action plan or back to work strategy contributes to better decision-making and more relevant interventions.

5. In general, monitoring and accountability activities are viewed by HRDC staff as being ineffective. HRDC staff note the lack of information to clearly measure results, plan client activity and gauge third party effectiveness. As well, the availability of participant and cost data places limitations on the calculation of reliable cost efficiency/ effectiveness measures and thereby, its application as a key local decision-making tool.

Intervention and Service Activity

6. There is a high level of satisfaction with both staff assistance received and program content. This is highest for those participating in Training and SEA EBSMs. Clients however felt insufficient information was provided prior to selecting their intervention. As well, both clients and staff are concerned that some clients, particularly older and less experienced clients not as familiar with HRCCs, can effectively use the self-serve resources.

7. Clients participating in more time intensive Employment Benefits (TWS, SEA, Training, JCP) components of EBSMs also use self-serve services to a high degree. This points to the integrated nature of these Employment Support Measures with other HRDC employment services and that these various activities cannot be viewed in isolation. During the period of review, approximately 70 percent of EI claimants used some form of EBSM or self-serve service.

8. A vast majority of clients who used 3rd party services [group/individual counselling, Employment Resource Centres (ERCs) and Job Finding Clubs (JFC)] were aware of the federal role in funding their EBSMs.

9. Approximately 90 percent of Francophones were able to access services, in designated language areas, in their language of preference.

Client Outcomes

10. Initial results indicate that EBSMs have a positive employment impact, in both gross and incremental terms. Incremental employment results were estimated for EI claimants only, not reachback clients. All EBSMs contributed to incremental employment, although this varied with the type of EBSM and client group. There was less impact for older clients, members of visible minority groups and those with less recent employment attachment. The evaluation's incrementality analysis and client feedback points to lower impact of short-term group sessions compared to other initiatives.

11. The evaluation confirmed the employment status reported by HRDC's accountability "found work indicator". However, the absence of detailed data collected on clients, mix of services received and the resulting employment characteristics, restricts interpretation of this measure and comparability across areas.

Observations and conclusions based on results from the formative evaluation of the Ontario Region EBSMs have been used to develop lessons learned to support refinements of the EBSMs and to share experiences with interested parties both within and outside HRDC.

1. Program Environment. Duplication or overlap among services was viewed as occurring only occasionally in Ontario. Community development and labour market research activities, however, were identified as potential areas of duplication. Developing more coherent and cohesive delivery and dissemination strategies for information and services has been facilitated in sites featuring co-locations, communication between departments and community consultations.

2. Planning. Business plans vary significantly across HRCCs, as do approaches to developing them. Planning decisions are strongest when they are based on a combination of factors. Typical factors that have been used include past expenditure patterns, client volumes, and availability of HRCC resources and expertise. Additional approaches that have proved useful include; use of locally based labour market information (LMI) and community consultations.

3. Information and Referral. The client qualitative and quantitative research indicated the importance of maximizing client information on the program options available to them and ensuring connection with the appropriate resources as rapidly as possible. Many clients also desired greater personalized attention. Sites have responded to this in a variety of ways, for example: having a "meet and greeter" at the front-end operation and providing structured information and referral through the Employment Resource Centres.

4. Delivery. One of the key shifts in the delivery of EBSMs has been toward third party delivery (with third parties taking responsibility for services ranging from case management to Employment Resource Centres [ERCs]) to program delivery). In terms of dealing effectively with third parties, factors of success include: open and transparent communication; ongoing upgrading of third party deliverers; and ensuring sufficient knowledge of HRDC, in particular, for third party deliverers who are located off-site.

5. Employment Resource Centres. Employment Resource Centres (ERCs) are important complements to employment benefits. The benefits of an ERC are greatest when it is accessible in a location that is highly visible and convenient for clients. While having the resource centre within the HRCC office is ideal, situations where this is not feasible should be dealt with through an effective communications strategy to ensure client awareness of and access to the ERC. Accessibility of resources (e.g., computers, workshops) and accessibility to the office itself (e.g., hours of operation) should be reviewed on an ongoing basis.

6. Developing Partnerships. Successful partnerships -involving HRCCs and community stakeholders were generally associated with a prior history of working collaboratively, commitment of the individuals directly involved and investments of time and resources. Developing employer "partners" was facilitated through developing a promotional strategy and materials directed to local employers. Having clients approach employers themselves for the Targeted Wage Subsidy (TWS) program can also be an effective way of increasing the visibility of this program among employers.

7. Employment Outcomes. The preliminary results from the formative evaluation indicate positive results when EBSMs are closely linked to job opportunities and have a work experience orientation.

The formative evaluation results are encouraging, particularly in light of the organizational changes in which the EBSMs were implemented. The absence of comparative benchmarks preclude categorical statements of achievements. The evaluation findings identify advances made and areas for improvement that can be used by HRDC to make refinements in the provision of EBSMs to clients in Ontario.