Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

This chapter presents evidence from the formative evaluation concerning the success of Employment Benefits and Support Measures (EBSMs) to date. It is important to bear in mind that the results presented in this chapter provide only an early glimpse of the success of EBSMs. As relatively little time has elapsed since the Labour Market Development Agreement (LMDA) was first implemented, it is likely that many of the impacts and successes that will ultimately be observed for the EBSMs have not yet begun to present themselves.

Additionally, measurements of program successes in this formative evaluation are obtained through a quasi-experimental design. Quasi-experimental designs provide an alternate means for examining causality in situations not conducive to experimental controls and where there may be few, if any, alternative approaches to examine the phenomenon of interest, such as the impact of program participation on employment behaviour and labour market attachment among participants in LMDA programs. While these designs were developed to control as many threats to validity as possible (for example, pre-existing differences between the two groups which compete with the variable of interest [i.e., program participation] to explain any observed differences, otherwise known as confounding factors), they lack at least one of the other two properties that characterize true experiments, randomization and a control group. Although the current evaluation does employ a comparison group, participants and comparison group members alike were not randomly assigned to their respective groups and as such, the inherent differences between members of these two groups (i.e., differences that prompted some to participate and others not to participate) present an alternative explanation to any observed program impacts. Although multivariate analyses can be used to simultaneously control for numerous potentially confounding factors, it is nonetheless early (i.e., too little time has elapsed since the LMDA was first implemented) to assess the incremental impacts of the EBSMs. As such, the descriptive findings related to program impacts are presented for illustrative purposes only. The calculation of incremental results will be pursued as part of the summative evaluation.

5.1 Impacts on Clients — Descriptive Results

5.1.1 Self-Reliance

Third-party and government respondents suggested a number of programs and characteristics of service delivery that may contribute to clients assuming more responsibility for their employment situation, including the Self-Employment program (possibly because it provides very definite timelines for the client to succeed), training and certification, basic education, vocational life skills, job skills, post-training follow-up and assistance, the provision of advice, creation of self-worth, the availability of Labour Market Information (LMI) and client involvement in the assistance. Another third-party respondent felt that, on the whole, the simple acquisition of additional employment-related skills, be it job search or specific training, is likely to decrease clients' reliance on income support. As well, government respondents felt that programs are moving clients off income support, but that a number of other factors are needed in order for a reduced reliance on income support to be realized, including the availability of jobs that pay higher than income support amounts, interventions that reach clients earlier on in their employment history, more coordination between programs, more community partners, more personalized contact, more focus on motivation and long-term guidance.

Barriers that were perceived to exist and that may impede client self-reliance included a lack of daycare funding and funding for other non-employment related needs (for example, rent, food, etc.), the loss of disability benefits or social assistance (SA) when clients participate in the EBSMs, the inaccessibility of services outside of Whitehorse, the need for lifestyle changes (i.e., adapting to a lifestyle that includes employment, scheduling, etc.) and clients' lack of ability to plan ahead in order to meet labour market demands. Another barrier to self-sustainability involves the general mindset of individuals who have come to rely on income assistance (EI or SA), as some clients tend to remain on EI until their claim has run out and look for work only when they have nothing to lose by quitting a new job. Finally, some training deliverers felt the rate of completion of training courses was much higher when delivery agents could pay clients for attending classes as if they were salaried employees.

The use of Negotiated Financial Assistance (NFA) was felt by some third-party deliverers to increase the commitment of participants to their particular program because the amount of assistance is based on individual client need and the client is more active in the development of their employment plan. NFA was thought to contribute to clients taking primary responsibility for identifying their employment needs and sharing in the cost of assistance but was felt to have less of an impact on reducing reliance on income support (for example, EI or SA) or increasing employment because environmental factors, such as high unemployment, were felt to have a stronger influence on these outcomes. One respondent felt that NFA does not pose a barrier to training because clients who are unable to contribute anything to the cost of training may receive the full amount of their training.

Qualitative evidence from third parties provided mixed results in terms of the extent to which clients are required to take responsibility for preparing a return-to-work action plan. While some respondents indicated that everyone was required to have an action plan, others noted that only a moderate proportion of clients developed action plans. Of those clients who develop action plans, survey and qualitative evidence suggests that the large majority (between 71 and 80 percent of program participants) follow the plan through to its end, a finding that some key informants felt may in part be attributable to the lack of jobs in the Yukon. As well, those third-party agents that required clients to develop an action plan indicated that they followed-up on the plans with clients and some also have detailed protocols for following up with clients involved in training.

5.1.2 Perceived Program Impacts

Overall, the survey results show that roughly four fifths of survey respondents feel that the EBSMs have had at least a moderate positive impact on each of the impacts measured. The EBSMs were most likely to have been perceived to have had a large positive impact (responded with a 6 or 7 on a 7-point scale) on respondents' self-esteem (55 percent), satisfaction with their career path (50 percent) and job search skills (47 percent). Respondents were less likely to feel the EBSMs had a positive impact on their job satisfaction (42 percent) and ability to make appropriate career choices (40 percent) (Exhibit 5.1).

Focus groups with clients revealed that many felt the true success of the programs was in life-skills development and personal growth. The programs provided a meaningful purpose for clients and a support network with individuals in similar circumstances. On-the-job experience, training and job simulation programs were felt to contribute the most to the success of interventions. Job placement programs were felt to be least effective when the job placement was for too short a period (i.e., 20 weeks).

5.1.3 Rated Importance of Assistance

Survey respondents were asked to rate how important the employment services they received were in helping them to get their current or most recent job. As shown in Exhibit 5.2, 35 percent of participant survey respondents who were employed in the post-program period felt that the program was very important in this respect (responded with a 6 or 7 on a 7-point scale). Twenty four percent rated the program as somewhat important (responded with a 3 to 5 on a 7-point scale) and 41 percent rated the assistance as not important (responded with a 1 or 2 on a 7-point scale) in helping them to get a job. Employment Assistance Services (EAS) and Training Purchases participants were the most likely to feel that the employment program was very important in helping them find a job (39 and 36 percent, respectively), whereas Self-Employment and Job Creation Partnerships (JCP) participants were the least likely to feel this way (19 and 17 percent, respectively).

It should be noted that this question was addressed only to those respondents who had a job in the post-intervention period and to respondents who were not continuers (i.e., Self-Employment participants who did not continue operating their businesses or Wage Subsidies or Employment Partnership participants who were not hired on by the host employer in the post-program period). If it is assumed that these continuers (i.e., those whose business/program jobs continued beyond the program end-date) would have responded "very important" to this question, the rated importance of the employment services rises dramatically among Self-Employed (19 to 74 percent) and JCP participants (17 to 65 percent).

While participant ratings of the importance of their employment program are somewhat modest, a comparison of participant and comparison group survey responses reveals that, overall, participants were somewhat more likely than comparison group respondents to rate the employment services they received as very important in helping them to get a job (Exhibit 5.3). This implies an advantage for participants in LMDA programs relative to non-participants.

Rates of Employment

Table 5.1 presents various employment outcomes for participant and comparison group survey respondents. Overall, these results suggest that some advantage may exist for participant claimants compared to comparison group claimants.

The first panel of Table 5.1 shows that a large majority of both the participant and comparison groups indicated that they have had a job since the end of the program or reference date. The incidence of employment at some time in the post-program period was high for all program respondents and ranged from a low of 78 percent of Employment Assistance Services (EAS) participants to a high of 100 percent of Targeted Wage Subsidies (TWS) respondents. Participant claimants were more likely than comparison group claimants to indicate that they have been employed at some time in the post-program period (86 and 76 percent, respectively), although no difference in employment rates for reachbacks was observed between these two groups.

A review of employment rates according to respondents' socio-demographic profiles (not shown) shows that the incidence of employment was higher among non-equity group respondents.

Another way in which the impacts of LMDA programs were measured involved a comparison of employment status in the week prior to the intervention or reference date and employment status at the time of the survey. Generally, these findings show positive shifts in employment from the pre- to post-intervention periods for all EBSMs and larger shifts for participants relative to the comparison group.

As shown in Table 5.2, there was a larger overall increase in employment from the pre- to post-program periods among participants relative to the comparison group. It is important to note, however, that the smaller observed increases in employment rates among the comparison group are at least partly attributable to a ceiling effect caused by their higher rates of pre-reference date employment. Unemployment rates (ninth row of Table 5.2) declined for both participants overall and comparison group claimants at the time of the survey but the former exhibited considerably larger declines in unemployment rates (67 to 24 percent for claimants and 67 to 26 percent for reachbacks) while comparison group claimants showed a more moderate decline in their rates of unemployment (46 to 39 percent). In contrast to the other groups, the rate of unemployment among comparison group reachbacks rose slightly from the pre- to post-program period (12 to 18 percent), although the rate of unemployment in the post-program period was much lower among these respondents relative to the other groups.

View Table 5.1

Positive shifts in employment were observed for all of the programs under study as well (sum of rows 1 to 7 in Table 5.3). All groups at least doubled their overall employment rate and the largest positive shifts in overall employment occurred for Self-Employment (SE) (19 to 81 percent) and TWS participants (23 to 100 percent). For all EBSMs except Self-Employment, the largest increase in employment was observed with respect to full-time, year-round jobs. For Self-Employment participants, the largest increase was observed with respect to the proportion of respondents reporting that they were self-employed. Unemployment was also much lower at the time of the survey, dropping dramatically for all EBSMs (ninth row of Table 5.3).

View Table 5.3

5.2 Outcomes by EBSM

In this section, we present key evaluation findings as they relate to each of the EBSMs. Most of the findings presented here relate to program impacts and are based on results from the participant survey.

5.2.1 Targeted Wage Subsidies (TWS)

The Targeted Wage Subsidies (TWS) program is designed for workers who are at risk of long-term unemployment, lack experience or face other barriers to employment, to help them to find a job and gain work experience. The program provides a wage subsidy to eligible employers to encourage them to offer jobs to such individuals.

TWS program respondents were most likely to indicate that they were interested (responded with a 6 or 7 on a 7-point scale) in entering the workforce on a full-time or part-time basis (100 percent) in the week prior to starting their first employment program, suggesting that the program was well matched to participants' interests. Over half (58 percent) of TWS respondents felt they would not have been able to obtain the job in which they were placed had the wage subsidy not been available.

The rate of program completion among TWS respondents was good, as only two TWS participants indicated leaving their wage subsidy jobs prior to the completion of the full period of funding (22 percent). Seventy-eight percent of TWS program respondents were retained by their program employer at the end of the program²³ and 90 percent of these respondents indicated that they were hired into the same job as when their job was funded. Those who were hired at the end of the program were hired into full-time year-round jobs (82 percent) or full-time seasonal positions (18 percent). Seventy-four percent of respondents who were hired at the end of the TWS program felt that the employment services they received were very important (responded with a 6 or 7 on a 7-point scale) in helping them to get this job and none of the respondents indicated that the program was not important.

5.2.2 Self-Employment (SE)

The Self-Employment benefit aims to create self-employment by providing financial assistance for a fixed period to individuals with sound business plans to start their businesses. A third party is partnered with the HRCC to deliver this benefit. The contractor provides a business planning workshop and an independent advisor for participants.

Pre-intervention interest ratings show that SE program respondents were placed into an appropriate program, as these respondents were most likely to indicate that they were very interested in starting their own business (81 percent) at that time. Fifty-two percent of Self-Employment program respondents felt they would have been unable to start their business if the Self-Employment program had not been available.

Over four fifths (81 percent) of SE participants took a training program or workshop to help them start their business, although roughly one quarter of SE participants indicated that their businesses had not yet started. The majority of Self-Employment businesses operate year-round (88 percent) and 36 percent of respondents whose businesses have started indicated that their businesses employed someone other than themselves. Self-Employment participants earned an average of $483 per week and only 17 percent worked at other jobs in addition to their business. Ninety-one percent of Self-Employment program respondents whose business had started indicated that it was still in operation at the time of the survey.

5.2.3 Job Creation Partnerships (JCP)

The objective of the Job Creation Partnerships (JCP) program is to create meaningful work experience opportunities through temporary employment that will lead to long-term employment opportunities. Under this benefit, delivery agents and local community partners combine their efforts to create jobs, with HRCC Whitehorse contributing up to 100 percent of the remuneration (the EI benefit). Employers are encouraged to provide a top-up to meet the prevailing wage rate, and the maximum term of the project is 20 weeks.

Job Creation Partnerships program respondents' high levels of pre-intervention interest in entering the workforce on a full-time or part-time basis (88 percent) suggests that the JCP program was the appropriate employment program for the majority of participants. Nearly two thirds (65 percent) of JCP participants felt they would not have been able to obtain their program job had the EI subsidy not been available at the same time.

In their JCP program jobs, respondents were most likely to indicate that they worked for a private employer (60 percent). Relatively few JCP program participants indicated leaving the program prior to the completion of the full period of funding (28 percent) and the majority (58 percent) were retained by their host employer at the end of the program.²⁴ JCP respondents who were hired on were most likely to have been hired into full-time year-round jobs (45 percent), followed by part-time year-round (19 percent), full-time seasonal (18 percent) and part-time seasonal jobs (seven percent) (11 percent did not respond). Fifty-three percent of the respondents hired at the end of the program felt the employment services they received were very important (responded with a 6 or 7 on a 7-point scale) in helping them to get the job. Only 15 percent felt they were not important.

One key informant who was familiar with the JCP program felt that the reporting and monitoring requirements for the program have resulted in less flexibility and an adherence to the strict rules of program delivery to ensure that accountability requirements are met. The activities that take place under JCP include the Student Training and Employment Program (STEP), Job Shadowing and Co-op programs. A broad range of organizations were known to participate, including YTG, the federal government, crown corporations, private businesses and non-governmental organizations (NGOs).

It was felt that the JCP program contributed to supporting community non-profit infrastructure by the simple fact that clients are placed with NGOs and because these placements sometimes involve developing projects to support this infrastructure. The overall impact of the program on the community was felt to be quite positive as it contributes to increased employment, encourages people to stay in the community, creates a learning opportunity for clients that is directly related to their professional training, results in jobs and savings and has potential social impacts since participants are less likely to become disaffected and marginalized if they are able to gain employment. It may also contribute to long-term employment at the local level because clients develop a relationship with the participating employer and develop a skill set that is customized to the employer's needs.

5.2.4 Training Purchases

Training Purchases are not an employment benefit under the LMDA but a now phased-out training mechanism. The objective of the Training Purchases benefit was to encourage the unemployed to acquire skills through education and training for the purposes of ending dependence on EI benefits and entering employment. Training is now supported under the Skills Development Employment Benefit which supports individuals who make their own training arrangements.

Training Purchases respondents were most likely to indicate that they were interested (responded with a 6 or 7 on a 7-point scale) in entering a training or education program (81 percent) and in entering the workforce on a full- or part-time basis (80 percent) in the week prior to starting their first intervention. Fully 79 percent of these respondents felt they would not have been able to enter the training program if financial assistance for course costs had not been available.

Respondents were most likely to indicate that the training programs they took prepared them for jobs in trades, transport and equipment operation (34 percent); business, finance and administration (17 percent); academic upgrading (12 percent); or computers and IT (12 percent). The majority of Training Purchases respondents reported that they received their training from a community college (85 percent) and most training programs were less than six months in duration (44 percent). Over two thirds (71 percent) of respondents indicated that they were required to contribute to the cost of their training and the average contribution was $792. The majority of Training Purchases participants either completed their training program (77 percent) or were still in the process of finishing their training at the time of the participant survey (9 percent).

5.2.5 Enhanced Feepayers (Predecessor to Skills Development Employment Benefit [SDEB])

The Enhanced Feepayers program (replaced by Negotiated Financial Assistance) provided negotiated income support to EI eligible clients taking training and was intended to encourage the unemployed to acquire skills through education and training. Feepayers paid for their own training and collected Employment Insurance benefits while taking training.

Enhanced Feepayers program respondents were most likely to indicate that they were very interested (responded with a 6 or 7 on a 7-point scale) in entering a training or education program (87 percent) in the week prior to starting their first intervention. A large majority of Enhanced Feepayers participants (84 percent) felt they would not have been able to enter their training program if they had not been eligible to collect EI while taking the course.

Respondents were most likely to indicate that the training programs they took prepared them for jobs in health-related fields (15 percent); computers and IT (13 percent); trades, transport and equipment operation (13 percent); and business, finance and administration (12 percent). Most received their training from a community college (78 percent). The average length of training programs was 33.4 weeks and the average number of hours per week that respondents spent in their programs was 32.4 hours. More than half (58 percent) of the Enhanced Feepayers respondents indicated that they had contributed to the cost of their training, with an average contribution of $3,575. A large majority of Enhanced Feepayers participants had completed their training program (82 percent) at the time of the participant survey.

5.2.6 Employment Assistance Services (EAS)

The Employment Assistance Services (EAS) measure supports contracts with third parties, such as private sector organizations and community groups, to help unemployed individuals obtain and maintain employment. The types of activities covered under EAS include employability skills enhancement, career counselling, self-assessment workshops and job finding clubs.

In the week prior to starting their first intervention (within the time period under investigation in this study), Employment Assistance Services (EAS) program respondents were most likely to indicate that they were very interested (responded with a 6 or 7 on a 7-point scale) in entering the workforce on a full- or part-time basis (91 percent).

The most frequently mentioned types of resources that EAS participants reported using on their own were newspaper advertisements (80 percent) and job bank kiosks or postings (68 percent). EAS respondents were most likely to have accessed self-service resources from a Human Resource Centre of Canada (HRCC) (77 percent), the Internet (29 percent) or a public library (22 percent). At the first place they received in-person assistance (i.e., from an HRCC or elsewhere), more than two thirds (70 percent) of the EAS participants who responded to the survey reported that they had spoken with staff to help them plan their strategy to get back to work. Forty-one percent reported that they had developed a return-to-work action plan²⁵ with an employment counsellor and 84 percent of those who had developed an action plan indicated that they completed the activities in their plan.

5.2.7 Local Labour Market Partnerships (LLMP)

A range of activities take place under the Local Labour Market Partnerships (LLMP), including labour market research for specific areas or groups (through community consultations, employer surveys and other methods); the development of labour market recovery plans (for example, in the situation of the Faro mine closure); the development of long-term training strategies for industry sectors (such as tourism); workshops, training initiatives and projects to create on-going employment aimed at various client groups (including women, First Nations and disabled persons); and program reviews to increase the efficiency and effectiveness of other employment programs.

All case study respondents agreed that the LLMPs have met local labour force priorities. The primary reason respondents felt this way was that the majority of the projects are focused on improving employment in some form or another, be it directly through the creation of employment opportunities, or less directly through training, the development of labour market information (LMI) through research, or supporting initiatives that have as part of their objectives the advancement of employment among certain groups. Labour force priorities are identified through a number of methods. In fact, the focus of the LLMPs themselves is often on identifying labour force priorities through primary and secondary labour market research.

All respondents felt that partnerships are a crucial ingredient for the success of LLMP projects and felt that the LLMPs have been quite successful in encouraging partnership. The partners involved in LLMPs represent almost every potential stakeholder in the territory and include employers, non-governmental organizations (NGOs), communities, community organizations, industry associations, sector councils, educational institutions (colleges and high schools) and tribal bands.

While most respondents felt that partners understood the concept behind LLMPs, they nonetheless felt that there was confusion concerning LLMPs and a need for promotion to increase awareness of the program. One respondent remarked that most partners are not even aware of the LLMP and that for many the program is simply a source of funding. Respondents reported that partners were generally satisfied with the processes to establish cost sharing, responsibilities and deadlines because they were involved in their development.

Respondents from community organizations were generally happy with the application process to obtain LLMP funding. All community respondents were satisfied with the support they received from HRCC staff, although they noted that HRDC may not have the necessary human resources to turn applications around in a timely fashion or to be proactive in terms of identifying projects that could be implemented to address community needs.

The impacts of the LLMPs on the community are diverse and include increased employment, improved employability due to enhanced skills, knowledge and mobility, a more highly skilled workforce for employers, direction and guidance for the development of labour market strategies, the receipt of funding for new projects to address LLMP identified needs, and improved partnership. Among the few weaknesses of LLMPs that were mentioned is low awareness of the program among stakeholders; and a few suggested there were insufficient human resources to support delivery of the program in a proactive fashion. Program strengths were felt to be its flexibility in the types of projects it can support, including projects that would not be supported through other funding sources, and the focus on community and partnerships. Interestingly, flexibility may also be a weakness of the program as one respondent noted that there might be a tendency for some organizations to use the program for things that go beyond the program's intended objectives.

5.3 Impacts on Communities

Evidence gathered through the formative evaluation suggests that EBSMs may have a number of impacts on the community. To begin, analysis of survey data shows that the Self-Employment program may lead to increased employment among participants and has contributed to the development of small business. Thus, businesses started through this program contribute to the economic health of the communities in which they are found. EBSMs may also benefit communities by reducing out-migration. A review of the administrative data revealed that a higher proportion of comparison group members than participants leave the Yukon, presumably in search of employment, suggesting that programs may contribute to clients staying in their communities (although it may also be that those most able to find employment elsewhere leave, while those with few options or skills remain behind and thus are available to participate in the EBSMs). Nonetheless, qualitative evidence suggests many programs and services are unavailable outside of Whitehorse and that clients may need to relocate in order to access these services. Finally, survey evidence suggests that Aboriginal and disabled clients account for a higher proportion of program participants than comparison group members (i.e., LMDA eligible clients). As such, EBSMs appear to be reaching all members of the community that require these programs and services.

5.4 Summary of Impacts on Outcomes

A number of programs and characteristics of service delivery were perceived to contribute to clients assuming more responsibility for their employment situation. Programs were felt to be moving clients off income support but it was recognized that a number of other factors, such as the availability of jobs, must be present before a reduced reliance on income support can be achieved and that a number of barriers to self-reliance exist.

Over four-fifths of survey respondents indicated that the programs have had at least a moderate positive impact on all aspects of their employment situation about which they were asked. Participants were also more likely than comparison group members to rate the employment services they received as very important in helping them to obtain their current job. Further, the rates of employment among participants increased dramatically from the pre- to the post-intervention periods. It is important to bear in mind, however, that too little time has elapsed for an accurate assessment of the impact of Employment Benefits and Support Measures (EBSMs) on clients to be made. These incremental results will be pursued at the summative stage of the evaluation.

Communities may benefit from EBSMs in a number of ways. The reduction of out-migration, the provision of services to the full range of clientele that require employment services and the development of small businesses are all examples of how EBSMs can have positive effects on the community.

Footnotes

23	Respondents who indicated they left the program before it ended (n=2) were coded as "not hired".
24	Respondents who indicated that they left the program before it ended (n=13) were coded as "not hired."
25	Only the Starting Points program develops what is called a "return-to-work action plan" and, as of July 1999, an employment counselling service was added to the Job Funding Club contract, so the "action plan" term would be used with these clients as well. Thus, it may be that this figure under estimates the number of action plans that are being developed simply because clients did not recognize the term.