Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

Under the terms of the Canada/Yukon Labour Market Development agreement (LMDA), the federal and territorial governments agreed to jointly manage active labour market measures for the unemployed in the territory. This LMDA came into being as a result of the federal government's commitment, announced in November 1995, to devolve responsibility for labour market development to the provinces and territories. This commitment was made in response to the expressed desire by territorial and provincial governments to assume greater control over employment development programs. This report presents the results of a formative evaluation of Employment Benefits and Support Measures (EBSMs) under the terms of the Canada/Yukon LMDA.

The Canada/Yukon LMDA, which came into effect on January 24, 1998, sets out the individual EBSMs themselves. Canada and the Yukon are jointly responsible for the design, management and evaluation of the EBSMs, which mirror those outlined in the Employment Insurance Act. Four employment benefits were the focus of the current evaluation: Targeted Wage Subsidies, Self-Employment, Job Creation Partnerships, and Training Purchases/ Enhanced Feepayers. The support measures considered here are Local Labour Market Partnerships and Employment Assistance Services. Another employment benefit (Skills Development Employment Benefit) and a support measure (Research and Innovation) are briefly profiled, but were not included in the context of this evaluation.

Multiple methods were used to collect data for this formative evaluation. Specifically, quantitative methods (i.e., surveys) were used to obtain early results which could be applied to conclusions about the full population of participants in EBSMs. Qualitative methods (i.e., key informant interviews, focus groups and a case study) as well as a literature review, were used to gather particular in-depth perspectives on the research issues.

Overall, the EBSMs were perceived to be functioning quite well, although various aspects of the implementation and delivery of the EBSMs require improvement. In general, the areas for improvement concern communications, information sharing and resource issues. The main findings and conclusions from the formative evaluation are described below.

Relevance of LMDA Programs

LMDA programs and services were generally perceived to be consistent with EI legislation and the priorities of the Yukon and were felt to be meeting the needs of clients, employers and communities. In instances where programs and services were felt to be less relevant (for example, Human Resource Centre of Canada [HRCC] not responding to clients' emotional needs, program eligibility, timing of wage subsidies), these perceptions could often be traced to a lack of understanding of the role played by EBSMs in the range of employment programming in the Yukon (for example, increased use of third-party providers in service delivery, targeting EI clients and need for programming in the off-seasons). Nonetheless, a number of factors were identified which may decrease the relevance of programs and services, including gaps in services provided and clients served, aspects of the design and delivery of programs (for example, emphasis on volume of clients served, definitions of success as simply employment), the need for coordination of program delivery, the need for more partnering (for example, with First Nations), communications problems (for example, information sharing between government, third parties and stakeholders), and resource issues (for example, need for training, supports for partnership).

Partnership

Partnering efforts have shown improvement since the LMDA was first implemented. While partnering efforts among governments seem to be working at senior management levels, more partnering may be required among and within the government partners, as well as among senior and less senior levels in terms of communication and information sharing. Respondents to this evaluation felt that the greatest improvements in partnering would be realized through increased communication and consultation with partners, be it better reporting of the results of joint planning exercises and evaluations or more coordination of effort and consultation in the design, delivery and funding of programs and services. The development of common priorities among the three partner departments, as well as the creation of appropriate supports for partnering activities, may also improve this aspect of the LMDA.

A range of partnerships exist among government, employers and the community, including community consultations and informal networks among third-party delivery agents. Respondents generally felt that these relationships would also improve through increased communication and information sharing.

Duplication and Overlap

Overall, LMDA programs and services tend to be somewhat complementary to other non-LMDA programming delivered in the Yukon. Although no evidence of duplication was found, the primary area in which overlap was perceived to exist was in the types of services delivered through Starting Points and Job Finding Clubs (for example, resume writing, job search, etc.). Although some respondents felt similar client groups were targeted by different programs, these sources of overlap are not likely to represent major problems in that the duplication of programming was occurring with non-LMDA targeted client groups (similar programming for youth and First Nations). In other cases, there is some question about whether the example represented a true overlap in programming (i.e., similar client group targeted but different eligibility requirements). One form of duplication, the use of multiple interventions, was perceived to have positive consequences for participants. Administrative overlaps may also be occurring and were felt to be due primarily to the lack of information sharing between governments and third parties which has required all parties to collect the same information for the same clients. Suggestions to reduce administrative overlap were focussed on better communication and coordination among all groups.

Service Delivery

From the perspective of clients, the delivery of services under the LMDA is thought to be good because of the relevant content of programming and the convenience of services (for example, no waits, staff well-prepared). Nonetheless, there is room for improvement of some aspects of service delivery, particularly the quality of services such as counselling and job search advice, and information on the types of programs and services available. Survey evidence also showed that a large majority of clients are accessing self-serve resources, and that this is typically occurring at an Human Resources Centre of Canada (HRCC) or through the Internet. The use of return-to-work action plans was only moderate, although there was high rate of completion of action plan activities.

Respondents provided mixed feedback on the extent to which programs and services are sufficiently flexible to adapt to the needs of the community when they are used as designed. Although a number of third-party deliverers felt programs were sufficiently flexible, it was clear that some lack a thorough understanding of the rules and guidelines for program administration.

The delivery of programs and services relies heavily on the use of third parties and they were perceived to be doing a good job. A primary obstacle to effective service delivery among third parties was the lack of information sharing to allow delivery agents to manage clients effectively. Positive changes to service delivery that have occurred under the LMDA include more autonomy among third parties, partnerships to support third-party service delivery and a more client-centered approach.

Non-Human Resource Development Canada (HRDC) government respondents generally felt that case management was not working, although HRDC respondents were more positive concerning this aspect of service delivery. Part of the negative perception may be that some respondents, particularly non-HRDC respondents, may have misunderstood how and by whom case management was to be done (and therefore felt it was not being done). Nonetheless, all respondents identified weaknesses of the case management approach, including data systems problems (i.e., Contact IV) which make it difficult to properly manage clients and the labour intensity of the approach.

Promotion of Programs and Services

LMDA programs and services have not been well promoted in the community. Many clients felt ill-informed of the nature of the programs and program objectives prior to their participation and there was no clear understanding of all program options, both LMDA and non-LMDA, available to clients. Similarly, there was a perception that third-party delivery agents needed to be better informed of the range of programs and services available in the territory and that there was a need to promote programs among employers. Although some advertising is taking place, word of mouth was felt by some to be the most effective means of promoting EBSMs, especially since governments lack funds dedicated to their promotion. In contrast to the perceptions of clients who felt they were ill-informed about the nature of programs, third parties felt the information they provided to clients accessing EBSMs was adequate and included information on other programs and services.

Labour Market Information

Among respondents to the qualitative components of the evaluation, mixed perceptions exist concerning the availability and quality of Labour Market Information (LMI). While some respondents reported a wealth of LMI, others felt that it is not readily accessible and that what information does exist is of limited usefulness because it is not specific to the Yukon and is never or rarely community specific. Overall the results suggest low awareness and/or access to the LMI rather than a lack of such information.

Monitoring and Data Collection

Current monitoring and data collection systems were found to be inadequate. The sharing of client information is generally perceived to be insufficient to support client service and program development. Further, a comparison of administrative files used in the development of the participant survey sample frame indicated that the existing tracking and monitoring systems might be failing to capture the full universe of EBSM program participants.

Impacts on Clients

A number of programs and characteristics of service delivery were perceived to contribute to clients assuming more responsibility for their employment situation. Programs were felt to be moving clients off income support but it was recognized that a number of other factors, such as the availability of jobs, must be present before a reduced reliance on income support can be achieved and that a number of barriers to self-reliance exist.

Over four fifths of survey respondents indicated that the programs have had at least a moderate positive impact on all the aspects of their employment situation about which they were asked. Participants were also more likely than comparison group members to rate the employment services they received as very important in helping them to obtain their current job. Further, the rates of employment among participants increased dramatically from the pre- to post-intervention periods. It is important to bear in mind, however, that too little time has elapsed for an accurate assessment of the impact of EBSMs on clients to be made. The incremental results will be pursued at the summative stage of the evaluation.

Impacts on Communities

EBSMs may lead to several positive impacts in the community. These include the reduction of out-migration, the provision of services to the full range of clientele who require employment services and the development of small businesses.

Assessment of Primary Measures

Changes to the accountability targets for the Canada/Yukon LMDA suggest that these targets are flexible and sensitive to the changing conditions, realities and restraints of the territory. The accountability targets have fallen just short of being met. The targets for the number of clients served, the proportion of participants who were active Employment Insurance (EI) claimants, and the number of participants returning to work were all very close to being met. The target for unpaid benefits resulting from EI clients returning to work, however, was considerably short of being met.

A number of potential measurement problems were identified in the administrative data and methods used by Human Resources Development Canada (HRDC) to track and measure LMDA success. These included not all EBSM participants being in the administrative dataset, the underestimation of returns to work, and the overestimation of unpaid benefits. These problems cast doubt on existing measures of success based on these data and indicate the need for HRDC to re-assess its methods in this respect.