Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

Overall, the Employment Benefits and Support Measures (EBSMs) were perceived to be functioning quite well, although various aspects of the implementation and delivery of the EBSMs require improvement. In general, the areas for improvement concern communications, information sharing and resource issues. The main findings and conclusions from the formative evaluation are presented in the following sections.

Relevance of Labour Market Development Agreement (LMDA) Programs

LMDA programs and services were generally perceived to be consistent with Employment Insurance (EI) legislation and the priorities of the Yukon Territory and were felt to be meeting the needs of clients, employers and communities. In instances where programs and services were felt to be less relevant (for example, Human Resource Centre of Canada [HRCC] not responding to clients' emotional needs, program eligibility, timing of wage subsidies), these perceptions could often be traced to a lack of understanding of the role played by EBSMs in the range of employment programming in the territory (for example, increased use of third-party providers in service delivery, targeting Employment Insurance (EI) clients and need for programming in the off-seasons). Nonetheless, a number of factors were identified which may decrease the relevance of programs and services, including gaps in services provided and clients served, aspects of the design and delivery of programs (for example, emphasis on volume of clients served, definitions of success as simply employment), the need for coordination of program delivery; the need for more partnering (for example, with First Nations), communications problems (for example, information sharing between government, third parties and stakeholders), and resource issues (for example, need for training, supports for partnership).

Partnership

Partnering efforts have shown improvement since the LMDA was first implemented. While partnering efforts among governments seem to be working at senior management levels, more attention to this may be required among government partners, as well as among senior and less senior levels in terms of communication and information sharing. Respondents to this evaluation feel that the greatest improvements in partnering would be realized through increased communication and consultation with partners, be it better reporting of the results of joint planning exercises and evaluations or more coordination of effort and consultation in the design, delivery and funding of programs and services. The development of common priorities among the three partner departments, as well as the creation of appropriate supports for partnering activities, may also improve this aspect of the LMDA.

A range of partnerships exists among government, employers and the community, including community consultations and informal networks among third-party delivery agents. Respondents generally felt that these relationships would also improve through increased communication and information sharing.

Duplication and Overlap

Overall, LMDA programs and services tend to be somewhat complementary to other non-LMDA programming delivered in the territory. Although no evidence of duplication was found, the primary area in which overlap was perceived to exist was in the types of services delivered through Starting Points and Job Finding Clubs (for example, resume writing, job search, etc.). Although some respondents felt similar client groups were targeted by different programs, these sources are not likely to represent major problems in that the duplication of programming was occurring with non-LMDA targeted client groups (similar programming for youth and First Nations) or, in other cases, there is some question about whether the example represented a true overlap in programming (i.e., similar client group targeted but different eligibility requirements). One form of duplication, the use of multiple interventions, was perceived to have positive consequences for participants.

Service Delivery

From the perspective of clients, the delivery of services under the LMDA is thought to be good because of the relevant content of programming and the convenience of services (for example, no waits, staff well-prepared). Nonetheless, there is room for improvement of some aspects of service delivery, particularly the quality of services such as counselling and job search advice, and information on the types of programs and services available. Survey evidence also showed that a large majority of clients are accessing self-serve resources, and that this is typically occurring at an HRCC or through the Internet. The use of return-to-work action plans was only moderate, although there was a high rate of completion of action plan activities.

Respondents provided mixed feedback on the extent to which programs and services are sufficiently flexible to adapt to the needs of the community when they are used as designed. Although a number of third-party deliverers felt programs were sufficiently flexible, it was also clear that some lack a thorough understanding of the rules and guidelines for program administration.

The delivery of programs and services relies heavily on the use of third parties and they were perceived to be doing a good job. A primary obstacle to effective service delivery among third parties was the lack of information sharing to allow delivery agents to manage clients effectively. Positive changes to service delivery that have occurred under the LMDA include more autonomy among third parties, partnerships to support third-party service delivery and a more client-centered approach.

The lack of information sharing among governments and third parties has led to administrative inefficiencies by causing all parties to collect the same information for the same clients, delivery agents to provide similar assessment services, clients to complete similar forms each time they access services from different delivery agents, and employers to be approached by different service providers. Suggestions to reduce administrative inefficiencies were focussed on better communication and coordination among all groups.

Non-HRDC government respondents generally felt that case management was not working, although HRDC respondents were more positive concerning this aspect of service delivery. Part of the negative perception may be that some respondents, particularly non-HRDC respondents, may have misunderstood how and by whom case management was to be done (and therefore felt it was not being done). Nonetheless, all respondents identified weaknesses of the case management approach, including data systems problems (i.e., Contact IV) which make it difficult to properly manage clients and the labour intensity of the approach.

Promotion of Programs and Services

LMDA programs and services have not been well promoted in the community. Many clients felt ill-informed of the nature of the programs and program objectives prior to their participation and there was no clear understanding of the range of program options, both LMDA and non-LMDA, available to clients. Similarly, there was a perception that third-party delivery agents needed to be better informed of the range of programs and services available in the territory and that there was a need to promote programs among employers. Although some advertising is taking place, word of mouth was felt by some to be the most effective means of promoting EBSMs, especially since governments lack funds dedicated to the promotion of EBSMs. Information provided by third parties to clients accessing EBSMs was perceived by clients to be adequate and included information on other programs and services.

Labour Market Information

Among respondents to the qualitative components of the evaluation, mixed perceptions exist concerning the availability and quality of Labour Market Information (LMI). While some respondents reported a wealth of LMI, others felt that it is not readily accessible and that what information does exist is of limited usefulness because it is not specific to the Yukon and is never or rarely community specific. Overall the results suggest low awareness and/or access to the LMI rather than a lack of such information.

Monitoring and Data Collection

Current monitoring and data collection systems were found to be inadequate. The sharing of client information is generally perceived to be insufficient to support client service and program development. Further, a comparison of administrative files used in the development of the participant survey sample frame indicated that the existing tracking and monitoring systems might be failing to capture the full universe of EBSM program participants.

Impacts on Clients

A number of programs and characteristics of service delivery were perceived to contribute to clients assuming more responsibility for their employment situation. Programs were felt to be moving clients off income support but it was recognized that a number of other factors, such as the availability of jobs, must be present before a reduced reliance on income support can be achieved and that a number of barriers to self-reliance exist.

Over four fifths of survey respondents indicated that the programs have had at least a moderate positive impact on all aspects of their employment situation about which they were asked. Participants were also more likely than comparison group members to rate the employment services they received as very important in helping them to obtain their current job. Further, the rates of employment among participants increased dramatically from the pre- to post-intervention periods. It is important to bear in mind, however, that too little time has elapsed for an accurate assessment of the impact of Employment Benefits and Support Measures (EBSMs) on clients to be made. These incremental results will be pursued at the summative stage of the evaluation.

Impacts on Community

EBSMs may lead to several positive impacts in the community. These include the reduction of out-migration, the provision of services to the full range of clientele that require employment services and the development of small businesses.

Assessment of Primary Measures

Changes to the accountability targets for the Canada/Yukon LMDA suggest that these targets are flexible and sensitive to the changing conditions, realities and restraints of the territory. The accountability targets have fallen just short of being met. The targets for the number of clients served, the proportion of participants who were active EI claimants, and the number of participants returning to work were all very close to being met. The target for unpaid benefits resulting from EI clients returning to work, however, was considerably short of being met.

A number of potential measurement problems were identified in the administrative data and methods used by HRDC to track and measure LMDA success. These included not all EBSMs of participants in the administrative dataset, the underestimation of returns to work and the overestimation of unpaid benefits. These problems cast doubt on existing measures of success based on these data and indicate the need for HRDC to re-assess its methods in this respect.