Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

Under the terms of the Canada/Prince Edward Island Labour Market Development Agreement (LMDA), which entered into effect on April 26, 1997, the federal and provincial governments agreed to share responsibility for active labour market measures for Employment Insurance (EI) eligible unemployed in the province. In accordance with this Agreement, the two governments are responsible for designing and delivering programs that have goals and objectives similar to the active Employment Benefits and Support Measures (EBSMs) identified in the Employment Insurance Act. Under this flexible partnership or co-management model — wherein Canada and PEI have equal roles in design, management and evaluation — the federal government retains responsibility for the financing and delivery of Employment Insurance benefits, measures, and services. The employment benefits reviewed during the formative evaluation were the Targeted Wage Subsidies (TWS) program, the Self-Employment (SE) program, the Job Creation Partnerships (JCP) program, Purchase of Training and Skills Development. The support measures that were evaluated included Employment Assistance Services (EAS) and Local Labour Market Partnerships (LLMP).

Evaluation Methodology

The formative evaluation was conducted between December 1998 and October 1999, though the final analysis and reporting continued until March 2000. The evaluation employed multiple lines of evidence to assess issues pertaining to the rationale, design and delivery, impact and success of EBSMs, as well as EBSM-specific issues. Data collection methods included:

• a series of 30 in-person key informant interviews with members of Labour Market Development Agreement (LMDA) Committees and Working Groups, various government staff and stakeholders;
• a total of 12 focus groups were conducted with Human Resource Centres of Canada (HRCC) and provincial delivery/front-line staff, stakeholders, clients and employers;
• a review of program-related documentation, literature and administrative data; and
• telephone surveys of 1,164 EBSM participants and 485 non-participants.

Key Findings

Relevance

The evaluation evidence indicates that, for the most part, the Canada/PEI LMDA and EBSMs are relevant to the needs of the targeted industry sectors, employers, communities and Islanders. The LMDA focuses on resource and seasonal industries (e.g., agriculture, fishing, and tourism), the staples of the Island, as well as on targeted high-value industries such as information technology and aerospace, which have the potential to create long-term jobs for the province. Still, there were some opinions expressed that the LMDA needs to be better focused on the unique needs of PEI (e.g., programs should be better adapted to the seasonal economy, high level of unemployment, and low annual earnings of Islanders). Moreover, there was a pervasive concern that the relevance of the EBSMs is limited by the restrictive program eligibility criteria imposed by the EI Act. This concern appears to stem from a widely held misperception that the mandate of the LMDA encompasses the entire PEI labour force, when in fact the Agreement is designed to serve EI clients only. In any case, the focus of EBSM assistance on EI eligible clients was thought to leave some major gaps in needed programming. In particular, many respondents felt that small business development, skills upgrading for currently employed or under-employed Islanders (in part to supply the skills required in new targeted industries), and the needs of youth and persons with a weak attachment to the labour market are not being adequately addressed by the LMDA or any other initiatives.

There was some conjecture among key informants on the issue of focusing on seasonal industry. On the one hand, taking measures to extend the duration of the work season in seasonal industries was seen as acceptable given that these industries will be a reality on the Island for the foreseeable future. Still, respondents holding this view also noted that seasonal jobs should be seen only as stepping stones to longer term employment, that seasonal workers should be equipped with multiple skills to make them portable, and that there is a need to promote industries complementary to seasonal industries. On the other hand, the majority of key informants suggested that the emphasis for the LMDA should be on developing new year-round industries so that the Island can end its dependence on seasonal industry.

On the issue of the complementarity of LMDA and other programming, federal and provincial officials at the senior management level generally felt that the EBSMs are fairly well harmonized with other programs, though at the middle management and front-line levels many respondents perceived there to be work needed to resolve issues related to duplication, overlap and a lack of coordination among federal and provincial programs (e.g., wage subsidies, self-employment and youth programs). It should be noted, however, that these areas of overlap existed before the signing of the LMDA in 1997.

Design and Delivery

Considering the complexity of the task, the implementation of the Canada/PEI LMDA has gone reasonably well to date, though further development and improvements will be needed. On the positive side, the high degree of cooperation and collaboration among all LMDA players, the delivery of successful initiatives such as adult basic education/literacy and aerospace, and HRCC staff's understanding of their role in EBSM delivery were all noted by interview and focus group respondents as successful aspects of LMDA implementation. On the other hand, a lack of reliable, timely labour market information and client tracking information for LMDA planning and management, excessive administrative requirements and the associated delays in project approvals, and somewhat of an HRCC staff shortage were regarded as key weaknesses that will require attention. Also, both federal and provincial managers identified the lack of technical resources to support needed LMDA information systems as a problem. Although productive partnerships with community organizations have been developed and despite the fact that local-level consultations with stakeholder organizations had been conducted during the development of the 1998-1999 LMDA Business Plan, stakeholders as well as front-line HRCC staff felt that consultations and partnerships with "grass roots" community organizations still need to be improved. Moreover, there is a need to increase awareness of the LMDA and EBSMs at the community level.

The EBSMs were generally regarded as flexible and responsive to local and client needs. Interview and focus group respondents had some reservations, however. In particular, respondents identified a need to further refine and adapt the programs to the unique economic needs of PEI (e.g., many seasonal industries, relatively high unemployment, and low annual earnings), to harmonize LMDA and social assistance programming, and to better serve and inform clients about available programs. The perceived strengths of Human Resources Development Canada (HRDC's) approach to delivering the EBSMs include a cooperative and positive effort on the part of highly experienced HRCC staff, flexible and decentralized program delivery, and a reasonable amount of paper work for clients/funding recipients. On the other hand, some clients and stakeholders perceived that service delivery is inconsistent from one HRCC to another and that the service from HRCC staff (and the HRCC environment as a whole) is unwelcoming and lacking in empathy.

In the survey, clients indicated being most satisfied with the quality of education or training they received and with the knowledge of Employment Counsellors, but comparatively less satisfied with the quality of referral services and with the information available to help them choose suitable programs. In addition, LMDA programs and services are being successfully delivered in both official languages as intended. Only a small minority of survey respondents (three percent or 28 clients) indicated that they were not able to get program information in their preferred language.

Federal-Provincial Partnership

The federal-provincial partnership has been working reasonably well, though some "growing pains" are still being experienced. Both federal and provincial managers identified several strengths of the partnership and the co-management approach. In particular, these respondents perceived a high degree of cooperation between partners and noted that co-management facilitates collaborative decision-making, mutual understanding and coordination of federal and provincial initiatives. They acknowledged that the partnership does take a lot of effort and compromise, however. For example, front-line HRCC staff observed that the partnership has been difficult and frustrating at times, and that the degree of cooperation between the two levels of government could have been better in the pilot of Skills Development. Moreover, as already noted, most respondents observed that co-management adds another layer of bureaucracy and complexity to the LMDA, resulting in delays in project approvals.

Many HRDC staff have been reluctant to embrace the new co-management approach due to their concern that program delivery may eventually be fully devolved to the Province, which causes them anxiety over their job security. Although most senior managers felt that the LMDA is generally compatible with broader government objectives, provincial key informants asserted that the EBSMs need to be further adapted to better match the objectives of the provincial government.

Success to Date

The results of the accountability target attainment computations indicated that the EBSMs exceeded both the 1998/99 return-to-work and unpaid EI benefits targets. Despite exceeding the return to work target, questions were raised about whether or not all returns to work were being captured by the information systems.

Qualitative evidence of impacts on participants indicated favourable employment outcomes, if not immediately, then expected in the long term because of positive skill and psychological (e.g., self-confidence, self-esteem) impacts. Still, less than one half of survey respondents said their EBSM intervention was important in attaining their current job although this reflected a more positive perceived impact relative to the comparison group. Employers consulted in focus groups agreed that skill and psychological impacts are occurring, although a number said they were often unable to retain their wage-subsidy workers because of financial difficulties.

Quantitative evidence from the survey indicated that at this stage of evaluation, only Self-Employment led to consistently positive employment and income-support outcomes. TWS and Training/Feepayer (TFP) increased the likelihood of being employed for 12 consecutive weeks. EAS, TWS and JCP produced negative earnings outcomes. More conclusive evidence of EBSM impacts on employment and income support will be available at the summative stage of evaluation.

Results were mixed for perceived employer impacts. On the one hand, employers in the focus groups said that TWS relieved some employers of cash-flow pressures in hiring and that training unskilled workers enabled some businesses to be sustainable and even competitive. On the other hand, some employers were unable to retain workers because of cash flow problems and others said the EBSMs would be unable to channel workers into sectors and occupations where they were in demand. In addition, some employers in certain areas (e.g., seasonal, information technology, new/small businesses) did not think the LMDA as implemented could meet their specific needs because of perceived shortcomings in the rules. Some employers believed the EBSMs would be insufficient to effect a change in attitudes in PEI with respect to valuing training and dependence on seasonal industries and income support. Finally, some employers had problems with the exclusion of certain groups (e.g., non-EI eligible unemployed persons) who would have been able to fill their needs.

For impacts on the community, some key informants and focus group participants felt it may be too early to address this issue. Among those who were able to, there were mixed results. Some respondents felt the LMDA may have helped in addressing short-term needs of communities and Islanders, but there was concern that the exclusion of non-EI eligible persons may limit the LMDA's ability to address long-term needs. Others mentioned that, to truly benefit the province, there needs to be greater emphasis on interventions providing labour market development than on those providing job creation and wage subsidies. On the other hand, some respondents spoke of the benefits of the LMDA for the community, including the extension of the "shoulder" season, community learning centres, lifelong learning and adult literacy training measures, the technology mentoring program, lower social assistance (SA) caseload, and partnerships between communities and the government.

Rural-Urban Differences

A review of the qualitative and quantitative evidence indicates that there were few clear rural-urban differences in terms of perspectives on the LMDA and in its impacts. With respect to the latter, multivariate analysis indicated that the EBSMs had positive impacts on rural residents in terms of the likelihood of seasonal employment, the percentage of weeks employed and three consecutive months of employment, but were beneficial for urban residents in terms of full-time employment. In addition, they produced negative effects for rural residents in terms of earnings. We also attempted to contrast the views of focus group participants located in urban, rural and isolated rural centres, but were again unable to observe any clear pattern.