Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

3.1 Overview

Views on the degree of success of Labour Market Development Agreement (LMDA) planning and implementation, as expressed in key informant interviews and focus groups, are presented in this chapter. It is important to identify the strengths and weaknesses of LMDA implementation to date so that needed improvements can be identified. This information will allow LMDA management to respond to problems early in the implementation phase so that the Agreement and associated Employment Benefits and Support Measures (EBSMs) are ultimately as successful as possible in achieving their objectives.

Key findings are as follows:

• Strengths in LMDA implementation to date include the high degree of cooperation among the federal and provincial government partners in the Agreement; the implementation of promising initiatives, such as the adult basic education/literacy initiative; and the fact that Human Resource Centre of Canada (HRCC) staff have a good understanding of their role in delivering the EBSMs.
• Major weaknesses are a lack of information to support LMDA planning and management (i.e., useful labour market information, valid measures for tracking the progress of clients); excessive administrative requirements and associated delays in the approval of project applications; and somewhat of a staff shortage at HRCCs due to federal downsizing. In addition, there is a perceived need to improve the promotion of the EBSMs in communities and to strengthen the consultations and partnerships with "grass roots" community organizations so that they have more input into LMDA planning and implementation.

3.2 Most Successful Aspects of Implementation

In the view of many key informants with both the federal and provincial governments, the major strength of the LMDA implementation has been the high degree of cooperation, communication and collaboration among all LMDA players, including good federal-provincial cooperation in the co-management of the Agreement. Government managers and staff as well as stakeholders cited some examples of successful LMDA initiatives, including the adult basic education/literacy initiative, the aerospace initiative, Self-Employment and Targeted Wage Subsidies. In addition, front-line HRCC staff noted that they have a fairly good understanding of their role in the delivery of the EBSMs because these benefits and measures are similar to the former Human Resources Development Canada (HRDC) programs.

3.3 Least Successful Aspects of Implementation

Key informants perceived there to be numerous difficulties with the implementation of the LMDA to date. Chief among federal and provincial managements' concerns was a lack of reliable, timely information (i.e., useable labour market information and data tracking Employment Insurance (EI) clients) to support LMDA planning, management and evaluation. In addition, managers expressed concern about the somewhat excessive administrative requirements and the slow, cumbersome approval process due to the fact that major decisions under the co-managed LMDA require negotiations and agreement between federal and provincial partners on the Management Committee and on the Operations Committee.²⁵

HRCC managers and staff had some unique concerns about the LMDA implementation. HRCC managers are having some difficulty adjusting to the federal-provincial approval process and to the fact that they no longer have the authority to approve programming exceeding $75,000 in their local area. On occasion, HRCC managers have felt that these decisions, made by the Management Committee, have not been ideally suited to the needs of their area.

For front-line delivery staff at HRCCs, the administrative requirements (and the added level of bureaucracy) associated with the LMDA have created some confusion and have increased their workload. Also, the interviews and focus groups indicated that many HRCC staff are anxious about the possibility that EBSM delivery will eventually be fully devolved to the provincial government (as it has been in some other provinces), and they are concerned that this could result in the loss of their jobs. As one key informant put it, "full devolution hangs over our heads." Front-line employees' anxiety about their job security is a serious issue, considering their crucial role in the delivery of services at HRCCs. It will be important to ensure that staff anxiety does not interfere with the delivery of services under the EBSMs.

In addition, front-line staff in the focus groups made the following observations about LMDA implementation: not enough emphasis is placed on proper employment counselling (to screen clients and point them in the right direction) and on encouraging clients to take responsibility for their own progress toward employment; and a belief that the Management Committee is not sufficiently aware of client service issues at the "grass roots" level and of the possibility of overlap between EBSMs and other programs.

Similar to the government respondents, stakeholders expressed some confusion and frustration over the slow, complicated application and approval processes associated with the co-managed LMDA. Many are uncertain about the respective responsibilities of each level of government. Moreover, consistent with the findings of the 1998 national EBSM evaluation, key informants representing a variety of community/stakeholder organizations (i.e., youth, persons with disabilities, business and education) observed that awareness of the LMDA is quite low in communities, suggesting that communications and promotion of the EBSMs could be improved. In focus group discussions, some stakeholders perceived that the public and business community on the Island are somewhat cynical about the ability of government to "get it right" with respect to delivering the EBSMs and meeting clients' needs. For instance, employers using Targeted Wage Subsidies expressed a need for better matching of clients with suitable occupations, and for more background on a client's work history so they can better assess the client's capabilities and suitability. Also, some stakeholders were of the opinion that there were insufficient local-level consultations regarding the LMDA (despite the fact that consultations had been held in all local areas for input into the development of the LMDA Business Plan).

3.4 Partnerships with Community Organizations

Both federal and provincial key informants observed that there are several productive partnerships between government and community organizations related to the LMDA. Some factors that have hindered the development and maintenance of good partnerships were also identified, however. In particular, LMDA-related changes in the decision-making and approval procedures for program funding, downsizing and staff turnover at the Human Resource Centres of Canada (which can result in inconsistent program delivery), and changes to the Employment Insurance Act have created some confusion and difficulties for community organizations in their efforts to work with government. Also, due to some confusion about the respective responsibilities of the two levels of government, community organizations have sought assistance from both federal and provincial offices, sometimes attempting to play one off against the other to their advantage (though it was noted that this was also done prior to the LMDA).

Although some stakeholders acknowledged that there have been productive partnerships and that local community organizations (including the francophone community) have been consulted for their input into the LMDA Business Plan, several stakeholders identified some problem areas that hinder partnerships. Many stakeholders (representing youth, industry associations, community education groups, and people with disabilities) identified a need for further improvements in local-level consultations and more community input into LMDA planning and implementation. Similarly, in the national EBSM evaluation, an overall finding was that community partners felt they had been consulted on local labour market plans but not on the mix of benefits and measures to be delivered. In the present study, organizations representing persons with disabilities also called for more flexibility in EBSM eligibility criteria to accommodate different client groups with unique barriers to employment. The adult basic education/literacy community felt inadequately consulted by government, and was not supportive of the fact that Holland College was awarded the contract to administer community-based programs for the delivery of adult literacy training.

In the focus group discussions, some stakeholders observed that good partnerships have existed with HRDC for some time and that they have continued under the LMDA. On the other hand, others perceived that there is room for improvement in partnerships between HRDC and "grass roots" community organizations, an observation also made by some front-line HRCC staff. In the view of stakeholders, partnerships have been hindered by: a resistance on the part of HRDC to treat stakeholders as equal partners; a lack of trust between government and community groups; LMDA "growing pains" and confusion concerning areas of responsibility; a lack of communication with stakeholders regarding program objectives; and insufficient bilingual resources/services at HRCCs, which has meant that some francophone stakeholders are not always served in their preferred language.²⁶ In the national EBSM evaluation, community organizations also indicated that more effort needs to be devoted to the development of partnerships.

3.5 Adequacy of Available Resources

In the view of both federal and provincial key informants, the primary resource issue with the LMDA pertains to the lack of technological resources to support the information systems needed for proper planning, management and evaluation. With respect to human resources, it is perceived that federal downsizing has created somewhat of a staff shortage for delivering the EBSMs — an observation also made in the 1998 national EBSM evaluation. The number of staff was reduced on the promise of more technology (to compensate for having fewer staff), but the technology has yet to be delivered. Moreover, the existing technology does not have the capability of real-time management information as requested by the LMDA Management Committee nor will it permit easy input, tracking or evaluation.

Regarding financial resources for the EBSMs, federal respondents felt that, although the budget is very tight, it is probably sufficient at present. On the other hand, provincial officials argued that more financial resources would be helpful, particularly for training. In addition, some provincial respondents questioned why LMDA funds lapsed in one province could not be transferred to another province (like PEI) where the funds could be put to good use. There are no mechanisms at this time to permit the interprovincial transfer of funds.

Stakeholders representing persons with disabilities felt that EBSM funds provided for hiring staff and client monitoring are insufficient, and that the one-year program contracts are too short for them to plan and provide a sustainable service to clients. These respondents preferred the previous three-year contracts. In addition, community education representatives perceived that HRDC has less flexibility to fund community organizations under the LMDA than it had in the past.

3.6 Adequacy of Existing Information and Monitoring Systems

In the key informant interviews, senior and middle managers as well as staff with both the federal and provincial governments agreed that the current information monitoring systems are inadequate and that the difficulty in obtaining reliable, practical information on the labour market/labour force and on EI clients' progress impedes proper planning, management and evaluation of the LMDA. The LMDA includes a provision for the creation of a working group to identify the information needs and system connectivity requirements for implementation of the Agreement, but no funds were specifically allocated for these purposes. As an illustration of this issue, some provincial respondents described a case where they asked for some LMDA performance measurements and received three different estimates within a one-year period. It was also discovered during the development of the survey samples that a reliable flag to distinguish client and reachback participants does not exist in the administrative database. As a result, provincial respondents have little confidence in the validity of the currently available performance measures due to shortcomings in HRDC's information systems. Key informants noted that these limitations with information systems are not unique to PEI, but are a problem across the country.

Several aspects and causes of this problem were identified, including the following:

• The Province does not have expertise in the type of information monitoring system required for the LMDA, nor has HRDC in PEI been required in the past (prior to the LMDA) to produce this type of information.
• The Province cannot access the National Employment Service System (NESS) and HRDC cannot access provincial information systems, due to a lack of a federal/provincial information-exchange agreement. Such an agreement would be developed in accordance with privacy legislation and departmental guidelines. At co-located HRCCs, HRDC staff handling EI and provincial staff working with Social Assistance could benefit from access to each other's information systems because they share many clients on the Island who need to rely on both EI and SA. In focus group discussions, front-line staff expressed frustration at not being able to track a client through the entire system because HRDC holds the client data at certain stages whereas the Province does at others. Some key informants suggested that a new integrated federal-provincial client database be developed to address these problems.
• In the federal system, only the top priority client information is recorded (e.g., EI payments to clients); but other potentially useful data, such as clients' level of education, occupation code and industry sector code, are not being entered consistently. Moreover, local offices feel the need for clear directions from LMDA management on what information should be recorded.
• HRDC case managers are very busy and sometimes have little time to devote to data entry and coding, and as a result, data integrity may suffer.
• The information in the Contact IV system is limited, and third-party delivery agents and staff may not take the time to enter the data properly. There are insufficient resources and time for HRDC staff to properly monitor the third-party agents.

A number of third-party stakeholder organizations (representing youth, self-employment clients and the training community) claimed in interviews that they do collect some useful data on client characteristics and outcomes. Other stakeholders perceived that there are problems with the information monitoring, however. For instance, it appears that some organizations, depending on the nature of their activities, are required to complete more extensive monitoring than others and this can result in inconsistencies. Some stakeholders also argued that LMDA monitoring should be more comprehensive, incorporating the results of follow-up surveys with clients, narrative reports of client progress, information on the full range of interventions in which clients participate (not just those delivered by HRDC), and a broader range of outcome measures (e.g., educational achievements, lifestyle changes and volunteer as well as paid employment).

3.7 Summary

On balance, given the complexity of implementing the LMDA, things have gone reasonably well so far. The high degree of cooperation and collaboration among all LMDA government partners, the delivery of some successful initiatives (e.g., adult basic education/literacy initiative, aerospace initiative), and HRCC staff's good understanding of their role in EBSM delivery were all noted as successful aspects of LMDA implementation to date. On the other hand, a lack of reliable, timely information to support LMDA planning and management, excessive administrative requirements, and somewhat of an HRCC staff shortage were regarded as key weaknesses that will need to be addressed. In addition, although many respondents acknowledged that productive partnerships with community organizations had been developed, stakeholders as well as front-line HRCC staff felt that consultations and partnerships with "grass roots" community organizations need to be improved. Moreover, there is a need to increase awareness of the LMDA and EBSMs at the community level.

Footnotes

25	The LMDA Operations Committee, which initially reviewed strategic projects and proposals of over $75,000 before they went to the Management Committee, has not met since the fall of 1998 in order to streamline the application and proposal review processes.
26	As is discussed in Section 4.8 of this report, only three percent of program participants who were surveyed indicated that they were not able to obtain program information in their preferred language (which was French for over half of these 28 survey respondents). This result indicates that HRCC services are easily accessible in French.