Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

5.1 Roles and Responsibilities

Clarity of Roles and Responsibilities

Mixed opinions were expressed regarding the clarity of partners' roles and responsibilities in the LMDA. Although the majority of respondents in interviews, focus groups and case studies felt that their LMDA roles and responsibilities are basically well understood and satisfactory, many pointed to areas of confusion. For example, former federal staff who were transferred to the Province are still adjusting to their new role and environment/culture, while some provincial staff feel that the former staff are still in "federal mode". Also, there is not a clear understanding of federal versus provincial roles and responsibilities for the reception function and for the service delivery model at co-located sites. These and other areas of confusion appear to be due largely to inadequate communications both within and between levels of government.

Some HRDC officials expressed concern about a lack of clarity around roles and responsibilities across the board. For instance, one key informant observed that an overall structure and process for implementation does not exist, but that this is necessary for the success of something as complex as the LMDA. Another argued that there is not yet a clear understanding of how the federal government will manage its business in this new environment and what role it will play with regards to the LMDA (e.g., "We don't necessarily want to be policing the Province, but at the same time, there are a number of issues and concerns that we have in terms of legislation"). It was also perceived that HRD-NB employees in regional sub-offices may be unclear about their accountability requirements.

On the provincial side, officials noted that there has been some confusion regarding the respective roles and responsibilities of HRD-NB and the Department of Labour for Skills Loans and Grants (SLG). Provincial key informants also observed that there is some confusion over LMDA accountability responsibilities, and over federal responsibilities in program areas perceived to overlap somewhat with provincial programs (e.g., the federal pan-Canadian programs) as well as which level of government should first serve youth, persons with disabilities and Aboriginals.

Another related issue noted in interviews and case studies is the fact that some former HRDC employees transferred to the Province did not have their seniority recognized by the Canadian Union of Public Employees (CUPE), the union representing the Province. To make matters worse, these staff sometimes work alongside other employees whose seniority was recognized. This state of affairs has had a negative impact on their morale, and they are concerned about their job security.

Roles for Additional Players

The consensus was that the current players/partners — HRDC, HRD-NB and the DOL — are probably sufficient at this stage, and that the LMDA is complicated enough at present. Later, after the LMDA is fully implemented, there may be room for additional players to play a bigger role in LMDA consultation, planning and design. These partners might include private sector employers, the not-for-profit sector, local community organizations, and economic development commissions.

5.2 Implementation of the LMDA

Most Successful Aspects of Implementation

Several aspects of the LMDA implementation were regarded as successful by key informants, focus group participants and case study respondents. In particular, there has been good cooperation between the three players and two levels of government, and a willingness to work together in the field through the local implementation teams and regional working groups (which included the Canada Employment and Immigration Union (CEIU), a component of the Union of the Public Service Alliance of Canada (PSAC)). In the view of some respondents, having control over the programs and having field staff in communities has improved local flexibility and the pursuit of regional economic development priorities. At the outset, the transfer of funds through the federal authorities went quite smoothly as did the transfer of federal staff. There were good communications, sound change management and careful attention to HRDC staff being transferred to the Province who received a good package in terms of money and protection. Moreover, program delivery and good client service were considered to have been maintained throughout the process. The fact that the LMDA has become functional, in spite of the technical complexity of the task (three departments with different information technologies), was viewed as an achievement in itself.

Least Successful Aspects of Implementation

Respondents from key informant interviews, focus groups and case studies also identified numerous aspects of the LMDA implementation which, in their view, were not so successful. Their observations are summarized in this section.

Both federal and provincial officials commented on the negotiation process. Due to pressures on both the federal and provincial sides to finish negotiations and get the Agreement signed, the LMDA was signed hurriedly, with the understanding that many of the details (which were presented in general, management terms/concepts rather than in practical terms) would be worked out at the implementation stage. However, many of these details remain to be worked out, creating confusion and frustration for all players. A provincial official also cited the exclusion of the Canada Employment and Immigration Union (CEIU), a component of PSAC, in the negotiations as a weakness.¹¹ More broadly, there have been some problems and confusion for the public associated with the fact that three distinct events happened simultaneously — the passing of the EI Act and its changes, the reduction of the federal government's consolidated revenue fund (thereby significantly reducing the funds available for non-EI clients), and the devolution of programs to the Province through the LMDA.

Some disruption was created for staff because of an unanticipated delay — due to negotiations between HRD-NB and CUPE — between the first wave (in July-August 1997) and second wave (in February 1998) of federal staff transfer to HRD-NB. This delay caused problems for affected staff, who felt anxious, stressed, and as though they were in a "no man's land" with no direction because they were still federal employees with federal managers but were working with provincial programs. Moreover, this interrupted some good momentum which had existed at the time. In addition, the delay in negotiating an agreement on transferred staff's pension (i.e., negotiation between the federal Treasury Board and the provincial Department of Finance) was another weakness — this has not been completed yet, but is a big (emotional) concern for staff.

Additional problems related to the staff transfer were identified by study respondents. It was perceived that the Province probably was not totally prepared for the changes (i.e., in terms of having the proper systems and training in place). Similarly, staff did not feel adequately prepared and trained for their new role and the PBMs. In addition, provincial employees felt that federal employees being transferred were receiving special treatment (good compensation package, etc.), while some provincial managers did not feel that they got the best (former) federal employees. There has been "culture shock" for both federal and provincial employees and difficulty adapting to a new environment for transferred staff. Finally, the issue of the lack of provincial recognition of the seniority of former federal staff has not been properly resolved, and there is some remaining uncertainty and insecurity with respect to what will happen at the end of staff's three-year job security agreement.

Resources have also been a source of disagreement in the LMDA implementation. On the one hand, some federal officials perceived that the Province is not contributing enough effort/resources in the partnership, considering that HRDC has transferred so much to the Province. On the other hand, HRD-NB officials complained about the lack of resources to cope with numerous costs associated with co-location and the addition of 35 per cent more staff to their department — costs which were not fully thought through at the time of signing the LMDA. There were disagreements over some of the financial interpretations that the federal authorities made under the LMDA (e.g., number of positions to be transferred and salaries). These concerns were echoed by DOL officials.

Some problems particular to SLG were identified. The transfer of programs over to the Province to produce seamless service and one-stop shopping has not yet been successful in the case of SLG, and will not be achieved until the co-located sites are all in place. Clients are still confused about "what door" they need to go to for SLG funding. In addition, the recruitment of qualified staff for SLG has been disruptive. In the view of provincial officials, the program was not described accurately and positively, and little interest was shown from suitably skilled (former) federal employees. It has been an "arduous" process attempting to fill positions with transferred federal employees, and recruitment has been very difficult. Related to this, case study respondents noted that case managers could have benefited from more training in financial assessments and the negotiation of clients' financial contribution (which are perceived to be done inconsistently), and that the services of an expert in the delivery of large, complex programs (like SLG) would have facilitated implementation. Also, the delay in implementing the loan component of SLG has caused difficulties for client service.

More generally, inadequate communications and coordination within and between levels of government have impeded implementation. In addition, communications to worker and employer clients about the changes (they need to know about) have not yet been done. There has been a less than optimal connection between worker clients' needs and the delivery of active benefits and measures (e.g., clients may be on waiting lists, and may be confused as to who should be helping them). Also, the new case managers come from a variety of backgrounds, for example, a hard-nosed employment counselling focus versus a social counselling focus where a supportive, accommodating approach is the norm. Because these case managers are not a uniform group, it is unreasonable to expect consistency in their approach and decisions at this early stage of the LMDA.

Implementation Plans

Joint implementation planning was done by the senior management Joint Implementation Committee, and this was helpful in terms of sorting out roles and responsibilities. This planning also involved: the design of a collaborative planning framework for the local level (though this was not distributed due to delays in co-location); the setting of results targets, using national benchmarks initially with the understanding that these could be refined later if necessary; and the development of a monthly reporting format for the Province. In addition, jointly developed implementation plans were prepared by other committees at more junior management levels and by local implementation committees, and local joint planning meetings were held for a while. There have, however, been many delays in implementing plans (e.g., delays related to the transfer of federal employees, problems related to resources and incompatible information systems, delays in co-location). Still, the delivery of the PBMs has been proceeding since the beginning.

Some joint planning was attempted regarding the information technology aspects of the LMDA, with the assistance of a consultant, but this was not implemented as designed. What has been required is a series of revised plans and weekly meetings to track progress. Systems officials with all partners identified the lack of a dedicated project manager - to oversee and coordinate the overall LMDA implementation process, facilitate communications among the three players, and deal with the various problems which arise - as a barrier to successful planning and implementation.

Further problems with the planning process were identified. For instance, key informants felt that there has been poor communications and management of the planning process — that it has not been "process mapped" or "project managed" - and that there has been no clear step-by-step plan and schedule for implementation. Also, there have been difficulties in integrating the planning processes of the three players, and insufficient infrastructure for proper planning for LMDA implementation.

Disruption to Staff and Clients

Given the complexity of the task of implementing the LMDA and PBMs, there has been remarkably little disruption to client service in the view of many respondents. Reasonably good service has been maintained and there have been very few serious complaints. One exception has been the disruption to client service due to the delay in implementing the loan component of Skills Loans and Grants. Clients have had to keep calling back to ask when the full SLG program and the loan component would be available, but have not been given a definite answer, which interferes with their educational planning for the coming year (because they are unsure if they will have sufficient funds for the duration of their program). In addition, delays in co-location have created some confusion for clients, who may be sent to many different places for assistance (e.g., the HRCC, HRD-NB and job finding clubs).

The LMDA implementation process has been somewhat disruptive for staff. Although the initial transfer of federal employees apparently went quite well (e.g., in terms of communications, change management and the financial package), some staff now working for the Province feel isolated and confused, and are having difficulty adapting to their new environment and corporate culture. In addition, there are concerns about job security and the recognition of seniority for transferred federal staff, as discussed earlier. Also, many staff continue to feel the need for clear communications regarding the PBMs and their new roles and responsibilities. For instance, a single, automated, user-friendly information system on the new programs would be very helpful for front-line staff.

A number of issues for HRD-NB staff were raised in focus group discussions. It was noted that HRD-NB staff only have partial access to EI information and this presents difficulties for HRD-NB counsellors, who cannot follow up on an EI applicant's file because they now have fewer organizational and communication links with HRDC staff responsible for EI files. Another perceived problem is that HRD-NB does not fully understand the EI clientele, which is different from the social assistance clientele. For example, many EI clients are seasonal workers, so their need for employment programs is concentrated in certain periods of the year. HRD-NB staff did not realize that many EI clients do not require employment programs during the summer because they typically find employment independently in that season, yet require assistance during the winter. In addition, in the case study of Employment Assistance Services (EAS), HRD-NB staff expressed concerns about their lack of experience (and staff) for their new responsibilities, which include program administration and the management of contracts with third-party organizations.

The HRD-NB counsellors found that they were not reaching their target clientele because the counselling services they offer are voluntary and EI clients were not seeking counselling unless they were referred by HRDC staff. A functional referral system had yet to be implemented. Furthermore, the HRDC staff working in EI are behind in their work and do not have time to coordinate with HRD-NB. Another issue pertains to case loads. HRD-NB tends to judge the performance of the transferred personnel (who are still working in HRCCs) by comparing their case load with that of the HRD-NB staff who take care of social assistance. A problem with this comparison, however, is that they are not the same clients — clients on EI come much more often. So the transferred staff see a lot more people per day than the HRD-NB staff who take care of social assistance recipients.

Cooperation from Partners and New Partnerships

There has been a history of good cooperation between the federal and provincial governments in New Brunswick, and the LMDA has provided an opportunity to strengthen the partnerships among HRDC, HRD-NB and DOL. The New Brunswick Labour Force Development Board (with labour and management representatives) was also consulted regarding the LMDA implementation. In addition, consultations have been done with community organizations, though some respondents in the evaluation felt that these could be improved.

Views were mixed on the degree to which the LMDA has helped to create new partnerships up to this point. Some key informants and focus group participants perceived that new partnerships have been developed, including partnerships between: provincial departments and regional development associations and industry associations (e.g., trucking); SLG officials and industry/trade groups and the community colleges; and the DOL and Department of Economic Development, Tourism and Culture. In addition, case study respondents noted that they have received good cooperation and in some cases developed partnerships with various community and third-party delivery organizations. On the other hand, many respondents in the evaluation did not think that significant new partnerships have been developed at this stage.

Adequacy of Monitoring Measures

The evaluation results indicate that there are some serious problems related to the adequacy of current information systems for the monitoring of program delivery and results. There appears to be a need for clarification on accountability requirements, for a clear definition of valid results measures, and for better integration of the (currently incompatible) information systems operated by the three players. There are also related client privacy issues because client information will need to be shared among the players. Systems officials feel the need for direction on these points from program management.

In interviews, some HRDC officials expressed concern about the adequacy of the monitoring done by the Province, noting that accountability does not appear to be a primary concern of provincial authorities. For instance, some respondents had not received the quarterly reports on results which the Province is required to produce. Monitoring is also a problem for the Department of Labour in that it does not do case management from the perspective of workers, but rather only deals with employers. In addition, at the local level, planning is a crucial role for staff. In order to do this properly, however, good local-level information is needed but not currently available. This problem is compounded by the fact that the Province does less local-level planning than HRDC does, and the timing of planning differs between the two levels of government.

HRD-NB officials argued that there is a need for one integrated information system (or at least compatible systems) that all players can agree on, clear information requirements and agreement on how this information is to be used. A concern was also expressed about the lack of comparison or benchmark measures for results monitoring, given that the PBMs (in particular, SLG) are sufficiently different from the previous programs to make comparisons difficult. Skills Loans and Grants staff are able to track their own clients through their internal financial data system (which is useful for process evaluation), and SLG client monitoring is done primarily through the Department of Labour system and NESS, but also through the HRD-NB case management system.

Department of Labour officials complained that the Department never received an adequate explanation of the rationale/basis for the LMDA results targets. Accountability regarding LMDA results and financial information remains unclear. In the view of these officials, adapting the existing information systems to collect valid results measures, and related issues such as gaining security access to the federal systems, will be an ongoing problem for some time.

DOL respondents also observed that existing systems cannot currently provide accurate reports of activities at the local level. Moreover, there is a time lag between an employer hiring employees and entry of this information into the system, and results information can get lost due to the incompatibility between systems (e.g., the information might be entered into one system, but a report generated from another incompatible system). With respect to results, the employers are really "in the driver's seat" because they decide who they will take on as employees. Consequently, field staff with the DOL feel that they have little control over these results.

Like the key informants, focus group and case study respondents identified some problems with information monitoring systems. For example, HRD-NB staff have lost some screens to which they had previously had access, even though they have remained working at the same desk. Also, former federal employees now at HRD-NB are experiencing stress because they are told to keep the funds allocated to clients to a maximum amount per client, but cannot properly keep track of this in the absence of regular, accurate monitoring of client expenditures. In the case studies, several respondents noted that there is low awareness among program staff of the results targets and/or how these are being monitored. On a positive note, however, third-party monitoring of EAS clients was viewed as effective due partly to the fact that contractors are held accountable for the impact of their services on clients.

Remaining Issues for Implementation

The major outstanding issues for implementation of the LMDA were discussed earlier. These issues include: resolving the resource issues (e.g., for co-location); adapting and integrating the information and monitoring systems; defining accountability measures and resolving the privacy issues so that client information can be shared among the three departments; clarifying the roles and responsibilities of all staff, and improving communications on these matters both within and between levels of government; and completing co-location, which is discussed in the next section.

Some additional issues noted by respondents were as follows:

• the pan-Canadian programs and whether or not the federal government is going to devolve any further responsibilities to the Province;
• the income tax issue with Skills Loans and Grants (i.e., the fact that unemployed clients have to pay tax to Revenue Canada on their training assistance);
• completing the negotiations on the agreement for transferred staff's pension (i.e., negotiations between the federal Treasury Board and the provincial Department of Finance);
• issues related to meeting the needs of persons with disabilities and rural clients;
• improving the usefulness of Labour Market Information;
• how HRD-NB and DOL are going to case manage clients (i.e., clarifying which department is responsible for the client at various stages of case management and service delivery); and
• the need for the federal government to gradually let go of its previous role.

5.3 Co-location

Despite the fact that a number of barriers (e.g., disagreements over resources, inadequate and incompatible information technology and resistance to organizational change) have delayed the co-location process, the majority of respondents in Phase I of the formative evaluation had not lost sight of the ultimate goal of one-stop shopping and seamless delivery of services and believed that co-location will provide many benefits for clients. In Phase II of the evaluation, many of the same barriers were still being encountered, and while a few sites are partially co-located, only one site (in Fredericton) is fully co-located. While clients are not dissatisfied with sites that have undergone partial and full co-location, the HRSC personnel have not seen the kind of improvements in their business operations that would lead to improvements in service delivery and their confidence in the advantages of co-location is waning. Three sites with various configurations of co-location (from full to not at all) were evaluated in depth by case studies. Focus group discussions and interviews with key informants provided further insights into the issue of co-location at a number of different sites that were either partially co-located or not at all.

Obstacles to Co-location

One of the most frequently reported obstacles to successful co-location was the difficulty of resolving administrative and operational questions between the two levels of government (federal and provincial), and among the three different government departments. For example, differences exist between HRDC and HRD-NB regarding hours of operation, statutory holidays, norms for client confidentiality, client service methods (i.e., appointments versus walk-in service), and views on how primary reception should operate. Several respondents from both the case studies and key informant interviews thought that HRDC and HRD-NB should have harmonized their operations and client service methods before co-locating. Other respondents would also have preferred administrative and operational questions to be decided upon centrally before proceeding with co-location either partial or full co-location (questions like responsibility for administrative costs, space allocation, signage, how reception would function and responsibility for its associated costs).

A less frequently reported but important obstacle to the co-location process has been financial. Many of the buildings currently housing HRDC or HRD-NB are too small to accommodate all three departments. In addition, the departments often have long-term leases, moving and/or renovating costs have been deemed too high and, in some cases, HRDC and HRD-NB disagreed on the best location for a new co-located HRSC (e.g., a city centre location versus a lower rent location farther away). These financial obstacles are the principal reasons why two of the regional sites evaluated in the case studies during Phase II have not been co-located. One of those sites has been slated for co-location in the near future when funds become available, but at the other regional site, HRD-NB and HRDC have decided not to co-locate in part because of their long-term leasing arrangements.

Social and organizational obstacles to co-location were also found to exist. Lower levels of management and staff directly involved in program delivery identified cultural differences between departments which stem from the characteristics of the traditional clientele of each department and the types of services provided. Co-location has brought about a collision of these different organizational cultures and poses a few problems (e.g., different visions between departments on the type of service delivery model to be adopted, unwillingness of EI clients, social assistance clients and employer clients to mix; security measures of one department are not suitable for the image or type of service of another).

A lack of communications between departments was reported by both staff and clients as another barrier to co-location. Key informant interviews with staff revealed that staff found it difficult to obtain information on the other departments' activities and focus group respondents perceived that the departments did not communicate with one another on their programs and activities.

Technical obstacles such as incompatible information systems between the province and HRDC and a lack of adequate technical resources, such as fax machines and photocopiers for each department, were reported by staff members.

Resistance to co-location by staff and management has also been reported as an obstacle. There was reported low enthusiasm from federal employees to co-locate and DOL staff would have preferred to co-locate with the economic development groups rather than with HRD-NB and HRDC. Key informant interviews and one case study indicated that in several instances the province delayed making decisions on whether to co-locate or pulled out and then re-entered the co-location process illustrating their unwillingness to participate in the process.

Expected Benefits and Impacts of Co-location

Throughout the evaluation, the majority of respondents expected that co-location would produce a number of benefits and have an overall positive impact. The evaluation fieldwork indicated that where co-location had not yet been implemented, most respondents expected that service delivery would be improved by co-location. Clients liked the idea of one-stop shopping, and staff felt that client service would be improved by increased communications and co-operation between the government departments. Personnel in key informant interviews indicated that co-location had a great potential to improve client service if the sites became central information centres for all social programs and if new partners could be brought to the co-located sites (i.e., if more government services were offered under the same roof). They felt seniors and persons with disabilities would benefit from co-located sites which improved their access and that processes would be sped up by co-location.

Where full or partial co-location had taken place, the majority of respondents found that client service had either improved or had stayed the same. Clients reported that co-location provided one-stop service, easier access and greater convenience. Personnel noted that the pooling of resources for self-serve information services and for the resource centres has provided better services to the public, that some program applications (such as DOL's Partners program), are being processed much faster, and social assistance clients are receiving more information and better services. Many personnel thought however, that client service really had not been altered by co-location (either full or partial co-location).

A smaller proportion of respondents felt that service had declined (e.g., information on programs was difficult to obtain, employers did not know who to talk to, the number of visits clients had to make increased, and line-ups at reception were longer). More staff representatives than clients perceived a decline in service and those that did felt that this was mainly due to the transition period.

A second expected benefit of co-location was improved client flow, but this has apparently not been the case. Client flow has been disrupted for a number of reasons. Clients found the co-located layouts confusing, signage is lacking or confusing, and reception services lack adequate information and direction in order to efficiently assist clients and refer them to the appropriate department or person. A large proportion of respondents identified reception as a very important factor influencing client flows and that the infrastructure for reception should be revisited to make co-location work for all three government departments.

A third benefit that co-location was expected to produce was increased inter-departmental communication and increased access to information for staff on other departments' programs and activities. Many respondents who have experienced full or partial co-location found however, that there has been little inter-departmental communication or knowledge exchange and that the three departments are not able to share as much information on clients as they would like. Improving information exchange and co-operation between departments were identified by many staff representatives as ways of improving client flows and client service and of decreasing duplication and misuse of programs.

Respondents reported that co-location has had a negative impact on management and decision making processes because decisions must be made jointly by three departments at the co-located site. Administrative problems do not get resolved quickly and mutually satisfactory decisions are difficult to reach in a timely fashion.

Co-location has also created some problems in staff relations (for example, former federal staff regret losing their seniority in the devolvement to the province, and there may be some conflict among SAR Case Managers and EI Case Managers regarding job complexity and remuneration), but it has also led to sharing of resources for professional development.

In general, client representatives felt that co-location once fully implemented would benefit them while personnel representatives were more skeptical and felt that co-location had the potential to positively impact services only if co-operation between departments and direction on how to better implement co-location were improved.

5.4 Service Delivery

Effectiveness of Client Flows

In order to ensure convenient access to federal and provincial programs as specified in the LMDA, co-located employment centres were intended to be established in the province. At the time of this report, only one co-located site was in operation. In many of the other locations, DOL staff and the transferred staff (mostly former federal employees who are now HRD-NB case managers) worked from within an HRCC. Apparently, some of these changes have had an impact on client flow. One key factor is the reception area which in many cases is run by HRDC staff. Many respondents report that the referrals made by EI application staff to direct clients towards case management is not systematic, both in co-located and non co-located offices. It appears that this is primarily caused by organizational barriers and a lack of communication between HRDC staff (who process EI applications) and HRD-NB staff (who conduct case management). There is often a misunderstanding of the roles and responsibilities of the staff of the other departments, both in co-located (as mentioned earlier) and non co-located environments. These factors may explain why client flow is not optimized and partly explain why in many offices, there are fewer people participating in programs such as SLG (according to respondents). It is generally thought that co-location will improve client flow; the experience in the active co-located site tends to show, however, that organizational barriers can remain despite the fact that all three departments share a common building.

Client flow problems were reflected in some of the comments provided by focus group participants (program participants and employers). The majority found out about the program in which they participated through word of mouth or through another party. Many clients mentioned that the programs are not well known among the general population. Clients and employers generally agreed that the programs are not well advertised by the Departments. According to some staff members, many clients were confused about the roles and responsibilities of each Department, especially in the months that followed the transfer of the federal employees to HRD-NB.

Flexibility and Responsiveness to Client Needs

From many viewpoints, programs are believed to be more flexible than before the LMDA. EAS is considered to be very flexible because services are generally offered to all unemployed people not just EI claimants or reachback clients. The program is also flexible in allowing local HRCC/HRD-NB officials to choose what types of EAS are needed to best meet local needs.

The most frequently cited example of the flexibility of the new programs is SLG. The fact that the program is geared towards individual clients to meet their personal needs in the area of training is the main reason why SLG is considered to be more flexible than the previous block purchases. The program is also considered to be flexible because of the autonomy given to case managers, who can negotiate directly with clients and adapt funding to individual needs. On the other hand, employers and DOL staff felt that the program should be more flexible to allow for block purchases in some circumstances (e.g., a newly established employer looking for workers). Although the degree of consistency of the application of the program from one office to another has raised some concern, the interdepartmental consultation committee appears to have been effective in establishing a number of standards to guide local case management, according to interview respondents.

There is a general agreement that programs meet the needs of those who participate in them. Clients are generally satisfied with programs and services, although many find case management to be superficial. The delay of the implementation of the SLG loan component did cause some dissatisfaction. Access to services in both official languages is not a problem, according to clients (even though some offices have admitted experiencing difficulties maintaining bilingual staff in key front-line positions). Employers also report that programs meet their needs, and adjustment services participants are generally satisfied with the assistance provided by DOL staff.

Many respondents mentioned, however, that the current monitoring mechanisms are inadequate to assess the capacity of programs to adapt to local and provincial needs. The available information mainly comes from NESS, which provides mostly short term information on client participation and impacts. There is some concern about the reach of the programs in the province overall: many programs are limited to EI and reach back clients, which represent only a portion of people looking for work in the province.

Access to PBMs by Designated/Equity Groups

Even though there are no specific LMDA programs that target these groups, local HRD-NB staff mentioned that their offices target many groups such as youth, women, Aboriginals, persons with disabilities, people who have been out of the labour force for a long period of time, and seasonal workers. The focus on each of these groups varies from one region to another. Some local staff members said that many people from these groups participate in both generic programs (i.e., intended for all clients) or specific local initiatives such as specialized EAS for equity groups. A number of officials mentioned, however, that only adequate monitoring mechanisms could really allow them to assess the degree of participation of equity group members in programs at the local level.

Many people in these groups, namely youth, older women, and persons with disabilities, have limited access to programs because many of them are not EI claimants or reachback clients, according to staff. Findings also suggest that many do not realize that they are eligible to receive services such as EAS, even though they have never worked or have been out of the labour force for a long period of time. Again, it was mentioned that these services could be better advertised to reach target groups. Other respondents suggested that there are fewer programs and projects that target equity groups as a result of the LMDA.

A number of officials mentioned that reaching the Aboriginal population is a challenge for the province, especially on the reserves which were traditionally served by HRDC officers. It appears that the division of responsibilities between both orders of government has not been clearly sorted out both in the context of the implementation of the LMDA and in general.

Labour Market Information and Labour Exchange

Labour Market Information (LMI) and Labour Exchange are key information systems for clients, case managers and local officials. According to the LMDA, the national labour market information and labour exchange systems remain a federal responsibility. It was expected, however, that both levels of government would cooperate to provide and share information from these sources.

The full implementation of accessible LMI and Labour Exchange systems had not yet been completed at the time of this report. Technical problems have prohibited the access to these systems by some of the provincial staff. Some information, for example, is only available on the Internet, which many provincial employees in local offices did not have access to from their desktop. Implementation was also affected by the working relationship between the departments, which has not been optimal on this issue because of misunderstandings about the roles and responsibilities of each level of government.

Findings also suggest that the current Labour Exchange mechanism does not meet the needs of employers since most do not use it to find workers, and that people with disabilities (e.g., the visually impaired) can have difficulties accessing labour market information because of the computerized systems used to deliver it.

5.5 Use of Services

The results of the participant survey clearly demonstrate that participants used the resources from government offices (i.e., HRCC, Canada Employment Centre) to complement their participation in employment programs. When asked if they had used services from a government office when they became unemployed, 70 per cent of respondents indicated that they had. Those respondents who had accessed services were most likely to have done so at an HRCC (70 per cent). Respondents were much less likely to have accessed these services from an employment (16 per cent of respondents unable to distinguish the specific government office), HRD-NB (13 per cent) or DOL office (one per cent) (Exhibit 5.1). When asked what kinds of services they had used on their own at a HRCC, respondents were most likely to indicate having used a Job Bank kiosk (76 per cent). Participants also accessed reports on the local job market (40 per cent), information on training or education programs (36 per cent), a computer (33 per cent), and directories of local services or programs (32 per cent) (Exhibit 5.1).

Participants were also asked if they had received services from a case manager or employment counsellor from the government prior to their first employment intervention and 28 per cent reported that they had (Exhibit 5.2). EAS and Job Action participants were more likely than Rural Experience or Partners participants to have accessed these services prior to their first employment program (51 and 39 per cent versus 25 and 17 per cent respectively). Of those who had received services, respondents were most likely to have received employment counselling (40 per cent), and referrals to another organisation (36 per cent), and job search advice on an individual basis (35 per cent).

Fully 49 per cent of respondents who had received services from a counsellor prior to their first employment program indicated that they had developed an action plan with an employment counsellor and of these respondents, 83 per cent had completed the activities in their plan (Exhibit 5.3). Among the reasons given for non-completion of the action plan, respondents were most likely to indicate that their action plan was still in progress (50 per cent), they had found a job (11 per cent), there were no spaces or funding available for programs (nine per cent) or that they had changed direction and were no longer interested in pursuing the original plan (seven per cent).

5.6 Satisfaction with Services

Client Satisfaction

Participants were asked separately to rate various characteristics of the services they had received from a government office and from a community agency (Exhibit 5.4). Respondents who had received services from a government office were most likely to indicate that they were very satisfied (responded 6 or 7 on a 7-point scale) with the level of knowledge of staff (82 per cent), followed by the speed of service (74 per cent), the information on available programs and services (64 per cent) and the quality of service (55 per cent). Ratings of satisfaction tended to be higher for services received from a community agency, with the exception of information on available programs and services. These respondents were most likely to be very satisfied with the level of knowledge of staff (85 per cent), followed by the quality of service (78 per cent), the speed of service (77 per cent), and the information on available programs and services (61 per cent).

Questions more specifically related to program interventions generally received higher satisfaction ratings. Fully 89 per cent of clients indicated that they were very satisfied with the quality of education or training they received, while 85 per cent were very satisfied with the quality of the job with their employer.

When asked how well the assistance received met their needs, respondents were most likely to indicate that it had met their needs to a small extent (45 per cent versus 31 per cent to a moderate extent and 22 per cent to a large extent). Of all the services they had received, whether from an employment program, government office or elsewhere, respondents to the participant survey were most likely to report that a loan or grant for education or training (18 per cent)¹² and employment experience (16 per cent) was most important for them in getting the help they needed (Exhibit 5.5). It is interesting to note that the largest proportion of respondents indicated that they did not know which service was most important to them (28 per cent), suggesting that services may have had only limited importance to a minority of respondents or, conversely, that they were unable to choose among the multiple "important services" they received.

The majority of respondents to the participant survey (63 per cent) indicated that they had used HRCC or Canada Employment Centre services prior to April 1, 1997 (Exhibit 5.6). Respondents were most likely to indicate that the service before and after April 1, 1997 was the same (53 per cent), although a slightly higher proportion felt it was better after April 1, 1997 (26 per cent) than before this time (18 per cent). Similar results were obtained with respect to HRD-NB and DOL services, although only 30 per cent of respondents indicated that they had used these services before April 1, 1997 (Exhibit 5.6).

Employer Satisfaction

The results of the employer survey show that, in general, employers tend to be quite satisfied with the wage subsidy programs they have used. The majority of employer respondents were very satisfied with the program overall (72 per cent) and were somewhat less likely to report that they were very satisfied with the program participants overall (63 per cent) (Exhibit 5.7a and b). With respect to various aspects of the programs themselves, employers were most likely to be very satisfied with the quality, comprehensiveness and clarity of the information about the program (91 per cent), efficiency and knowledge of the program officer (87 per cent), and support and service provided by the office (87 per cent). The lowest satisfaction ratings were observed for the marketing and outreach of the program (68 per cent reported being very satisfied) and the non-reimbursement period (63 per cent reported being very satisfied).

Fully 45 per cent of the employers who responded to the survey indicated that they had been involved in a similar wage subsidy program prior to April 1, 1997. On average, these respondents had been involved in 6.2 programs before that time. When asked to rate the work skills, attitudes to work and learning, and overall job readiness of participants in the program they had used most recently, 40 per cent of employers who responded to the survey indicated that program participants from the most recent program were better (responded with a 5, 6 or 7 on a 7-point scale) than participants from similar programs they had used in the past. Forty-six per cent of these employers indicated that the participants were the same, and only nine per cent felt that they were worse (Exhibit 5.8).

Employers were also asked if they had used any government assistance office in New Brunswick prior to April 1, 1997 and 48 per cent of respondents indicated that they had. When rating the quality of service of their most recent visit to an employment office with their experiences prior to April 1, 1997, 52 per cent of respondents felt that the service was the same, while 26 per cent felt that it had improved and only 12 per cent felt that it had become worse (Exhibit 5.8).

5.7 Bilingual Service

A key evaluation issue involved whether LMDA programs and services are being delivered in both official languages and evidence from the formative evaluation suggests that this requirement has been met very well. All LMDA program participants surveyed for the evaluation were asked if they had been able to receive services, program information or actually participate in programs in the official language of their choice and fully 97 per cent were satisfied with this aspect of the service. Only one per cent of survey respondents indicated that this had not been the case (n=24).¹³ Of these respondents, 64 per cent (n=17) were unable to identify the program or service for which they were unable to receive services in the official language of their choice. Three respondents indicated that they had been unable to receive employment counselling and two had been unable to access self-service resources in their official language. Furthermore, two Entrepreneur respondents and one SLG respondent indicated they had been unable to receive either services, program information and materials, or actually participate in the program in French. One Job Action program participant and one SLG participant had been unable to receive these services in English.¹⁴

The language needs of employers were also met very well. Of those employers who had participated in employment programs prior to April 1, 1997, none indicated that they had been unable to receive services in the official language of their choice at that time. Of all respondents to the employer survey, only three indicated that they had not been served in their official language and two indicated that they had been unable to obtain materials in their official language after April 1, 1997.

The qualitative components of the evaluation support the conclusion that the language needs of program participants and employers are well met by programs and services currently offered under the LMDA. Of all the employers and participants who were interviewed for the case studies and focus groups, only one focus group participant indicated having had difficulty accessing services in the official language of their choice. Evidence from the case studies and key informant interviews suggests that only minor, temporary difficulties have occurred with respect to providing services in either official language (i.e., loss of bilingual staff who have not yet been replaced). It is likely that these temporary circumstances account for the few employers and participants who were unable to receive services in their official language.

5.8 Use of Other Services

When asked if they had used employment-related services from an agency other than HRD-NB or a Human Resource Centre (i.e., from a third-party service deliverer), only 16 per cent (n=250) of respondents to the participant survey indicated that they had (Exhibit 5.9a). These respondents were most likely to have received job search advice on an individual basis (27 per cent), employment counselling (21 per cent), job search advice in a group (20 per cent), or referral to a job (19 per cent). Forty-three per cent of those respondents who indicated having received services from another agency indicated that they had developed an action plan with a counsellor from the agency and of these respondents, 74 per cent had completed the activities in their plan (Exhibit 5.9b). The most common reasons for non-completion of action plan activities were that the action plan was in progress (28 per cent), they found a job (25 per cent), or that they had changed direction and were no longer interested in pursuing the original plan (22 per cent).

EAS participants were also asked if they had received services to help them find a job that were not delivered or sponsored through a provincial or federal government office, and only 15 per cent of these respondents indicated that they had. These respondents were most likely to indicate receiving listings of job postings or vacancies (35 per cent), information on the local job market (30 per cent), job search advice (29 per cent) or training and education programs (21 per cent) (Exhibit 5.10).

5.9 Suggestions for Improvement

When asked for suggestions of how to improve the wage subsidy programs with which they were involved, employers were most likely to suggest a better match of participants to businesses (21 per cent), a longer period of wage subsidy (20 per cent), more timely or faster reimbursement and approval (15 per cent) and more wage subsidy (13 per cent) (Exhibit 5.11). It is interesting to note that 27 per cent of employer respondents were unable to offer a suggestion of how the programs might be improved, thus suggesting that for a large minority of respondents the current programs suit their needs fairly well.

Information from the qualitative components of the study (i.e., key informant interviews and focus groups with clients and employers), provided a number of suggestions of how the PBMs might be improved. These suggestions included broader eligibility criteria, more freedom to choose job candidates, better promotion, increased relevance to client and employer needs, adding the loan component to the SLG program, providing clearer guidelines for the administration of the SLG program, more follow-up after programs start, more courses and more timely courses for the Entrepreneur program, and faster reimbursement and approval processes for employers.

Footnotes

11	Although the CEIU did not participate in the negotiation of the Agreement, the union was involved in the LMDA implementation, which is the focus of this evaluation.
12	Eighty per cent of those who felt a loan or grant was most important were SLG participants
13	Please note that the data is weighted, thus the raw numbers do not always match the reported percentages perfectly. The raw numbers are presented in lieu of reported percentages because of the low n.
14	Multiple responses were allowed for this question, thus the total number of responses reported here sums to more than 100 per cent of those who answered the question.