Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

This section provides a summary of the conclusions drawn and lessons learned from the formative evaluation of EBSM (the key conclusions are highlighted in italics).

The relatively new Employment Insurance regime was introduced at a time when many other changes, most notably downsizing, were affecting the department of Human Resources Development Canada. EI was also introduced together with a new philosophy which was hoped would result in a major culture change in the way the department does business. Two of the most important components of this new philosophy are the focus on results (i.e. employment and reduced dependency on EI) and the flexibility which has been given to the local level to make the majority of their decisions without having to obtain approval from the Regions.

One of the main conclusions of this formative evaluation of EBSM is that while management and staff at all levels seem to have accepted and adapted relatively well to the emphasis placed on results, the accuracy of the data currently being collected to measure results is seriously put in doubt The introduction of EBSM, combined with other change factors such as downsizing, has meant that staff at the local level have often seen their responsibilities increase without the necessary resources to deal with them. Tasks such as case management, negotiating financial contributions and developing partnerships take time, of which the remaining staff have very little. As a result, tasks such as following up on the action plans developed with clients or entering data into systems have been put on the back burner because of a lack of time or resources. The implication is that unless resources are assigned to tracking and entering data, these tasks will not get done.

Regions have tried to emphasize the need to achieve results while helping a mix of clients and soften the earlier message that focused on the importance of achieving results. However, some HRCCs are still struggling with the concept of having a balanced portfolio, that is, assisting a mix of client types and of clients who will achieve results in both the short term and medium to long term Indeed, a comparison of survey results for EBSM participants and non-participants indicate that there might be "creaming" done by the HRCCs, especially for JCP and TWS. This formative evaluation has shown that, while HRCCs have been asked to have a balanced portfolio, the emphasis on short term results has dominated the implementation of EBSM. Together with a lack of experience and good local labour market information, the emphasis on results has made it difficult for a number of HRCCs to develop strategic client targeting plans.

Negotiated Financial Agreements are another change to which many staff have had to adapt. Overall, NFA, which will be an integral part of Skills Loans and Grants, is well accepted by both staff and clients The groundwork for SL&G; has therefore been laid and the philosophy that asks clients to contribute to the cost of their training has been endorsed by both clients and staff.

Generally, local level flexibility has also been very well accepted and integrated by staff, and is viewed as a very positive aspect of EBSM. However, one of the key concerns regarding local flexibility is that it may lead to uneven service delivery (especially in large centres such as Vancouver, Toronto and Montreal), which staff found difficult to justify, even in a context where flexibility is emphasized. This would indicate that more communication work linking flexibility and results-based management is needed.

With downsizing, HRCCs have increasingly had to deal with issues related to their capacity to deliver services with fewer staff. One way that the local level has done this is to use third-party organizations to deliver services. The increased use of third-parties to deliver services has had impacts on staff since many now have to incorporate tasks, such as negotiating contracts and monitoring the work of third-parties, into their work without necessarily having been trained to accomplish these tasks. Furthermore, the way contracts are handled by HRCCs may have an impact on the efficiency with which services are delivered by third-parties and, consequently, on the use that is made of HRDC's funding. The short notice given to third-parties and the relatively short time frame for contracts were identified as major sore points that limited third-parties' ability to efficiently deliver services.

Survey respondents only indicated moderate levels of satisfaction with the employment services received or used. Two of the most frequent complaints expressed by focus group participants were the lack of awareness of the programs available to clients and the difficulties experienced in accessing staff within HRCCs. It would appear that lack of knowledge about EBSM interventions is one of the main barriers to accessing them.

HRCCs and their third-party partners appear to struggle with the concept of partnerships, and what this concept implies in terms of respective roles and responsibilities, beyond giving money for one party and receiving it for the other. While the issue of partnerships was identified as an area that needs to be developed, the caveat is that developing partnerships takes time and resources, which are increasingly limited for a number of HRCCs. Organizations which do have long-standing relationships with HRCCs or individuals within HRCCs did not feel that they had been well informed about the EBSM-related changes and, for a number of them, consulted prior to implementation. One group that reported feeling somewhat in the dark concerning available programs under EBSM was community organizations who do not have close links to local HRCCs. These organizations not only felt confused concerning the interventions available, but also about how to go about contracting with HRCCs.

The focus of the new EI has been placed on results in terms of employment and unpaid benefits to the EI account. It is still very early to make firm conclusions regarding the impact of EBSM on the creation and development of employment. More information on this issue will be obtained during the summative evaluations to be conducted in conjunction with the provinces/territories under the LMDAs.

Furthermore, EBSM are only one of the factors that can have an impact on the employability and employment of clients. External factors over which staff and clients at the local level have little control, such as the state of the local economy, also have an impact on the potential for clients in a given community to find employment (which can be mitigated by helping clients become "job ready" for available employment opportunities). Nevertheless, it can be confirmed at this point that two of the major benefits of EBSM are providing clients with job experience through interventions such as TWS and JCP, and improving their self-confidence. Self-confidence was said to play a major role in helping clients remain optimistic about finding employment, and limit the potential for them to become discouraged and give up their job search.

Official Languages

One of the sub-objectives of this formative evaluation of EBSM was to assess the extent to which the services delivered by HRCCs and third-parties are available to clients in their official language of choice, as well as the quality of the services provided in one or the other of the official language While access to information is generally not an issue of concern, access to interventions in the official language of choice of participants can be a problem in official language minority communities. One reported reason is that demand for interventions in the minority official language is relatively low.

However, demand might be low because the official language minority is not aware of the interventions available under EBSM. It was felt that third-party organizations that represent this community are in the best position to deal with minority official language issues. A number of HRCCs have already started to use these organizations to deliver services and it can be anticipated that, if this trend is maintained, access to interventions for the minority official language communities should improve. However, it must be recognized that there are factors, such as the availability of interventions like TWS and JCP in which clients can participate in their language of choice, that are labour-market related issues that cannot be controlled by HRDC. One thing over which the department does have some control is making language training available to clients. Without such training, it can be expected that employment opportunities for clients of the minority official language, and even those of the majority official language who live in bilingual communities, will be limited.