Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

The formative evaluation of the Employment Benefits and Support Measures under the Canada/Nova Scotia Labour Market Development Agreement (LMDA) identifies major areas of positive implementation as indicated by the high degree of satisfaction stated by evaluation participants. The evaluation also recognizes improvements required in order to modify and facilitate the delivery of Employment Benefits and Support Measures (EBSM). It is important to note that the evaluation was conducted during the early stages of the Labour Market Development Agreement and that important changes have been made since the evaluation was conducted in the summer of 1998. Human Resources Development Canada (HRDC) has been making improvements to the delivery of EBSM with a continual focus on partnerships and results.

The recommendations from the evaluation report and action planned or taken to date by HRDC are noted below.

Recommendation 1: Policy

Many of those interviewed for the evaluation indicated that the flexibility of EBSM and devolution of power to local staff present challenges to new and experienced workers. HRDC should examine the degree of flexibility it has granted the delivery network in the application of EBSM. A brief policy manual laying out how and when to use each EBSM component and specifying rules that cannot be broken should be developed.

Response

A guiding principle of EBSM is to grant local offices the flexibility needed to allow them to make significant decisions about implementation at a local level to better assist their communities. Management, however, has recognized that flexibility can lead to some inconsistencies in service delivery and, as a result, a number of steps have been taken to bring coherence to program delivery within Nova Scotia.

HRDC Nova Scotia Region has developed an Accountability Policy Framework and a process to establish priorities and procedures for the effective and accountable delivery of programs and services. The approach incorporates four strategic directions: the establishment of an accountability framework; the development of tools for data capture, evaluation, monitoring and quality assurance; the provision of training and support for the development and implementation of the accountability framework; and capitalization of effective risk management practices.

Policy papers have been developed for Nova Scotia priority areas such as youth, persons with disabilities, and access to information technology. The policy papers establish parameters and provide guidelines around how HRDC and the partners can assist specific client groups. The policies are first approved by the Strategic Planning and Policy Committee and then forwarded to the Service Delivery Committee for the development of an action plan. Once an action plan is developed and approved, it is shared with all relevant service providers.

A document detailing appropriate uses of each EBSM component has been developed and is currently being validated with Regional and National staff. This tool also contains a series of Questions and Answers that will assist delivery staff in moving towards a coherent approach in the delivery of EBSMs to clients.

Recommendation 2: Administration

Central administrators and local managers should ensure that delivery staff adhere to the guidelines established for the program . To the extent such guidelines as targeting and selection, case management, and action plans are deemed important for effective program delivery, HRDC should require their use at the local level (which may entail further resources and training).

Response

The annual business plans set goals and priorities for the local area including clients targeted for services based on a community needs analysis as well as local office capacity to respond.

To foster and support program effectiveness, and to ensure that proper procedures with respect to targeting and selection, action plans and case management are followed, specific activities are set out in the Accountability Framework Implementation Plan:

• The provision of an outline of expectations related to employment programming in the annual business planning process by identifying and integrating client targeting, LMDA priorities, key performance measures, and regional priorities;
• A review of delivery methods and resource requirements for service delivery;
• Identification of gaps in tools and training that are required to improve the delivery of programs and services; and
• Evaluation of employment service delivery in the local office.

In addition, a study is currently underway in the Region to identify characteristics that put EI clients at greater risk of moving to social assistance. The results will be useful in the targeting and selection process to identify "at risk" clients.

Recommendation 3: training

Many local program staff are attempting to carry out their duties under considerable uncertainty regarding the rules around EBSM delivery. training for newly implemented programs must be more specific, especially when the program has a high degree of front-line discretion. Key policy rules that must be followed must be made clear. training on the appropriate application of Job Creation Partnerships and Labour Market Partnerships especially are needed.

Response

HRDC has been delivering training and developing tools required for an effective and accountable delivery of programs and services. The region has recently held training sessions for program officers on the delivery of employment interventions to provide them with the skills to negotiate, contract and monitor agreements within the Terms and Conditions of EBSMs.

In addition, a variety of tools and desk aids that provide staff with documentation requirements to ensure accurate and complete data capture of client case management and action plans have recently been developed to assist delivery staff. Specific tools and training include a National Employment Service System (NESS) user guide, Contact IV user guide, contracting with external service providers, and a monitoring guide for those managing external service providers. An evaluation tool has also been developed to measure employment service delivery in the local office and to identify the integration of training as well as training gaps in the workplace.

Recommendation 4: Program Monitoring

Program monitoring needs to be strengthened. Given the importance of sound results measures to policy formulation, HRDC should provide the resources needed to collect relevant data and ensure managers and staff have access to the information. HRDC needs to reiterate the importance of collecting and using results information to line managers.

Human Resources Centre of Canada (HRCCs) must also ensure that third parties take monitoring seriously and follow through to ensure the third parties are collecting required data.

In support of the monitoring function, HRDC should come to an agreement as to what types of data are required for appropriate monitoring, and design one data system which is simple to use for data input as well as for the extraction of results data.

Response

Effective monitoring and quality assurance is an area emphasized in the Accountability Framework Implementation Plan. A Regional Monitoring Policy has been developed and was recently issued to all relevant staff as part of the Accountability Implementation Plan.

As mentioned previously, training that was recently delivered to program officers had a large component devoted to project monitoring. A monitoring guide was also developed for program officers responsible for managing external service providers to ensure compliance with the terms of the agreement as well as proper client documentation.

An information package is also being developed to ensure external service providers are familiar with HRDC's accountability policies and are putting them into practice.

The HRDC management information systems are under constant review for relevancy. A recent NESS and Contact IV streamlining exercise has been completed.