Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

• Introduction
• Program Background
• Program Description
• Evaluation Approach
• Key Findings
• Overall Conclusions

Introduction

This brief summarises the evaluation findings of the Opportunities Fund (OF) for Persons with Disabilities Program. The evaluation was funded and managed by Evaluation and Data Development at National Headquarters, Human Resources Development Canada (HRDC).

Program Background

In May 1996, the Canadian government established a Federal Task Force on Disability Issues (the Scott Task Force) to examine the role of the federal government in the area of disability. Among the issues addressed by the Task Force was the labour market integration of persons with disabilities. The Task Force recommended that the government ensure that mainstream labour market programs accommodate persons with disabilities. However, as many persons with disabilities have not had a strong labour market attachment, they are not eligible for employment programs available under traditional Employment Insurance (EI) and Human Resource Investment Fund (HRIF) programs. To address this gap, the Task Force identified a need to target labour market programming specifically at persons with disabilities. The OF for Persons with Disabilities was announced in February 1997 in response to this recommendation. The program is the responsibility of HRDC, with the guidance of a national reference group.

Program Description

The objective of the OF for Persons with Disabilities is to assist persons with disabilities prepare for, obtain and maintain employment or self-employment, resulting in increased financial independence.

Potential OF clients must meet certain criteria to qualify for assistance under the program. First, they must self-identify as disabled and wanting to work. Second, potential OF clients must be legally entitled to work in Canada. Third, they must be in need of assistance to prepare for or obtain employment or self-employment, though they do not necessarily have to be job-ready. Fourth, OF clients must be ineligible for Employment Insurance (EI) benefits. Finally, clients must commit to an action plan designed to assist them into employment or self-employment.

There are a number of guidelines that have been established for the funding of OF projects. OF activities must help individuals with disabilities move into paid employment or self-employment, provide for the sharing of practices and experiences, and augment and complement existing programming funded through other sources. OF activities should also provide for:

• the active participation of persons with disabilities and their organisations;

• lever funds from other public and private sector sources;

• involve employers, unions, and other partners to develop employment opportunities;

• provide post-placement follow-up with participants and employers; and

• fit into a balanced client strategy so that the full range of persons with disabilities is accessing the OF.

Evaluation Approach

To address the evaluation issues, five sources of information were used.

Literature and Document Review

Program documentation and literature on the labour market were examined as part of this review. As well, a summary of evaluative studies of lessons learned from prior experience of programs for persons with disabilities was reviewed.

Key Informant Interviews

In total, 22 key informant interviews were conducted to examine program rationale, implementation issues and effects and cost-effectiveness. Respondents included non-governmental organisations who are members of the national reference group; HRDC headquarters representatives; HRDC regional representatives; representatives from other similar programs aimed at persons with disabilities; non-funded applicants; and local Human Resource Centre Canada (HRCC) representatives.

Case Studies

To provide more detailed illustrations of program activities, five case studies were conducted; one of a national project, one of the Quebec umbrella organisation, and three of regional/local projects in Victoria, Red Deer and Toronto. Each case study involved a review of documentation and three to five interviews with respondents involved in the project.

Survey of Third Party Delivery Organisations

In total, 253 interviews were conducted with organisations that provided work term placements for OF clients or that co-ordinated and managed services for clients. Issues covered in the survey interviews included program activity and evaluative information such as satisfaction with the program. A "faxback" form was designed to enable respondents to provide detailed feedback on such issues as gaps in programming for persons with disabilities, best practices, and suggestions for improvements to the OF.

Discussion Groups

A total of 14 small discussion groups were conducted with participants and delivery agents/ stakeholders. Groups were held in Halifax, Rouyn/Noranda, Montreal, Ottawa, Toronto, Edmonton, Red Deer and Vancouver.

Key Findings

Program Rationale

The OF was intended to provide a response to a gap in labour market programming for individuals with disabilities that was created by reforms to EI and HRIF legislation. The Scott Task Force on Disability Issues identified this gap. According to the literature review and key informant interviews, there is a continued need for the OF program. There are both equity (citizenship) and efficiency arguments to support the integration of persons with disabilities into the labour force. The literature review also noted that as the labour market ages, the incidence of disabilities will increase, creating a greater demand for effective programs that will meet the needs of individuals with disabilities who are interested in labour market participation.

Implementation

The evaluation results suggest that the guidelines established by the OF are viewed as appropriate by those involved in the program and the implementation of OF has largely been consistent with the guidelines. The focus of OF projects examined in this evaluation is clearly on improving the employability of the participants. Most projects also were consistent in providing follow-up with clients and incorporating results targets. Sponsors noted, however, the need for patience in assessing outcomes and that depending on the nature of the disability, some clients may not be able to be fully self-sufficient. The project activity profile indicated further that there is a broad balance in the types of clients and severity of disability, though persons with severe disabilities were less likely to be present in wage subsidy programs and some key informants identified specific groups that were not yet well-served by the program.

Program Impacts and Cost Effectiveness

Leverage of existing assistance programs may have been less than anticipated initially. Among the organisations who delivered programs and services, only one-third indicated accessing other programs and services and OF was often likely delivered in combination with social assistance rather than other labour market interventions. While the limited success in leveraging may again be the result of the relative recency of the program, there may also be a need to focus attention on building capacity among organisations to approach other government and corporate/union partners and developing appropriate communications to raise the profile of the program among potential partners.

It is too early to assess the impacts of OF on the skills acquisition, labour market and quality of life outcomes of the participants at this time; the majority of the current participants have not yet completed their program. Consequently, lessons learned about the effectiveness of the different types of interventions also cannot be assessed at this stage of the evaluation. One very promising finding was that the majority of the organisations that provided a work placement were very satisfied with the OF participant and nearly half intended to hire the participant in the future.

Future Monitoring and Evaluation Issue

Unfortunately, information on what works and lessons learned may be delayed or may not be available on an ongoing basis. Although most organisations intend to monitor their clients, a minority (14%) did not develop action plans and do not intend to follow-up with their clients (11%). Only about one in five organisations are using the Contact IV software to collect client information at this time. Indeed, only half of the project sponsors who were interviewed as part of this evaluation had heard of Contact IV. There were also reports of difficulties or delays in transferring local level data to the national system. Without a formal system to comprehensively track results, information available on client outcomes will not be consistent or comprehensive, limiting the usefulness of these data to assess what works and what doesn't. It should also be noted that at present the fields that are uploaded from Contact IV to the National Employment Services System do not include participants' telephone number, which presents a barrier to future contacts.

Evaluation research may not be able to fill this void. The decentralised delivery of the program coupled with the absence of complete national level data presents challenges in terms of directly contacting OF participants to assess the impacts of the program. The extent to which OF agreements with the sponsoring organisations include a provision for the collection of client contact information to be released for research purposes is not clear. During the course of this evaluation many project sponsors and HRCCs were unwilling to release participant information citing confidentiality concerns. The lack of participant-level contact information would severely limit the ability of future evaluation studies to assess the effectiveness of the interventions and the lessons learned.

Overall Conclusions

The evaluation findings support the rationale of the program. This support of the program is based on citizenship rights for all, and labour market efficiency contention. In general, early evidence suggests implementation has been consistent with the program guidelines. The exceptions are in the areas of data collection, and a minority of 14 percent of the organisations did not monitor their clients.

In the areas of program impacts and effects, it is noted that the program so far has had insufficient leveraged funding. This could be the result of the relatively short history of the program, but it may also suggest a need to focus attention on partnership building. It is too soon to estimate the program's success in skill acquisition, but the majority of the organisations surveyed are satisfied with the individual participants.

These information and findings are all based on interviews and a survey of participating organisations. The final estimate of program results will depend on the aggregated individual experiences of program participants, for which, a direct survey is needed to verify the impacts and effects of the program.