HRDC - Summative Evaluation of Ontario jobLink Resource Centres

Directorates

· Director General's Office

· Data Development

· Strategic Evaluations

· Partnership and Continuous Evaluations

Publications

· Evaluation Reports

· Evaluation Briefs

· Labour Market Development Agreement Series

· Lessons Learned Series

· Evaluation Tool Kit Series

· Evaluation Forum Newsletter

· From Theory To Practice

1. Introduction

1.1 Background

The jobLink initiative was established in Ontario in 1994. The initiative provided funding to resource centres (RCs) in local communities in Ontario to improve the access to employment services and support for social assistance recipients (SARs) which would ultimately lead to self-sufficiency. Funding in 1994-95 for the initial sites was jointly shared by the Ontario Ministry of Community and Social Services (MCSS) and the federal government under a Co-operation Agreement signed between the two parties in early 1995.² In-kind contributions were also provided by local municipalities where the resource centres were located.

Federal funding was provided under the Strategic Initiatives program of Human Resources Development Canada (HRDC). This program, which provides funding on a cost-shared basis with provinces and territories, was designed to test innovative approaches to addressing high priority areas of employment, learning education and income security. In total, $800 million has been allocated over fiscal years 1995/96 and 1999/2000 to:

test innovative and cost-effective ways of reforming our social security programs;
experiment with imaginative ways of addressing areas such as employment, training, income support and services;
help people develop the skills they need to find, keep and create jobs;
better serve those Canadians unable to support themselves through employment; and
provide opportunities for program coherence and integration to reduce the jurisdictional and structural barriers between labour market/training/education and social services.

While the jobLink initiative established guidelines for planning and implementation of the RCs, the termination of the program in 1995 removed any centralized co-ordination in the development of the RCs. Operations and programming, therefore, were developed to reflect local priorities.

The purpose of this study was to conduct a process and summative evaluation of jobLink resource centres at selected communities in Ontario. These communities were Toronto, Sudbury, Windsor and Ottawa. The evaluation was built upon a process evaluation undertaken of Windsor and Sudbury in 1996. The objectives of the evaluation were to:

assess the impact of the program on worker client's employment, employability and access;
assess jobLink RC's role in building community consensus and capacity;
identify jobLink RC's role in improving service co-ordination, filling gaps and reducing overlap in delivery of government services;
assess cost-effectiveness/efficiency of jobLink RC's services; and
identify issues critical for future discussions of collocation and co-ordination of federal and provincial service delivery.

The purpose of this Final Report is to present the results of the evaluation. Each of the evaluation issues above are addressed in the following chapters based on the evidence available.

1.2 jobLink Ontario³

The goal of jobLink Ontario, as indicated in Local Implementation Planning, was to involve various levels of government and non-governmental organizations (NGOs) working together to provide social assistance recipients with a single integrated system of services that would assist them in entering the labour market. Such services included, for example, education and training opportunities, information on the local labour market and job opportunities and support for self-employment. jobLink was established to provide to social assistance recipients greater access to labour market services at the local level.

jobLink resource centres were one vehicle for achieving the objectives of the jobLink Ontario program.⁴ Resource centres were set up in local communities to provide social assistance recipients (SARs) with information regarding employment, training and local supports available to them.⁵ In co-ordinating the programs and services delivered by the federal, provincial and municipal governments as well as community organizations, the resource centres were a single access point for SARs.

Services could be accessed directly by the individual or with assistance from staff. The services provided to participants included individual employability assessments and employment planning, education and training programs, access to computers and computer-based information, supports (e.g., child care, counselling), resource inventories and self-help tools. In providing these services, resource centres were expected to build on and co-ordinate with existing programs and services, but not to duplicate them.

To encourage community collaboration in the development of innovative approaches to service delivery and to ensure consumers were more involved in the planning, implementing and monitoring of the jobLink service system, RCs were encouraged to establish Community Advisory Committees. Comprised of service providers, consumers and other key stakeholders or community partners, the key areas of responsibility of the committees would be community information/ consultation/linkages; service planning, monitoring and evaluation; and advice/ recommendations on program design and policy development.

Initial sites for the resource centres were identified through a local jobLink Ontario planning process. Through this community process, a plan for implementing a local jobLink system was to be developed and then submitted to a central approval process. Of importance to this community planning process was the involvement of consumers, program/service providers, planning groups, educators/trainers, private sector employers and organized labour.

In the proposals developed by the communities, a number of key areas had to be addressed: the planning process; existing conditions, including a profile of the community, labour market and the consumer, and the identification of existing services and gaps; suggested improvements to existing services; and an implementation and monitoring plan for providing services, programs and supports that would meet the objectives of jobLink Ontario.⁶ The initial sites for the resource centres included: Metropolitan Toronto; City of Windsor; City of Cornwall; City of Kingston and Kingston Township; Regional Municipality of Ottawa-Carleton; Regional Municipality of Waterloo; Regional Municipality of Hamilton-Wentworth; Bruce County; Districts of Sudbury and Manitoulin; City of Sault Ste. Marie and District of Algoma. Start-up of the resource centres in these sites was scheduled for January 1995.⁷

With the change in government in June 1995, jobLink Ontario was terminated. While this caused some initial uncertainty regarding the future of the resource centres, the RCs continued to function. MCSS has continued to provide direct funding to RCs, with community-level support also provided by Human Resource Centres of Canada (HRCCs). Resource centres have also faced a transition period with the introduction of Ontario Works in 1996 — legislation to replace General Welfare Assistance and the Family Benefits Act.

1.3 Study Issues

The primary purpose of this evaluation study was to measure the impacts and effects of the integrated resource centre delivery sites, which were initially established through jobLink. The evaluation also incorporates process elements, however, to examine the operation of these RCs. The issues, which form the basis of this evaluation, are: client impact; cost effectiveness/efficiency; building partnerships; community impact; and case management. Each of these is discussed, in turn, below.

Under the issue of client impact, questions concerning employment and employability of worker clients (e.g., how long did it take clients to find employment, what were the characteristics of employment, what are the results for staff versus self-assisted services) and the incremental impact/value added (e.g., were there incremental improvements for RC clients compared with non-clients, did RC counselling have a demonstrated impact on clients) are addressed. In addition to employability or labour market outcomes, a series of non-labour market outcomes are measured. Client/community/agency satisfaction (e.g., what was the worker client's overall opinion of RC service, did the RC meet community expectations, how did agencies rank their working relationships with the RC) with the services provided by the RC are also addressed under the issue of client impact.

The second issue, cost-effectiveness/efficiency, refers to essentially operational issues. The variations in delivery provide an excellent opportunity to determine the relative efficiency of different types of services and methods of delivery, though this is dependent on the type of tracking that is available. Client services and client service ratios (e.g., what is the average cost per participant) are addressed to the extent possible.

In terms of the community partnership model, the evaluation of RCs provides information on lessons learned on building partnerships with community agencies. Questions addressed under the third issue, building partnerships, concern: facilitating partnerships, what worked (e.g., did advisory committees aid the partnership process); inter-agency relationships (e.g., were partners and RC on-site staff able to support and work effectively); and intra-RC relations, member agencies, staff (e.g., what mechanisms contribute to building a team approach within the RC).

The fourth issue that is addressed is community impact. The original objectives of the jobLink initiative focused on, among other things, improvement in access to employment services through a "one-stop shopping" approach. In addressing the issue of community impact, questions concerning the identification of gaps (e.g., to what degree has the RC succeeded in integrating, enhancing and closing gaps in existing services), building community capacity (e.g., to what degree has the RC added to the community's service capacity), reduction in duplication (e.g., to what degree has the RC reduced duplication in services offered), referrals (e.g., what are the links to other community agencies) and development of technology infrastructure (e.g., how has the RC contributed to the technology infrastructure for service delivery in the community) are examined.

The final issue to be addressed under Phase 2 is case management. The questions examined under this issue concern: information capturing systems/ procedures (e.g., how effective are the RC client tracking and information collection systems in terms of tracking agency referrals); mix: self versus staff service (e.g., what is the monitoring accorded self-serve clients); client profile and client selection (e.g., what is the RC's client profile of case managed clients, does the RC apply client selection criteria); and referral agency links.

1.4 Lines of Evidence

There are five major components in the methodological approach for this evaluation:⁸

Document/literature review. The purpose of the document/literature review was to place the development of the RCs within the policy context of federal and provincial partners, including the potential impact of new policies on RCs (e.g., Ontario Works). Prior evaluation work and local RC documentation were also reviewed. The document review contributed to addressing some of the evaluation issues, including understanding RC administrative and partnership arrangements.
Surveys of RC clients. In total, 493 telephone interviews were completed with RC clients distributed roughly evenly across the four sites — Toronto, Sudbury, Windsor and Ottawa. The objectives of this survey address both process and summative evaluation issues. For example, the survey provides an indication of clients' utilization of services, levels of satisfaction with services received, as well as their views of any alternative services that were available to them. Summative issues related to clients' labour market and non-labour market outcomes are also examined. The client survey data have been weighted by age, sex and year started social assistance to reflect the client population (as represented by the sampling frames that were received from the resource centres).
Survey of non-RC clients. In total, 400 telephone interviews were completed with non-RC clients. This comparison group was drawn from the General Welfare Assistance caseload, selected to be comparable with RC participants in terms of sex, age and year started receiving social assistance. The comparison group was also screened to ensure that they were receiving social assistance during a period comparable with RC clients. The comparison group data were weighted to be more comparable with the participants in terms of age, number of dependants, education and pre-program employment status.
On-site visit. An on-site visit was conducted with the Ottawa site to provide a more detailed examination of the operation and community environment at this location. Key informant interviews and a focus group with clients constituted the principal methods used to gather the on-site data. As well, the site visit included direct observation of the operation of the RC and the collection and review of RC and local regional economic documentation. A limited number of key informant interviews were also conducted with representatives of the Windsor and Sudbury RCs to update information collected during an earlier formative evaluation.
Administrative data analysis. Two sources of administrative data were examined for this evaluation: basic program/RC administrative data and General Welfare Assistance data. Linkages to HRDC data files, particularly the Status Vector files, were also established.

View Exhibit 1.1

1.5 Caveats

Four important caveats should be noted in the interpretation of study findings. First, this evaluation includes only four of at least 10 initial jobLink resource centres and cannot, therefore, be considered to be representative of the jobLink program overall, but rather reflects the situation in the sites selected. Second, because of confidentiality concerns at some sites, RCs were not able to release their program files, which limited the data quality analysis and subsequent examinations of program activities and impacts. Third, due to delays in processing the comparison group file, the survey of participants and the survey of the comparison group were conducted several months apart. Though the comparison group responded to the survey items in the same timeframe as participants, there was a greater potential for recall bias. The similar timeframe for the studies of the participant and comparison groups, however, does remove any possible effects of changes in the economic cycle. Finally, despite our attempts to construct a comparable group of non-participants through sample selection screening and then weighting, there were some remaining differences between the two groups. These differences were controlled in the data analysis through the use of multivariate analysis techniques. Finally, due to rounding, figures presented in tables and charts may not add to 100%.

1.6 Document Organization

This report contains five additional chapters. Chapter 2. of this report provides a brief operational description of the four jobLink sites covered in the evaluation. Chapter 3. presents a profile of jobLink participants. Chapter 4. describes the types of adjustment and employment services used by jobLink participants and their rated satisfaction with the resource centres. Chapter 5. analyses the labour market and non-labour market outcomes of participants compared with non-participants. Summary and conclusions are presented in Chapter 6.

Footnotes

²	*Canada/Ontario Co-operation Agreement Concerning jobLink Ontario.*
³	See Appendix A for a more complete description of jobLink Ontario.
⁴	Six other components include: Making the Transition from the Existing System; Ontario Training and Adjustment Board programs and services; jobLink Ontario Innovations Fund; employer-based training programs; jobLink Ontario for First Nations and Aboriginal peoples; and HRDC programs and services.
⁵	Ontario Community and Social Services and Human Resources Development Canada, jobLink Ontario: *Local Implementation Planning*, 1994.
⁶	Ontario Community and Social Services and Human Resources Development Canada, *jobLink Ontario: Local Implementation Planning*, 1994.
⁷	Ontario Community and Social Services and Human Resources Development Canada, *jobLink Ontario: Local Implementation Planning*, 1994.
⁸	For a more detailed discussion of the methodology for this evaluation, please see *Phase 2 Evaluation of jobLink Resource Centres Field Report*, submitted to Human Resources Development Canada, December 1997.

Last Modified: 2000-08-16	Important Notices