Direct Job Creation Programs : Evaluation Lessons - December 1998

Direct Job Creation Programs : Evaluation Lessons

View Whole Report

• Purpose
• Background and Context
• Methodology and Data
• Key Findings
• Evaluation Lessons
• Experience in Other Countries
• Conclusions
• Biographical Notes

This brief presents the main findings of an evaluation study that reviews direct job creation programs introduced by the federal government. The review brings diverse evaluation findings into a common focus. It also draws evaluation lessons about what works and what does not work.

Canada has experimented with a variety of direct job creation programs over the last twenty-five years.

The main emphasis of a direct job creation program is: (a) to create jobs of a short-term nature that will reduce the number of unemployed; (b) reduce dependence on social programs; and (c) to some extent, stimulate the economy to create more jobs in the longer-term.

Although many of the direct job creation programs introduced by the federal government have been evaluated, program comparisons can be difficult to make for a number of reasons:

• The results are scattered over numerous reports.

• The evaluation studies have not followed a consistent methodology.

• Available program data do not always match. For example, in some instances, program expenditures were reported for the financial year while program participation data were reported as weeks of employment during the calendar year.

• Some programs lasted for only a short time and were discontinued before a formal evaluation was done.

Direct job creation programs have also been used in other OECD countries. This raises the question of how Canada's experience compares to other countries.

The evaluation review focused on federal direct job creation programs in the public sector. This means it excluded direct job creation programs initiated by provinces. Also excluded were indirect job creation programs, which aim at improving the employability of job-seekers and increasing long-term employment with greater emphasis on training and skill development.

The review was not intended to be exhaustive. Instead, a list of direct job creation programs was selected on the basis of two criteria: the programs were large in terms of program expenditure; and the programs had a relatively long life and therefore had been subject to a formal program evaluation.

To make program comparisons, the review developed a measure of the incremental cost of job creation for each program.

As the review pointed out, the cost per job is often measured as the ratio of program expenditure over the number of jobs created. Such an approach provides only a crude measure, however, because two additional factors need to be considered. First, job duration can vary by program, with some jobs lasting only 10 weeks while others last for almost a year. Secondly, some jobs attributed to a program might have been created even without the program intervention.

Therefore, the review incorporated available information on the incremental employment impacts of each program and the average duration of jobs to develop estimates of the incremental cost of job creation for each program.

Data on incrementality ratios were obtained from program evaluation studies. The evaluation studies were also the main source of data for program expenditures, number of jobs created, and average duration of participation in a program. Where this information was incomplete, program administrative data and departmental annual reports were used as supplementary sources.

Expenditures on job creation programs

Public expenditures on direct job creation programs are considerably lower in Canada than in most other OECD countries. For example, in 1996-97, expenditures on direct job creation programs accounted for 1.6 percent of spending on all labour market measures in Canada, compared to 2.3 percent in the US, 10.2 percent in Australia (in 1995-96), and 13.5 percent in Belgium (in 1996).

Also, the amounts spent on direct job creation in Canada have been declining in recent years, falling from 2.3 percent in 1993-94 to 1.6 percent in 1996-97. This downward trend has been accompanied by a shift towards Developmental Uses of UI.

The evolution of Canadian job creation programs

During the last twenty-five years, direct job creation activity by the federal government can be considered to have passed through three distinct phases.

Pre-CJS (Canadian Jobs Strategy) (from 1971 to mid-1980s): In the 1970s, this regime focused on new programs for students and youth of the baby boom generation. Examples of programs included Opportunities for Youth, Local Initiatives Program, Local Employment Assistance Program, Canada Works, and Young Canada Works. During the first half of the 1980s, increased emphasis was put on programs to respond to the 1981-82 recession. These programs included Canada Community Development Projects and Canada Community Service Projects under a revived Canada Works program, Local Economic Development Assistance, and Local Employment Assistance and Development.

Canadian Jobs Strategy (1985 to 1990): This regime developed a new philosophical framework that focused more on "those most in need" and "doing what works best". This regime saw a reduced emphasis on direct job creation programs and the introduction of indirect employment programs such as Job Entry, Job Development, Skill Investment, Skill Shortages, and Community Futures.

Labour Force Development Strategy/ Employment Insurance (1991 to the present): This regime phased out traditional types of direct job creation programs in favour of job creation under Developmental Uses of UI.

Incrementality and costs of job creation

Comparing the estimates of incremental cost per job for each program (in 1986 constant dollars), the review found that the cost-effectiveness of direct job creation programs improved considerably over time. For example, the incremental cost per job was $19,400 for the Local Initiatives Program in 1975 and $26,700 for the Local Employment Assistance Program in 1979. By comparison, the incremental cost per job was $9,600 for Developmental Uses of UI in 1983 and $7,630 for the On-Site program in 1993-94.

The lessons drawn from available evaluation findings highlight design issues plus the potential contributions and limitations of direct job creation programs.

Canada's direct job creation programs have generally served as useful counter-cyclical policy tools by creating temporary jobs quickly and on short notice. Most of the programs were assessed to have funded projects considered to be of value to the community.

The programs were generally found to be appropriate tools for targeting certain disadvantaged groups. However, experience with the Employment Tax Credit Program indicated that targeted job creation worked better than general tax concessions. Also, job creation incrementality was low in the private sector relative to the public and non-profit sectors.

The role of direct job creation programs in developing skills and improving the employability of program participants remains an open question.

Combining the objectives of job creation and promoting scientific/technical research in the framework of the same program did not work. Also, the use of direct job creation programs to address concerns relating to regional structural unemployment was not always successful.

Canada Employment Centres (CECs), now called Human Resources Centres of Canada (HRCCs), provided a cost-effective delivery mechanism.

New and more consistent evaluation approaches are needed to assess the net impacts of job creation programs. Also, better planning of evaluation issues, data requirements, and data collection need to occur prior to program launch.

The Canadian experience with direct job creation programs generally falls in line with most other OECD countries.

Direct job creation programs in other OECD countries were similar to those in Canada in terms of target groups, eligible sponsors, program funding, and sectors where projects occurred.

Also, the incrementality ratios of Canada's programs compared favourably to those reported in other countries (i.e. 50 to 60 percent in the UK, and 65 to 70 percent in the US, compared to 74 percent or more for many programs in Canada).

In contrast with Canada, however, the UK and the US have kept the counter-cyclical and structural elements of job creation programs more distinct from one another. In this way they avoided the tension that can occur when these elements are combined in the framework of the same job creation program.

Evaluation evidence suggests that the cost effectiveness of direct job creation programs improved considerably over time as program design improved, ineffective elements were dropped, and better elements were expanded.

The evaluation lessons indicated that direct job creation programs have:

• served as useful counter-cyclical tools;

• funded projects considered to be of value to the community; and

• successfully targeted certain disadvantaged groups.

However, the evaluation lessons also highlighted some cautions, such as:

• incrementality tended to be low in the private sector relative to the public and non-profit sectors;

• conflicting goals can create tension when included in the framework of the same job creation program;

• simply helping program clients to re-qualify for UI/EI is costly over time; and

• planning for evaluation issues, data requirements, and data collection need to occur prior to program launch.

Ging Wong is currently the Director of the Strategic Evaluation and Monitoring of the Evaluation and Data Development Branch, Human Resources Development Canada. He developed and directed a comprehensive evaluation of UI regular benefits, and has conducted large scale research on collective bargaining, industrial organization, immigration, training, employment and related issues since 1980. He is also a founder and a Director of the Canadian Employment Research Forum.

Arun Roy is currently a Senior Evaluation Officer at the Evaluation and Data Development. He has a Ph.D. in economics from University of Western Ontario. In addition to authoring many departmental monographs, his articles have been published in journals that include American Journal of Economics and Sociology, Canadian Public Policy, Canadian Journal of Program Evaluation, and International Migration Review on issues relating to UI, labour market programs and policies, and immigration.