Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

Lessons from Strategic Initiaives Formative Evaluations

The Strategic Initiatives (SI) program was set up in March 1994 to promote the development of innovative social programming to address the changing needs of Canadians. The emphasis was placed on testing new approaches to help Canadians develop skills, get work and participate more fully in society.

Programming has been targeted to the needs of specific groups of Canadians, including Canadians with disabilities, Aboriginal Canadians, lone parents, children, displaced older workers, students, youth, high school drop outs, long term social assistance recipients (SARs), people making the transition between home, school and work, and those whose Employment Insurance benefits have come to an end.

Initially the federal Government committed $800 million for a two-year program. This money was subsequently re-profiled to cover the four-year period 1995/96 to 1998/99. Subsequently the 1995 Federal Budget reduced the total program amount to $413 million. With this budget cut, the federal government committed to respect all existing Initiative announcements made to date and to preserve funding for those provinces and territories which had not yet concluded agreements. A specific amount of $26 million was set aside over the four years for Aboriginal Strategic Initiatives.

As of December 1997, twenty-four Strategic Initiatives — affecting 100,000 people including more than 5,000 young people, 16,000 single mothers and their children, and 1,000 older displaced workers — had been announced. These initiatives cover a broad range of issues: education and training, child care, income security and social services. Twenty-three Aboriginal initiatives had also been announced.

Although a range of themes has emerged from the programming to date, the primary focus of the initiatives has been on employability and training.

This document summarises the lessons from the formative evaluations of nineteen projects funded under the Strategic Initiatives program. The focus of this exercise was to identify the lessons learned in a number of broad policy areas:

• supporting poor families and children;
• supporting youth;
• addressing long term dependency among the working age population;
• assisting individuals to overcome barriers to employment;
• partnership and policy/program integration between governments;
• partnership and co-operation with the non-profit sector;
• partnership and co-operation with the private sector;
• reducing reliance on government assistance;
• job creation; and
• initiating a new program.

Not all projects addressed all policy areas. Commonly, they focussed on one or two areas. However, this is not to say that they did not also provide lessons learned in policy areas other than the ones on which they were primarily focussed.

1.1 Notes on the Methodology

The lessons which are summarised were drawn from formative evaluations for nineteen projects. (A list of the evaluations reviewed is provided in Appendix A.) The consultants reviewed these evaluation reports and initially rated the extent to which each of the ten policy areas were relevant to that particular project and, if appropriate, the strength of the evaluation findings.

They then prepared a summary sheet for each project which identified the lessons appropriate for each specific policy area. These were reviewed by the Human Resource Development Canada (HRDC) client to confirm the consultants” assessment of the lessons learned from the evaluation reports. Based on their feedback, final summary sheets were prepared and used as working documents for the preparation of this report.

The identification of lessons learned was based solely on the formative evaluation reports. These reports have some limitations as sources of information on lessons learned:

• the evaluations were conducted at different stages of implementation of the projects. The evaluations reflect the status of projects at the time of the evaluation. However, many projects have evolved considerably since that time. The lessons learned and reflected here may be somewhat dated;
• given the variety of Strategic Initiatives programming and the range of activities covered in the formative evaluations, some lessons learned have been based on the experience of very few projects;
• the evaluations were formative and, consequently, the emphasis tended to be on providing feedback on implementation to facilitate taking corrective action to maximise the achievement of project objectives. The lessons learned are, therefore, preliminary and tend to be more process rather than outcome-oriented. In general, although a particular policy area might be of relevance in a given project, the strength of the evaluation findings pertaining to that policy area was frequently much weaker than the relevance of the area;
• the fact that this was a first experience in federal/provincial evaluations had an impact on the consistency of the evaluation reports. Since programming was developed and the evaluations were conducted at a regional level, this limited the ability of the federal government to ensure consistency in the evaluation approaches;
• a wide variety of consultants conducted the evaluation studies and there is significant variation in the quality and style of the reports.

1.2 Framework for the Summary of Lessons from Strategic Initiative Formative Evaluations

Even though many lessons from Formative Evaluations are only tentative, the review of the evaluation reports produced considerable information about employment supports and related programming. The challenge was to find a way to organise these lessons which would be useful for future programming and yet address the ten policy areas of specific interest to the Department.

To do this, the lessons have been divided into four sub-groups, based on their focus. The first three sub-groups cover the lessons learned from the types of activities which projects supported (the “what”). The final group covers lessons on the process of implementing these Strategic Initiatives (the “how”).

• Firstly, there are lessons about activities which address barriers to employment and long term dependence on income support. These projects contribute directly to increasing employment opportunities, reducing reliance on government assistance and addressing barriers to long term attachment to the labour force. These typically include training activities, projects to subsidise clients gaining work experience and business development support projects.
• Secondly, there are lessons about project activities which provide improved program supports — activities which contribute indirectly to improving employment opportunities by addressing specific barriers to accessing employment-related programming and employment. This includes, for example, improving assessment and counselling, increasing access to child care and providing better labour market information. Also included here are lessons from one project not specifically related to employment but which addressed issues related to the way in which a specific client group — persons with disabilities — receives services and supports.
• Thirdly, there are lessons related to the partnerships which formed a key element of the Strategic Initiatives program. This included partnerships among public, private and non-profit sectors. Finally, there are lessons on improved program design and delivery mechanisms, including innovative approaches to programming and the lessons learned about the process of implementing new Initiatives.

The report is organised around these four types of lessons. One chapter is devoted to each sub-group of lessons.

1.3 Coverage of Policy Areas

Ten broad policy areas provided the framework for the review of the formative evaluations. All policy areas are covered in the report. Table 1.1 below illustrates the relationship between the chapters and the policy areas. However, there will be overlap and duplication in the coverage of the policy areas in order to highlight the specific aspects of the Initiatives.

Providing assistance to individuals to overcome barriers to employment was the most relevant of the objective policy areas, in that it was addressed in the largest number of Strategic Initiatives reviewed. Almost all projects covered by the evaluations, included activities which addressed barriers, either to employment or to employment-related programs. The least relevant policy area was job creation. In only four of the nineteen evaluations reviewed was this policy area identified as being of high relevance. It was of medium or low relevance in an additional four projects.

In terms of target groups, youth were of greater relevance in the projects reviewed than poor families. In over half the projects, targeting youth was rated as being of high relevance. In only five projects was support to poor families and children of high relevance.

The goal of the Strategic Initiatives was to develop new approaches to providing support services to Canadians. As a result, it is not surprising to find that different programming approaches were of relatively high relevance in most projects. Projects focussed particularly on promoting partnerships and co-operation between government and with the non-profit sector. There was noticeably less focus on promoting partnerships with the private sector. In addition, all but one project provided the opportunity to initiate a new program.

As would be expected in formative evaluations, there were strong evaluation findings with respect to the programming approaches. Findings with respect to the experience of setting up new initiatives and the partnerships between governments and, to a lesser extent, the partnerships between government and the non-profit sector were clear. The process-oriented findings were much stronger than the outcome-related findings. There were virtually no evaluation findings on program impacts, particularly with respect to specific target groups. Findings with respect to addressing the barriers to employment were the one exception to this statement.

As a result, the focus of the lessons with respect to programming objectives and target groups presented in this report are based on different programming ideas which have been tried and which are expected, but not proven, to be successful in meeting the program objectives. It is only with the summative evaluation that information will be provided on whether any or all of these lessons are confirmed.