Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

Background to Strategic Initiatives

In November 1994, Canada and Saskatchewan signed a Labour Force Development Agreement (LFDA) designed to provide a framework for joint planning, management, service, program delivery, and evaluation of an array of new labour force development services that the two orders of government would develop. Consultation had identified a need for:

• better co-ordination of planning and service delivery across the two orders of government;
• more community involvement in the development of programs and services;
• greater involvement of employers and various industry sectors in identifying training priorities and providing training opportunities; and
• improve access to training and jobs among people with disabilities, Aboriginal people, women, members of visible minorities, people on social assistance, and youth.

The operating environment has been dynamic throughout the term of the Strategic Initiatives Program (January, 1996 - March, 1999) to date. The following events have had a significant impact on program design and implementation:

• in January 1996, the Canada-Saskatchewan Strategic Initiatives Program Contribution Agreement was signed. It outlined co-operative arrangements for the design, funding, implementation, and evaluation of three program areas in which pilot projects would be undertaken, within the guidelines of the Strategic Initiatives (SI) Program: Work/Study, Labour Market Information, and Career Services initiatives;
• on May 30, 1996, the Province of Saskatchewan received an offer from the Government of Canada reflecting the Prime Minister's November, 1995 offer that training and related employment services could devolve to the province;
• in Saskatchewan, the LMDA negotiations started in early 1996 and culminated in the signing of the agreement in 1998, thus enabling the province to assume more responsibility in the area of training and employment initiatives for Employment Insurance (EI) clientele; under Part II of the Employment Insurance Act;
• on April 16, 1997, Saskatchewan announced the Saskatchewan Training Strategy: Bridges to Employment, a comprehensive approach to the development of Saskatchewan's labour market. The Training Strategy was in part a response to the federal offer to sign Labour Market Development Agreements (made in November of 1995). The process to develop this strategy began approximately one year prior to its announcement. Parts of the Training Strategy build upon the Strategic Initiatives pilot projects; and
• in February, 1998, the Canada Saskatchewan Labour Market Development Agreement was signed, following approximately nineteen months of negotiations with the federal government.

The SI Program is part of the federal government's social security reform which is designed to remove disincentives to employment and encourage participation in the labour force. The intention of pilots in the three initiative areas is to provide programs and services that support the transition to work and build co-operative partnerships among industry, government, education and training institutes, and communities in order to meet labour force development needs.

Program evaluation was identified as an important component of the SI Agreement. In October of 1996, Calibre Consultants Inc. was contracted to conduct the evaluation of the SI Program initiatives in Saskatchewan. This has been done under the direction of the Federal and Provincial Co-chairs of the SI Evaluation Working Group.

The program evaluation component of SI was designed to be completed in two phases: the 'formative evaluation' which assesses the development phase; and the 'summative evaluation' which assesses the impacts of program implementation. To date, the formative evaluation has been completed and the summative evaluation is in its initial stages. The formative evaluation objectives are:

• to identify the extent to which the projects are contributing to the SI principles;
• to identify the extent to which the development process for the SI pilot project is efficient and effective; and
• to identify key learnings for consideration in the next phases of the pilot projects and in future programs.

The formative evaluations of each SI Stream were conducted between November 1996 and September, 1997. An overview evaluation document bridging each of the SI Stream formative evaluation documents was developed in 1998. The following methodologies were used in each of the streams, with some degree of variation in number or extent, by stream:

• extensive document reviews of SI related documents including: agreements; guidelines; application procedures; monitoring reports; fact sheets; newsletters; project reports; work plans; project contracts; descriptions of potential LMI networks and products; needs assessment reports; evaluation framework reports; focus group results; telephone survey data summary and analyses;
• in-depth key informant interviews with SI senior management stream Co-chairs, Working Group members, Stream Project Co-ordinators/Managers and staff;
• a number of participant, sponsor, and partner interviews; and
• several focus groups and workshops with a cross-section of stakeholders such as: job seekers; career counsellors; stream Co-chairs; working group members; program management and delivery staff; community based partners; industry and target groups.

The three streams of SI share many of the same features in their design and delivery. These similar features may be examined in several areas that include, but are not limited to, the following:

• infrastructure;
• administration;
• communication;
• partnerships; and
• finance.

Features common to all three streams have been identified.

The Canada-Saskatchewan Strategic Initiatives Program is very ambitious and complex in nature. From the outset, both orders of government agreed that expenditures on direct client/program delivery would be maximized and that dollar commitments to administration and personnel would be kept to a minimum. The decision by the two orders of government to apply a joint approach to co-ordinating and integrating the delivery of a large number of skills training programs and services within the province is a large task. To apply this approach to the development and testing of pilot projects, within three separate streams, based on the creation of innovative approaches that encourage alternatives for training and access to employment opportunities and information, as the SI Program is intended to do, is an even larger and more complex task. Developing and using a true partnership approach among all stakeholders, clients, specific target groups, communities, industry, education and training institutions, and governments necessitates the creation of support mechanisms in a number of different areas. Examples of support mechanisms include: allocation of sufficient and appropriate human, financial and administrative resources and systems; communication supports; infrastructure; compatible computer systems with technical support; reasonable time-frame allocations; and an evaluation process. Resources to meet obligations and fulfil expectations are necessary from the onset and through completion of initiatives as complex as the SI Program. There were many common findings.

1. The involvement of labour market, community, and other stakeholders in program design and planning has provided significant support and commitment to the SI streams.
2. The needs of designated target group members have been incorporated in the design and implementation of the SI Streams. Several individual projects have been approved for specific target groups where group members face employment barriers. In addition, representatives of the target groups have been involved in SI working groups.
3. While resources were allocated to maximize funding for the pilot projects and minimize funding for administration, the amount of time and human resources required to develop and implement the programs was not fully appreciated in the initial stages. As a result, issues surrounding administrative resource constraints have had some negative impacts including, in some cases, timelines that are too short to permit effective implementation and assessment of projects.
4. Although the involvement of both orders of government has increased time and resource requirements for administration and delivery, the groundwork being established is expected to result in increased co-ordination and efficiency for future training and employment services, especially as the Canada Saskatchewan LMDA is implemented.
5. The development of new partnerships and the strengthening of previous partnerships have been challenging, but worthwhile. Partnerships among labour market stakeholders have been more inclusive and collaborative than in the past. The partnerships have resulted in benefits such as communication and sharing of information; helping each other adapt to change; leveraging resources for training to better meet client needs and to develop more comprehensive information; reducing duplication in collecting information; and increasing linkages between training and employment through the involvement of employers. These partnership models provide good examples of how governments can work together and with other key stakeholders to build effective, efficient post-secondary training and employment service systems.
6. Time, resources, and energy are required to establish partnerships and develop effective relationships. This must be taken into account in program budgets and timelines. Compromises are required when partners have different priorities, styles, and approaches. Effective communications with partners, including consultation, feedback, and follow-up, are critical to ensure capacity to overcome partnership challenges, understand roles and responsibilities, and adjust to differing communication and working styles.
7. External communication was effective in creating awareness and generating applications to the Work/Study and Career Services Programs. Newsletters have been an effective form of communication to keep stakeholders updated on the project and product progress for each of the three SI Streams.
8. Labour Market Information and Career Services products are closely linked for clients using the outputs of these two streams. Close communication between the two initiatives has increased co-ordination between the two areas.
9. Technology can be beneficial in creating greater access to Career Services and Labour Market Information. However, the less experienced the client group is with the use of technology, the more focussed the effort must be to ensure the technology is user friendly, accessible, well understood, and adequately supported. This increased effort should be factored into timelines for product development.
10. The SI pilot projects will provide key learnings to government regarding new models for training, delivery of career services, and labour market information.
11. A significant finding which applies across the range of the Canada-Saskatchewan SI Program is that too much was undertaken with too few resources and within too short a timeframe. The progress to date has been significant. However, progress may have been greater had more resources, more start-up time, and a stable external environment been available.
12. Policies, procedures, and processes evolved to meet the requirements of the SI streams. Initially, incomplete policies and procedures sometimes contributed to delays and reduced administrative efficiency. Strategic Initiatives' administrative delays resulted in shortened timelines for project implementation and/or product testing and assessment in some cases.
13. Delays in operationalizing the Database/Tracking System for the SI pilots have reduced its contribution to timely decision-making. The inadequacy of resources dedicated to development, implementation, on-going maintenance, as well as data entry on the tracking system has resulted in projects being tracked by SI staff through a combination of manual and electronic systems.

Recommendations Common to All Three SI Pilot Areas

Based on the findings of the formative evaluation, the following common recommendations have been identified for the three SI streams.

1. Identify best practices of the pilot projects as soon as possible and recommend a plan that outlines how the successful innovations will be integrated into current and future education and skill training programs in Saskatchewan.
2. Continue efforts to ensure that the Strategic Initiatives Database/Tracking System is functioning properly and meeting its mandate to monitor the progress of projects and/or provide activity reports for timely decision-making. In the future, appropriate human resources must be allocated to systems development; procedures must be in place to minimize any duplication in efforts; and adequate staff training must be provided to ensure the information systems' effectiveness.
3. Prior to program announcement and initiation, any future joint initiatives should incorporate a pre-operational design and development phase with timelines and resources (human and financial) for the following activities:
   • development of partnerships;
   • use of focus groups with stakeholders to provide input into program design;
   • development of common visions and objectives;
   • development of detailed but flexible policies, procedures, and processes;
   • identification, development, and implementation of tracking system and monitoring requirements;
   • development of evaluation protocols, criteria and mechanisms;
   • development of a communication strategy; and
   • development and implementation of training program(s) for staff.
4. When working with two orders of government which operate within two independent and distinct systems and infrastructures, effective co-ordination and collaboration can only be achieved by identifying ways to integrate the dual systems without compromising their individual system integrity.
5. Identify opportunities, including participation in focus groups or workshops, for increased involvement of industry, community-based organizations, and other government departments in the development of future training and employment programs. Ensure non-government partners are able to participate in workshops and other activities and, when necessary, subsidize expenses.
6. Encourage the involvement of funding and training partners in the developmental stage of individual projects to ensure co-ordination of activities. Continue to seek opportunities to involve representatives from each of the target groups identified in the SI principles.
7. Ensure significant time and resources (human and financial) are provided to establish true partnerships and to develop relationships at the community level and between federal/provincial government departments and agencies, industry, and community-based organizations. Continue to strengthen these partnerships.
8. Develop a communication strategy for each of the three streams that takes into account each one's specific communication requirements. Generally, this involves communication that maximizes awareness of the SI stream and the progress of the projects within it for users and potential users; transmits clear and consistent messages and guidelines; encourages co-operative working relationships among project partners.
9. Ensure that a program evaluation component is included at the onset of newly designed programs. Prior to program initiation, it would be helpful to develop evaluation criteria, database/tracking system requirements, expectations, performance measurements, and benchmarks. A preliminary framework document was developed, however, as a result of urgency for implementation, inadequate and incompatible information and benchmarks resulted in an indirect correlation and translation of the framework document into action.
10. Continue efforts to refine reporting and monitoring forms to streamline the process and increase consistency.
11. Continue to monitor requirements for administrative resources and allocate funding as required for efficient and effective service.
12. Continue to monitor and link the development of LMI products, in conjunction with Career Services, and strengthen the linkage between the two streams at all levels.
13. Continue efforts to identify the scope of translation requirements for LMI products and a cost-effective solution to adhere to federal government policies for Official Languages. Incorporate implications of the federal government's commitment to official languages in future partnership agreements between the federal and provincial governments.
14. Ensure the results of the Strategic Initiatives pilot projects are included in the development work with respect to future labour market programs and services developed under the Canada Saskatchewan Labor Market Development Agreement.

---

Last Modified: 1999-06-16	Important Notices