Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

Background

The Canada and Saskatchewan Labour Force Development Agreement was signed in November, 1994, as part of an effort to revitalize Canada's social security system by eliminating barriers to labour force participation. In January, 1996, a Contribution Agreement was signed outlining cooperative arrangements for the design, funding, implementation, and evaluation of three pilot initiatives within the guidelines of the Strategic Initiatives program. Labour Market Information was identified as one of the three priority areas to pilot innovative projects, along with Career Services and Work/Study initiatives.

The Labour Market Information Initiative was designed to evaluate a province-wide integrated approach for the development and pilot testing of new multi-media LMI products. Labour market information products are being developed to meet the needs of planners, job seekers, employers, career counsellors, and other users. The ultimate goal of the LMI Initiative was to develop a comprehensive automated network to gather, analyze and use labour market information to better meet the needs of Saskatchewan people. To achieve this goal, three broad categories were identified for LMI pilot projects.

1. Development of LMI Data and Network: Develop a coordinated approach to data collection, database development, and access to databases.
2. LMI Product Development: Develop and pilot test LMI products that meet user needs.
3. Dissemination of LMI Products: Develop and pilot test a variety of access methods for database sharing and dissemination of LMI products.

As of July 31, 1997, 19 projects had been approved under the Labour Market Information Initiative, with total contracted values of $1,530,500 out of a budget of $2,200,000. Appendix One provides a description of the projects, their contracted value, current status, and any timing issues.

Evaluation was identified as an important component of the Contribution Agreement because the initiatives were testing new and innovative approaches to increasing labour force participation including use of partnerships, community and industry involvement, multi-media, and other approaches to reduce barriers. In October, 1996, Calibre Consultants Inc. was contracted to conduct the evaluation under the direction of the Federal and Provincial co-chairs of the Evaluation Working Group.

The evaluation will be completed in two phases: the formative evaluation which assesses the development phase and process, and the summative evaluation which assesses the impacts of the program after product development and implementation. The formative evaluation was conducted between November, 1996 and July, 1997. Methodologies utilized included a document review; 35 in-depth interviews with key informants and stakeholders; six focus groups with job seekers, employers, career counsellors, career service representatives, planners and policy makers; and two workshops with stakeholders. The stakeholder workshops were held to review research findings and to develop common understandings regarding issues that had been identified.

The Labour Market Information formative evaluation identified the following findings.

1. Labour Market Information products are being developed to meet the needs of planners, policy makers, job seekers, employers, career counsellors, and other users. The LMI Working Group identified gaps in labour market information and proposed projects to fill those gaps. The summative evaluation phase will further assess whether the process was effective in prioritizing LMI needs.
2. The LMI projects have been designed with consideration of target group needs. Target group needs have been incorporated into the Labour Market Information initiative through involvement of representatives on the Working Groups, and through specific target group projects identified to fill LMI gaps. Comprehensive labour market information that tracks employment of target groups is expected to assist policy makers in directing funding to training programs that will best meet the needs of target groups and will assist planners and policy makers in making better decisions.
3. The LMI projects are piloting various innovations in partnerships, information collection, product development, and information distribution that are new to Saskatchewan. A number of new information technologies are being adopted including the development of new networks and multiple application databases to improve information distribution and accessibility. The Program is making up-to-date technologies available to the partners to facilitate the sharing of labour market information.
4. The LMI projects are expected to provide better service to clients through more comprehensive, up-to-date, accurate, and accessible information. As a result, planners and clients are expected to make more informed decisions.
5. This program was a new and innovative approach in coordinating and delivering labour market information and services, which not only included a large number of partners but a variety of methods in collecting and using the information. Timelines for the LMI initiative did not take into account the time required for partnership development and the time to develop appropriate policies and procedures for a complex new program.
6. In order to increase timeliness and effectiveness, it was suggested that when implementing new programs of this nature in the future, a planning or pre-operational phase to establish partnerships and for the development and implementation of new processes, policies, procedures and systems of administration should be incorporated, as well as an appropriate level of resources (human and financial) allocated to such programs.
7. The roles and responsibilities of LMI staff and other stakeholders have evolved over the course of the LMI initiative to meet identified requirements. Contracting work to individuals and companies with specialized knowledge and experience has been a cost-effective method of acquiring expertise required to complete projects.
8. Newsletters have been an effective form of communication to keep stakeholders updated on LMI project and product progress. Continued communication among the partners will contribute to further sharing of information and less duplication in collecting LMI or developing similar products.
9. Measures and procedures are in place to monitor the LMI initiative's activities, progress of the LMI projects, financial expenditures, and impacts. Delays in making the Database/Tracking System operational for developing activity reports have reduced its ability to contribute to timely decision-making.
10. The partnerships among LMI developers and users are more inclusive and collaborative than they have been in the past. Linkages among both orders of government, employers, community-based organizations, target groups, education and training institutions, and other stakeholders will provide a foundation on which future innovations and product development can take place. The partnerships have resulted in many benefits, particularly the formal and informal communication and sharing of information among partners, reduced duplication in collecting information, and leveraging of resources to develop more comprehensive labour market information.
11. Time is required to develop trusting and effective working relationships. Partnership challenges have been overcome through effective communication involving consultation, feedback of information and follow-up. The time required for partnership development must be taken into account when planning program timelines. Partnerships require time and effort to develop appropriate understandings of partner organizations' communication and working styles. Projects involving different partners also require additional time to develop consensus.
12. Federal and provincial government staff have worked together as equal partners in designing and implementing the LMI initiative. A good working relationship has been achieved through the ability of both orders of government to compromise to meet common objectives. Although the involvement of both orders of government has contributed to increased time and resource requirements for administration, the groundwork established is expected to result in increased coordination and efficiency in the development of future LMI products.
13. Several innovative LMI projects are nearing completion. Rapidly changing technology has created opportunities for LMI product development and distribution as well as challenges in ensuring compatibility for distribution and access, and in keeping products up-to-date.
14. The value received from SI expenditures for LMI projects to date has been good. The resources allocated for the LMI administration were kept at a minimum to maximize the amount of funding for the pilot projects. Most of the costs incurred by the Working Group members who contributed their time to LMI projects in addition to regular duties were absorbed by the partner organizations. The LMI projects benefited from contributions of financial and human resources from partner organizations that leveraged the funding provided by the Strategic Initiatives program.

Recommendations

The following recommendations were identified as immediate priorities for the Labour Market Information initiative.

1. Continue to monitor and link the development of LMI products, in conjunction with Career Services, through discussions with partners and end-users to ensure:
   1. priorities are identified and maintained;
   2. the information released is valid and accurate;
   3. user-friendly and integrated LMI product formats are used during the next phases of development; and
   4. identify appropriate sites, where the LMI products will be available in the future (i.e., libraries, schools, HRCC, etc.).
2. Identify best practices of the pilot projects, as soon as possible after projects have been completed. Develop a plan that outlines how the best practices (processes and products developed) can be utilized in future program development and which partners will be responsible for the long-term funding and maintenance of the projects.
3. Continue to monitor requirements for administrative resources and allocate funding as required for efficient and effective service.
4. Continue to ensure that the Strategic Initiatives Database/Tracking System is functioning properly and meeting its mandate to monitor the progress of LMI projects and that appropriate human resources are allocated to update the system.
5. Develop a communication strategy for LMI partners and the general public during the next phase to:
   1. inform potential users about the various LMI products, particularly the Partnership Website;
   2. encourage open communication among LMI partners to build on the trust and cooperative relationships that have developed;
   3. identify the most appropriate method to distribute short summaries of the project findings to partners and stakeholders which articulate LMI successes (products and processes); and
   4. develop more communication linkages with people involved in LMI in other provinces.
6. Identify the scope of translation requirements for LMI products and a cost-effective solution to adhere to federal government policies for official languages.
7. Determine policies for partner recognition on websites, particularly the Partnership Website, developed with the support of Strategic Initiatives funding.

Recommendations: Future Labour Market Information Programs

1. When planning new programs and projecting timelines, incorporate appropriate up-front time to develop partnerships and to develop more comprehensive policies, procedures, processes, and administrative systems. Feasibility studies and economic analyses must be conducted.
2. Identify opportunities, including participation in focus groups and workshops, for increased involvement of industry, community-based organizations, and other government departments/agencies in the development of future programs.
3. When implementing new programs, incorporate a pre-operational (design and development) phase with timelines and an appropriate level of resources allocated to meet client expectations for timely and quality service and for the following activities:

   1. development of partnerships;
   2. use of focus groups and/or workshops with stakeholders to provide input into program design;
   3. development of common visions, objectives, and methods of achievement;
   4. development of detailed, but flexible policies, procedures, processes, systems, roles, responsibilities, and expectations;
   5. identification and implementation of tracking systems and monitoring requirements;
   6. development of appropriate communication strategies and mechanisms; and
   7. development and implementation of an appropriate training program for staff.
4. Incorporate implications of the federal government's commitment to official languages in future partnership agreements between the federal and provincial governments.
5. Ensure significant time and resources (human and financial) are provided to establish true partnerships and to develop relationships at the community level and between federal/provincial government departments and agencies, industry and community-based organizations.
6. Involve funding and training partners in the developmental stage of individual projects to improve coordination of activities. Feasibility studies and economic analyses are important in project planning. This is very important, especially from the operational point of view, for setting projects objectives and awarding unrealistic expectations.
7. Establish a permanent LMI advisory board or council to continue and sustain the partnerships established through this initiative.