Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

1.1 Overview of the Labour Market Information Strategic Initiative

The Labour Market Information (LMI) Strategic Initiative was one of a number of initiatives funded across Canada under the federal Strategic Initiatives Program as part of a process to obtain input for a review of the social security system in Canada. The Province of British Columbia, together with the Federal Government, entered into a Cooperation Agreement Concerning the Labour Market Components of the Canada/British Columbia Strategic Initiatives (Labour Market Strategic Initiatives Agreement) in August 1995 to fund initiatives designed to test approaches to social security reform in British Columbia in the areas of employment, learning and education, income security and services. The agreement provided up to $88 million (on a 50-50 cost-shared basis) to jointly design, direct, implement and evaluate six labour market initiatives in British Columbia.

The objectives of the Strategic Initiatives Program were to:

• contribute to the development of consensus on policy and program directions with the provinces and territories;
• provide the momentum for change and facilitate the transition;
• provide the capacity to develop and evaluate concrete initiatives consistent with reform options; and
• address the needs of Canadians in the areas of employment and training, learning and education, and security.

The LMI Strategic Initiative addressed the last of these objectives, that is, the needs of Canadians in the areas of employment and training, learning and education, and security. The objective of the LMI Strategic Initiative was: “to answer the need for the development, production and distribution of high-quality job and career-related labour market information and, particularly, its integration into career counselling in secondary and post-secondary educational settings, in order to support the school-to-work transition”².

According to the Labour Market Strategic Initiatives Agreement, the LMI Strategic Initiative was to address this objective by:

• studying emerging requirements for labour market information at the community level;
• identifying the specific products and services needed;
• producing and distributing these products and services as required;
• establishing standards for the development and dissemination of those products and services; and
• testing the relative effectiveness of alternative community-based delivery mechanisms.

The Joint Committee on Enhanced Labour Market Information provided overall direction for the LMI Strategic Initiative. The composition of the committee was intended to provide a broad range of input into the planning of the activities of the Initiative. The federal-provincial co-chairs of the Joint Committee made the final funding decisions. In addition to the Joint Committee, there were sub-committees that oversaw planning for each activity area. A sub-committee could consist of individuals on the joint committee as well as representatives of agencies with an interest in the particular activity area. Each sub-committee was responsible for establishing priority activities and funding implications, and generating an implementation plan for meeting the objectives of the LMI Strategic Initiatives within their area. The sub-committee chairs met to determine whether the plans of the sub-committees fit into the overall Initiative. Individual projects were managed by a working group, usually consisting of at least one person from the subcommittee as well as other people with an interest in the project.

There were five main activity areas for the LMI Strategic Initiative:

• Research and data development: to develop new sources of labour market data for use in career-related labour market information products and services;
• New and enhanced information products: to produce and distribute new career-related labour market information products, using existing or new sources of labour market information;
• Community-based information partnerships: to pilot test selected community-based projects, in collaboration with community or sectoral agencies;
• Implementation of new technologies: to test the relative merits of alternative delivery mechanisms and technologies for the dissemination of labour market information in schools, community agencies, homes and government offices; and
• Quality standards and training: research to develop standards for training of career practitioners and quality standards (currency, accuracy, etc.) for the development of labour market information, as well as pilot testing training approaches and strategies for delivering training.

In addition, the LMI Strategic Initiative linked with the Assessment, Counselling and Referral (ACR) Strategic Initiative to work together to produce products to meet the labour market information resource needs of the ACR Initiative. (The purpose of the ACR Initiative was to test and demonstrate an enhanced assessment counselling and referral system for people on income support to improve the linkages within and between employment programs, the individual on income support, and the labour market).

Through the Labour Market Strategic Initiatives Agreement, the governments of Canada and British Columbia provided contribution funding of approximately $8,210,000 for the Labour Market Information Strategic Initiative for the period from 1995/96 to 1998/99. Each funding partner also spent additional funds on administration costs, translations and other related expenses.

About 100 projects were funded by the LMI Strategic Initiative. Projects are defined as publications, community-based initiatives, websites, research documents, resource listings and guides, inventories and annotated listings of available sources of labour market information. Also included are brochures, displays, marketing, standards and training activities. These projects resulted in:

• the production and distribution of 26 LMI print resources, including updating and reprinting four of the print resources;
• the development and distribution of six facilitator’s guides and lesson plans;
• the on-line mounting of 31 resources, including three editions of Career Paths;
• the creation or support for the development of 12 websites, including BC WorkInfoNet, various local websites, Community Skills Centres websites and industry-specific and special interest websites;
• the creation of five CD ROMs for various audiences, such as Career Gateways, and
  Xplore Science Careers;
• the development and distribution of seven resource guides, annotated listings and inventories, including the brochure “What’s Key in Labour Market Information;”
• the production of 17 research papers, models and databases; and
• the delivery of 32 public forums, workshops and training sessions throughout the province, including 22 community-based LMI seminars in the spring of 1998.

1.2 Evaluation Purpose and Methodology

An evaluation framework for the Initiative was completed in January 1997. The framework report³ provided a detailed description of each of the components of the Initiative and identified the potential issues that could be addressed in the formative and summative evaluations. The formative evaluation was completed in July 1997.

The summative evaluation of the LMI Strategic Initiative reported here was conducted to assess the impacts and effects of the Initiative and to determine whether it has:

• enhanced coordination in the development, production and dissemination of labour market information;
• increased integration of labour market information into career planning and decision making;
• improved labour market information products and services to better meet the needs of a diverse range of users; and
• increased access to labour market information resources that meet the needs of a diverse range of users.

The evaluation approach entailed the following data collection components:

• Interviews and fax survey of career practitioners who provide assistance to individuals doing career planning, and analysts and program planners who are involved in the collection, dissemination and/or use of labour market information for planning of employment and education programs;
• Survey of end-users, including secondary students, income assistance and employment insurance recipients doing career planning with a federal or provincial contractor, and post-secondary students; and
• Case studies of 15 projects carried out through the LMI Strategic Initiative.

Each component is briefly described in the following sections. A complete description of each component, including sample selection and response rates, is provided in the companion report, The Summative Evaluation of the Labour Market Information Strategic Initiative: Detailed Findings.

Interviews and Fax Surveys

The primary data collection component entailed a combination of faxed surveys and telephone interviews of 150 people. Data were collected from each of the 150 respondents in two ways:

• A brief questionnaire containing the rating questions was faxed to the respondent. The respondent was asked to complete the survey, usually by circling the appropriate rating, and fax it back to the evaluator.
• Each respondent was also interviewed by telephone, after the completed survey was received. The telephone interview focused on open-ended questions, including those to followup on the reasons for respondents’ ratings.

The 150 respondents consisted of people selected as follows:

• Secondary classroom teachers providing career and personal planning instruction to grade 11 or 12 students; one teacher was selected in each of 20 schools; each school was from a different school district. The school districts were randomly selected so that four districts were selected in each of five regions of the province. The schools were randomly selected from the district.
• Post-secondary counselors who provide career counselling to post-secondary students at colleges and institutes in British Columbia. Counselors were randomly selected from throughout the province, with the restriction that four were from universities or university colleges, four were from colleges, and two were from institutes.
• Contractors providing career planning assistance to clients of either the Skills Development Division of the Ministry of Advanced Education, Training and Technology (MAETT) or the Human Resources Centres of Canada (HRCC) in British Columbia. Contractors were selected from a database containing the names of contractors providing career planning or employment assistance services to clients of either the federal or provincial government. Ten contractors were randomly selected from each of five regions in the province.
• Federal and provincial field personnel who provide, or oversee the provision of, employment planning assistance to federal or provincial clients. Fifteen federal and fifteen provincial field personnel were randomly selected to ensure equal representation from all regions of the province.
• Analysts and program planners who are involved in the collection, analysis, dissemination and/or use of labour market information for planning of employment and education programs. The 40 were selected to ensure all areas in the provincial and federal governments that pertained to planning employment or education programs were represented. Half of this group had been directly involved in the delivery of the Initiative, as members of the Joint Committee or one of the sub-committees of the Initiative, or directly involved in one or more of the projects. The other half had no direct involvement in the delivery of the Initiative.

Survey of End-Users

To provide additional information on the needs of end-users, a sample of end-users was surveyed. Data collection consisted of a mail-back survey of three groups of end-users:

• Grade 11 or 12 students enrolled in 1 of 10 career program classes: Out of the 20 secondary school teachers selected as described above, 10 (2 from each region) were asked to distribute the survey to their class.
• Employment Insurance and Income Assistance recipients receiving career planning services from a federal or provincial government: Ten of the 50 contractors (2 from each region) were each asked to distribute the survey to up to 40 of their clients.
• Post-secondary students enrolled in British Columbia: All 10 post-secondary counselors were asked about the feasibility of distributing a survey to post-secondary students. Counselors from three institutions agreed to distribute the survey.

Case Studies

Case studies were conducted on 15 projects. Each case study is presented as a profile of a project, or set of related projects that were funded by the Initiative. The profiles provide illustrations of some of the impacts of the projects, both large and small. In some cases, the impacts are local, as the project is only intended to have a local impact. In other cases, the impacts are much broader in scope. The profiles illustrate some of the challenges faced in carrying out projects of this nature, and some of the lessons learned in doing so.

Interpretation of Findings

The findings of this evaluation come from a variety of sources, with varying degrees of generalizability. To generalize findings to a population with a high degree of confidence, the findings must come from a sample that was representative of the population. This was achieved for the four sub-groups of career practitioners: secondary teachers teaching career planning, post-secondary counselors, government field personnel and contractors providing career planning services to government clients. In each case, the sample was selected using stratified random sampling techniques to ensure that the sample was representative of the population. In addition, a very high response rate was achieved, ensuring that the sample from whom data were collected was as representative as was the original sample selected. Thus, for the four groups of career practitioners it is possible to generalize the results to the entire sub-population from which they were selected with a high degree of confidence.

Confidence is lower for generalizations from the analysts and planners group, because the sample is not statistically representative of the population of analysts and planners. It was not possible to select a representative sample because it is not possible to completely define the population. Instead, an attempt was made to identify all possible components of the population and select a sample that included people from all components. In addition, this population is very heterogeneous with respect to their experience with, and needs for, labour market information. Hence, caution should be exercised in any attempt to draw conclusions about all analysts and planners based on the findings from the sample of analysts and planners.

The results for the survey of end-users can be generalized to each population of end-users with varying degrees of confidence, although with overall lower levels of confidence than for the above groups. Among end-users, confidence is highest for secondary school students because this sample has the greatest likelihood of being representative of the population from which it was selected — secondary students taking career planning instruction in BC. This is because the selection of this sample was tied in with the selection of secondary school teachers. Both were based on a stratified random sampling of school districts within BC, and of secondary schools within the selected districts. Once the school was selected, the appropriate teacher was identified and was asked to distribute the survey to an entire class of students taking career planning instruction. Hence, the sample of classes surveyed should be representative of the population of secondary students taking career planning instruction in BC.

Compared with secondary students, confidence in the generalizability of the findings is lower for post-secondary students and for clients of contractors, because we have no way of knowing how representative these samples are since their selection was less under the control of the evaluators. Although ten post-secondary institutions were randomly selected, only three could distribute a survey to their students. The students surveyed were those who happened to be seen by the counselors during the time period when the survey was being distributed, and may or may not be representative of post-secondary students doing career planning. Since most institutions are not represented in the sample, it is unlikely that the sample is representative of the entire population of post-secondary students doing career planning in BC.

Clients were surveyed by the contractors in the sample of career practitioners. Hence the contractors who surveyed their clients are representative of the population of contractors and their clients should be representative of the population of clients. The contractors were asked to distribute the survey to all clients they served during a specified time period. We do not know, though, whether the clients served during this time period were representative of all their clients, and we do not know how many clients agreed to participate (the response rate), hence we do not know how representative of the population of government clients is the resulting sample. None the less, the client sample is likely more representative of the client population than the sample of post-secondary students is of the population of post-secondary students and hence we can have higher confidence in the client data than in the post-secondary data.

For further information about sample selection and response rates, refer to the companion report, The Summative Evaluation of the Labour Market Information Strategic Initiative: Detailed Findings.

1.3 Organization of Report

This report presents a summary of the evaluation findings, organized by evaluation issue, with conclusions on each issue. The next chapter presents information about the projects that were selected for case study. Chapter 3 presents a summary of the findings on addressing the needs of users, including use of LMI, access to LMI resources and usefulness of the resources. Chapter 4 summarizes the findings as they bear on the four objectives of the Strategic Initiative. In Chapter 5, overall benefits and impacts of the Initiative are discussed, including the sustainability of its benefits. Conclusions that bear on the issues are provided throughout the report.

The complete findings are presented in detail in the companion to this report, Summative Evaluation of the Labour Market Information Strategic Initiative: Detailed Findings. Complete profiles of the 15 case studies are provided in a separate report: Summative Evaluation of the Labour Market Information Strategic Initiative: Profiles of Selected Projects.

Footnotes

2	Cooperation Agreement Concerning the Labour Market Components of the Canada/British Columbia Strategic Initiatives, Schedule C, 1995.
3	Evaluation Framework for the Labour Market Information Strategic Initiative, January 1997.