Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

This chapter presents the findings on the few remaining issues that pertain to the overall impacts of the Initiative and its sustainability, as well as findings on the cost effectiveness of the delivery of the Initiative. Also included are highlights of the lessons that were learned from the activities of the Initiative and a summary of the key findings and conclusions. The chapter ends with overall conclusions about the effectiveness of the LMI Strategic Initiative.

5.1 Overall Impacts, Sustainability and Cost Effectiveness

Overall Benefits and Disadvantages

People who had been involved in the Initiative were asked what they felt had been its most important benefits. The most obvious benefit of the LMI Strategic Initiative was the range of useful resources it created. This was seen as the most important benefit by about two-thirds of the respondents. However, about one-half identified partnerships as either the most important benefit, or just as important as the resources created. Because of the way the Initiative carried out its activities, people involved learned the value of working together collaboratively. This included federal and provincial personnel, as well as representatives from different career practitioner organizations, and career practitioners working in the field. This collaboration brought a broad perspective to the Initiative, increasing knowledge about the needs of various groups. It also resulted in end products that were aimed at more than one audience or one need. And it left a legacy of people who not only know how to work collaboratively, but they also know the benefits of doing so. The people involved in the Initiative were also asked whether there were any significant disadvantages of the Initiative. Only one theme emerged from the responses. About two-thirds were concerned that the sustainability of the activities of the Initiative had not been provided for at the beginning. They were concerned that the Initiative had raised expectations without the ability to continue to meet them.

Sustainability of Benefits

People who had been involved in the Initiative were also asked to consider what benefits, if any, would continue after the Initiative ended. Two-thirds felt that benefits would continue. Working in partnership was the benefit that about half expected would continue. Other benefits that were expected to continue were the awareness of the values of LMI, and the continued demand for more LMI resources to meet the needs that were addressed in the Initiative.

The remaining one-third of respondents were not sure whether benefits would continue, although they did identify partnerships as a benefit with the potential to continue. They were uncertain because they were aware of barriers to the continuation of partnerships, as were some of the other respondents who had identified partnerships as a benefit that would continue. The barriers that might inhibit the continuation of partnerships that respondents identified are:

• Lack of funding. The LMI Strategic Initiative provided funding, which made it possible for the various partners to work together. Partnerships need a purpose. Without funding, there will be no purpose in working together because there will be no activities to work on. Some partners were able to bring funding to the partnership, but many, particularly the not-for-profit associations of career practitioners, do not have a source of funds to draw upon. Without funding, the career practitioners’ organizations cannot participate in activities in the way they did during the Initiative.
• Uncertainty over roles relating to LMI. The LMI Strategic Initiative defined the roles of the federal and provincial governments and provided a forum for working together. With the end of the Initiative, uncertainty exists regarding the roles of the federal and provincial governments with respect to the collection and dissemination of labour market information under the Labour Market Development Agreement. This uncertainty also affects funding of new work, including updating existing resources, and the development of new resources and hence, has delayed activity.
• Limited time of partners. For most people, their role in the Initiative was carried out “on the side of their desk,” as it was not part of their job function, or was only indirectly so. This is not a sustainable way to operate. For partnerships to continue, labour market information will have to move closer to the centre of people’s desks. That is, involvement in the collection, production or dissemination of labour market information will have to become part of the job function. However, many people already have more responsibilities than they can handle in the time available. Effective partnerships cannot happen if responsibility for LMI is just added to an already full work load.

Cost Effectiveness

There are two issues pertaining to cost effectiveness that were addressed in the evaluation:

• Was the Initiative a cost-effective method of achieving its objectives?
• Has the Initiative identified potentially significant economies or efficiencies in the development, production and dissemination of LMI?

To address the first issue, respondents who had been involved in the Initiative were asked about the cost effectiveness of the Initiative. Many found this a difficult issue to address. One of the purposes of all the Strategic Initiative programs was to be innovative and to experiment with imaginative approaches. Hence, many of the activities of the Initiative were innovative or experimental in nature. Being innovative means there is nothing available for comparison. It also means the people are often doing things they have not done before. Some trial and error may be necessary in working out the best approach to a project. This will take additional time, and often money. It can, therefore, be difficult to be both innovative and cost effective.

Many of the people involved in carrying out the individual projects had never worked on a project of this nature before. As well as lacking experience in designing and managing projects like this, they did not know what costs to expect and could not budget accordingly. Others had experience but were doing something innovative, so did not have the knowledge that was needed for accurate costing. A number of the projects profiled in Chapter 2 illustrate these issues.

Some respondents were able to identify ways in which the Initiative was cost effective. Working in partnerships was often cost effective because the partners could share costs. In some cases, partners were able to supply funding to supplement the funds from the Initiative. Partners also donated in-kind support to projects, sometimes in the form of the use of technology, such as providing free use of a server for a website, or by donating their time to the project. Many of the community partnerships demonstrate both in-kind and financial contributions to projects. Partnerships can be cost effective also by avoiding duplication and by developing one resource that can meet the needs of more than one audience. Work Futures and BC WorkInfoNet are two examples. In addition, partnerships made it possible for people to share their knowledge and expertise, some of it gained through work on previous projects.

The experience gained through working on the Initiative means that future work of this type can be done in a more cost-effective way. Initial costs, such as the design of Work Futures and the editing tool created for it, have been absorbed by the Initiative. Future updates of the resource will not be as costly. Even new products could potentially benefit from this experience because people will have acquired knowledge that will help them budget effectively in areas such as size of print runs that will be required, costs of printing, and costs of various alternatives such as producing CD ROMs.

The partnership model can yield significant economies of scale or efficiencies in the development, production and dissemination of LMI. A number of instances of cost-effective delivery are illustrated in the projects profiled for the case studies. However, given the available information and the innovative and experimental nature of much of the Initiative’s activities, no overall conclusion on whether the Initiative was a cost-effective method of achieving its objectives is possible.

5.2 Lessons Learned

As a Strategic Initiatives Program, the LMI Strategic Initiative was to test and evaluate innovative and cost-effective ways of addressing its objectives to learn what works and what doesn’t. The Initiative has indeed been innovative. And lessons have been learned in the process. Through this evaluation some of these lessons have been captured. Chapter 2 provides a synthesis of the lessons learned from individual projects that were profiled for the case studies. The individual lessons will not be repeated here.

One of the key lessons of the Initiative as a whole was the value of working in partnership. The value of partnerships was frequently mentioned by respondents when citing benefits of the Initiative or lessons learned from carrying out individual projects. A number of benefits of working in partnership were identified by respondents, including:

• improved understanding of the needs of users;
• more cost-effective approaches to addressing needs;
• reduced overlap or duplication with other projects;
• sharing of resources, either in-kind or financial;
• sharing of knowledge about collecting, producing and disseminating labour market information; and
• creation of more user-friendly LMI resources.

Another benefit is that working in partnership is one way that lessons learned from other projects can be shared with other collectors, producers or disseminators of labour market information. Another lesson about partnerships is the benefit that can be achieved with a relatively small amount of funding through the cooperation and dedication of those involved. None the less, some funding is needed for partnerships to happen. Partnerships only exist to achieve some end and money is needed to do this. With the completion of the LMI Strategic Initiative, money for carrying out LMI projects has been greatly reduced, and with the reduction in funding has come a concomitant reduction in the amount of partnering occurring in this area.

Other key lessons learned through the activities of the LMI Strategic Initiative are summarized below.

• There is a receptive and even enthusiastic audience for appropriate LMI resources that address their specific needs for labour market information.
• Free and widespread distribution of a resource improves the chance that it will be used by the intended audience. Free distribution raises awareness of the resource and ensures that intended users have the resource, thus avoiding problems due to inadequate budgets for LMI resources, or lack of knowledge or time to find the resource.
• The internet is a highly effective method of disseminating labour market information, LMI resources, and information about what LMI resources are available and where to get them. A large, and growing, segment of the audience for LMI has enthusiastically embraced the internet as a source for LMI.
• Appropriate distribution of a resource may not be enough to ensure that a product gets used. Promotion is needed to ensure that potential users know about the product, and about how it can be used. Lack of time can hamper people’s ability to use the resource because they don’t have time to learn how to use it. Providing in-service or training on specific resources has been one method used in some of the projects of the Initiative to ensure that potential users take the time to learn how to use them.
• Distribution and promotion of each resource needs to be considered at the beginning of a project. In some cases, there was confusion over responsibilities for distribution and promotion between the funders and the recipients of funding. People involved in carrying out projects did not always have the necessary knowledge to carry out effective distribution and promotion and project budgets did not always allow for distribution or promotion. As a consequence, some potentially useful products have not received adequate attention because of lack of promotion and/or inadequate distribution.
• Projects with a technical component, such as creating a CD ROM or a website, or creating an interactive, on-line application, can be implemented more efficiently when the technical people are brought in at the beginning of the design phase. This can ensure that the design being considered is technically feasible, and can be done within the budget available. Those designing the project can learn about the trade-offs that will need to be made, and can be informed of the consequences of various approaches. In addition, the technical people can gain a better understanding of the concept that the designers are trying to achieve.
• The potential benefits of a successful project are not fully realized when the project cannot be sustained. Since projects funded through the Initiative were conceptualized as pilots, funding of projects was not contingent on the ability of projects to be maintained after funding ended. However, the success of the Initiative means that there are a number of projects, some at the local level and some at the provincial level, that were successful. Successful projects set up expectations among users and those who carried out the project. In the case of websites, users expect the site to remain functional. Many users have come to rely on resources like Work Futures, and expect that they will be kept up-to-date so that they will continue to meet their needs. People who carried out the project often put a lot of time into it, and are frustrated when they can’t maintain a website, or market a resource.

Every project funded by the Initiative was in some sense a pilot project. Some have gone on to become ongoing projects. Some were successful, but have ended. Others were failures, in that the goals of the project were not achieved. Even failures can provide valuable lessons about what doesn’t work and hence, make a contribution to the objectives of the Initiative. In fact, it is likely that more lessons were learned from the challenges faced in carrying-out projects, whether or not the challenges were overcome. The most successful projects were not necessarily without challenges, but they were able to effectively overcome the challenges that arose.

Through the case studies, one can see that a variety of lessons have been learned about collecting, producing and disseminating labour market information. Unfortunately, there is no forum for collecting and disseminating these lessons, except through this evaluation. The resources for this evaluation did not allow for profiles to be created for all funded projects due to the considerable amount of research that was needed to collect the information for each profile. As a consequence, any lessons learned from the projects not profiled are known only among those who were involved and learned the lesson first hand. These lessons are not available to benefits others. This points to a limitation of this evaluation, as well as to a limitation in the reporting requirements for funded projects, in that there was no requirement for final reports for each project that captured the necessary information about what worked and what didn’t, as well as reasons why projects were not able to achieve their goals. Only through continued partnering is there a mechanism to ensure that these lessons are shared in a meaningful way with others who can benefit from them.

5.3 Summary of Key Findings and Conclusions

The LMI Strategic Initiative resulted in the creation and dissemination of a wealth of LMI resources, in both print and electronic form. To encourage the effective use of these resources, it developed facilitator’s guides and provided workshops and training on the use of a number of LMI resources. To improve the capacity for the collection of local or sector LMI, and the development and dissemination of LMI resources, the Initiative funded a number of pilot projects in communities throughout BC, that resulted in other resources.

The Initiative made LMI resources available to their intended audiences by widespread distribution of free print copies, and by creating internet or CD ROM versions of a number of resources. In addition, the Initiative established the BC WorkInfoNet website, which provides links to a wide range of LMI resources, including ones created by the Initiative as well as many others.

Among all the activities of the Initiative, and all the products and resources created, four stand out as key resources because they address the needs of a broad range of users: Work Futures, the BC WorkInfoNet website, Making Career Sense of Labour Market Information and A Guide to the BC Economy and Labour Market. It is noteworthy that all four were produced early in the course of the Initiative. Resources produced near the end of the Initiative may, or may not, turn out to be just as useful, but at the time of this evaluation they were not in widespread use. It may be that time was the issue. Before people can make effective use of a new resource, they need to spend some time learning how to use it. But lack of time was an issue for many respondents, who felt they did not have time to learn how to use the resources.

The objective of the LMI Strategic Initiative was: “To answer the need for the development, production and distribution of high-quality job and career-related labour market information and, particularly, its integration into career counselling in secondary and post-secondary educational settings, in order to support the school-to-work transition”.⁵ The LMI Strategic Initiative addressed this objective by:

• improving labour market information products and services to better meet the needs of a diverse range of users;
• increasing access to labour market information resources that meet the needs of a diverse range of users;
• enhancing coordination in the development, production and dissemination of labour market information; and
• increasing integration of labour market information into career planning and decision making.

Although the objective specifically identifies the secondary and post-secondary school audiences, the focus of the LMI Strategic Initiative was broader than this to encompass the entire career-planning audience, including government clients on employment insurance or income assistance and the career practitioners who provide support to them as they strive to integrate, or reintegrate, into the labour market. The Initiative had an impact on all these audiences, but its greatest impact was on the career practitioners who provide support to government clients. The Initiative was less effective in addressing the needs of the secondary school audience. Teachers, who were less likely than other career practitioners to use LMI, were more likely to have difficulty accessing LMI and less likely to have access to the internet. They were also less likely to feel that resources met their needs. Contractors and counselors, on the other hand, who were more likely to use LMI, were less likely to have difficulty in accessing LMI and were more likely to have access to the internet. Compared with teachers, contractors were much more likely to feel that LMI resources met their needs, and counselors were somewhat more likely to feel this way.

5.4 Overall Conclusions

During the course of the LMI Strategic Initiative, a paradigm shift occurred regarding the use of labour market information. More than ever before, career practitioners and program planners were using labour market information. At the end of the Initiative, there were more and better quality LMI resources than there had been when the Initiative began. Many of these resources were the work of the Initiative, in whole or in part.

Because of the increasing concern with labour market issues, some of the activity of the Initiative, and the resulting LMI resources, may have occurred even if the Initiative had not happened. While there is no way to know how much would have occurred, the Initiative did, however, provide a forum for this activity, and for partners to work together to address the need for better LMI for career planning purposes. The Initiative also provided funding that made it possible to significantly increase the level of activity in this area compared with before the Initiative.

By partnering with, and involving, a wide range of career practitioners and program planners in its activities, the Initiative was responsive to the needs resulting from the paradigm shift. By increasing the number of relevant LMI resources for career planning, and increasing awareness and understanding of the value of using labour market information, the Initiative contributed to increasing the integration of labour market information into career planning and decision making.

Although there is a greater recognition of the value of LMI now, the paradigm shift is not complete. More work is needed toward raising awareness and teaching the value of using labour market information in career planning and program planning, and toward informing intended users about what LMI resources are available. Even once the shift is complete, and the need for labour market information in career planning and program planning is accepted by all, work will need to continue in order to continue to meet this need.

Lack of time is a factor that limits how quickly this shift can occur. Those who are not experienced with labour market information need to take time to learn about it and its uses. They need to know what resources are available to address their needs, and how to make effective use of these resources. This takes time, which is limited for many. Many intended users find it difficult to find the time to learn something new and hence, some are behind others in their facility with using labour market information.

Although the LMI Strategic Initiative has been effective in improving the quality of labour market information, there is a continued need for work in this area. Labour market information is timely information and becomes outdated relatively quickly. There is the danger that if resources used for career planning are not kept up-to-date, people will end up making ill-informed decisions. The potential damage will depend on how much the labour market has changed since the resources were developed. For specific sectors that have experienced a lot of change, either growth or decline, the result of using out-dated resources could be a reduction in effective functioning of the labour market. People could end up making career decisions that are not consistent with the changes in the labour market, resulting in higher unemployment, and/or skills shortages in the labour force.

In addition to the need to keep LMI resources up-to-date, there are needs that the LMI Strategic Initiative was not able to address. In particular, the needs of people with disabilities for accessible and relevant labour market information. Occupational information is also needed for people with low levels of education.

As a means of improving the functioning of the labour market, labour market information is serving a public good. One benefit of the Initiative is that it was able to fund the free distribution of many of the resources. This is likely an important factor in the Initiative’s effectiveness, as about half the people in the intended audiences do not have adequate budgets to meet their LMI needs. Lack of budget was not a limiting factor for these people because of the widespread free distribution of the resources, both in print and electronically. The ending of the Initiative has meant that there is less funding for the creation and dissemination of LMI resources.

Since audiences do not have enough money to purchase the LMI resources they need, the collection and dissemination of LMI cannot be self-sufficient. To continue to address the LMI needs of career practitioners and end-users, and encourage the further integration of LMI into career planning, continued government funding is needed for the collection, development and dissemination of LMI, and training in its use.

The principal benefits of the LMI Strategic Initiative have been the development of effective partnerships for the collection, development and dissemination of LMI resources and the creation of LMI resources that meet the needs of a diverse range of users. The needs are ongoing, but the shelf life of LMI resources is not. Work on updating resources and developing new resources will have to continue if the needs of users are to continue to be met. Whether the significant benefits that the LMI Strategic Initiative has achieved will be sustained depends on the availability of government funding to continue work in this area.

Footnotes

5	Cooperation Agreement Concerning the Labour Market Components of the Canada/British Columbia Strategic Initiatives, Schedule C, 1995.