Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

This section presents a summary of the main findings of the evaluation and the areas identified for improvement.

10.1 Summary of Main Findings

There continues to be a need for a national program that encourages the use of learning technologies to develop new technical skills and upgrade old skills — particularly in smaller, isolated communities. Currently, the OLT appears to be almost unique in Canada in funding projects that link learning technologies with non-institutional learning, such as community or lifelong learning, while maintaining an overall program focus on skills development and training for innovation.

Stakeholders also see a need for OLT funding and believe that program objectives are still valid. In the future, the federal government and the OLT are seen to have an important role in:

• Facilitating infrastructure development and access;
• Developing and facilitating partnerships; and
• Providing financial assistance and funding.

A wide variety of organizations applied for and received OLT funding. The types of organizations submitting applications and receiving funding from the OLT include educational institutions (39%), professional groups (28%), non-profit organizations (24%), public institutions (7%) and Native groups (2%).

The OLT helped develop partnerships with private sector, community and non-government organizations. The case study and survey analyses indicate that OLT funding supported partnership development, for example by making it possible for the project to be developed or to be developed with a broader scope and impact. At least one new partnership was developed by 77% of the surveyed project sponsors as a result of participating in the OLT program, and most sponsors developed more than one partnership. The range of project partners included community-based organizations (75%) and the private sector (33%).

Project sponsors attributed the OLT's success in developing partnerships to the emphasis the program places on the partnership approach and the ability of the program to provide focus and direction to community organizations looking to utilize learning technologies.

Project partners made a variety of contributions to OLT projects, including additional funding, in-kind-resources, content development, and expanded delivery networks. The case study analysis suggests that each dollar of OLT funding leveraged an average of $1.50 from project sponsors/partners.

OLT funded projects have contributed to increased access to and use of learning technologies among a wide variety of designated equity groups. The OLT has funded projects to promote learning technologies through the development, testing and use of a wide range of technologies (including computer-based training, the Internet, the development/use/testing of a website, and the development of specialized software or CD-ROM). For example, the many objectives of CLN funded projects included reducing barriers (23%), increasing learning skills (23%) and increasing accessibility to technologies (27%).

The evidence indicates that OLT projects had considerable success in reaching designated equity groups (e.g. persons in remote/rural areas, women, youth, persons with low literacy skills, seniors, and persons with disabilities). The evidence also indicates that OLT projects have worked to identify and respond to the needs of end-user groups identified by the project.

The available evidence on end-user impacts indicates that OLT projects benefited end-users. End-user focus groups for three of the four CLN project case studies identified a number of impacts, including increased confidence in learning and using computers, and reduced social isolation. The survey of project sponsors indicated that OLT projects have increased access to learning technologies and have helped workers take advantage of technology (82%), facilitated lifelong learning in the community (81%), and helped develop a more skilled workforce (77%).

OLT funding appears to be an important factor in the development of projects aimed at using technology to promote learning. Only 6 % of the non-funded projects proceeded in full without OLT funding, and another 28% proceeded in a reduced form. The rest (66%) did not proceed. The major reasons given by non-funded applicants for projects not proceeding after not being accepted by the OLT was the lack of alternative sources of funding.

The evidence suggests that the partnerships and projects developed through the OLT would continue in some form after their OLT funding support ends. There is considerable evidence that contributions made by project partners (e.g. additional funding, in-kind resources, content development, expanded delivery networks) were recognized by project sponsors and partners. Also, project sponsors and partners indicated high levels of satisfaction with their experiences collaborating for the OLT funded project. The value and satisfaction associated with their partnering experience under the OLT suggests that these groups will look for ways to continue partnering in some form after their OLT funding support ends.

In support of this conclusion, the survey of project sponsors indicated that the majority (82%) anticipated that partnerships developed for the OLT project would extend beyond the completion of the project. The experience with LTI funded projects, which generally ended in 1999, also corroborates this conclusion (with 73% of the partnerships reported to have extended beyond the life of the project).

The evidence also suggests that many OLT projects would continue in some form or produced outputs/products that will be used after OLT support ends. For example, evidence from the ten case studies indicates that two projects either continued or planned to continue, five projects involved one-time development activities to produce outputs/products that are either in use or expected to be used after project completion, and the remaining three projects are too new to determine whether they will be sustainable. Similarly, more than three-quarters (78%) of the surveyed project sponsors felt that their project would continue in some form after the end of OLT funding.

10.2 Areas Identified for Improvement

The broad over-reaching objective of the OLT needs to be recast in more realistic terms. The OLT was established with an annual budget of $6 million. The current objective of the OLT is broad and over-reaching in aiming to reduce the economic and social divide between those who have computer skills and those who do not. The current over-reaching objective needs to be recast to be something that a program with a limited budget can realistically achieve. An example might be to make the objective to improve computer skill sets of end-users and reduce their reluctance/inability to use newer technologies.

Program controls need to be put in place to minimize the risk that funds leveraged by OLT projects are not incremental, and to safeguard against overlap and duplication. Although there is evidence that OLT projects are leveraging additional funds, the current program design does not include safeguards to reduce the risk that leveraged funds are not incremental (i.e., would have been used for similar activities in the absence of the program).

There is a need for the program to be able to demonstrate that it has safeguards in place to prevent, or at least minimize, the risks that leveraged funds are not incremental and that OLT projects would have proceeded in the absence of the program. One method of accomplishing this could be to require the project sponsor to attest that the project is incremental (i.e. would not have gone ahead without OLT funding).

Similarly, given that some of the unaccepted projects gained funding from other sources, there is a need for the program to be able to demonstrate that it has safeguards in place to prevent, or at least limit, any overlap or duplication of activities funded by other sources (e.g. by coordinating activities with NGOs, other levels of government, etc.).

While OLT has been able to reach designated equity groups, certain groups may require additional program effort to be adequately reached. While OLT funded programs have been relatively successful in reaching persons in remote/rural areas, women and seniors, there has been less success in reaching the Aboriginal community, young mothers and ESL populations. Aboriginal groups, especially, may require a targeted initiative to address the multiple issues identified in serving this population.

There is a need for better outcome tracking at the end-user level. There is a need to provide for the tracking of end-users, for example by requiring project participants to register with project staff. The requirement for end-user data collection will be included in the new RMAF. It will be important, however, to test new tracking methods in a variety of end-use settings to ensure that the new methods can be easily applied without undue impact on project administration by sponsors or on program reach.

Project sponsors and partners lack experience partnering. Partnership activities required considerable resources and were particularly challenging for organizations that were inexperienced in partnering or that lacked the required knowledge to develop and mobilize formal partnerships (i.e. non-profit and community organizations). Lack of knowledge and partnering experience impacts both the development and maintenance of partnerships and can lead to conflicts during project development. In general, organizations would like more direction and assistance from the OLT in terms of the development and maintenance of partnerships.

OLT staffing levels and staff turnover have impacted the program. There is a perception that there is insufficient OLT staff to manage current projects. Further, high staff turnover within the OLT has created an impression that staff are not sufficiently familiar with the program or with learning technologies. Generally, project sponsors would like more interaction and direction from OLT staff.

There is a strong perception that OLT should allocate more resources to understanding and disseminating best practices and lessons learned. It is felt that OLT can act as a data warehouse for learning technology information, research and findings from projects. Stakeholders also requested additional opportunities for project sponsor interaction and information sharing between and within project initiatives.

Delays in receiving OLT funding (i.e., from the developmental phase to pilot phase as a result of the application process) resulted in waning project momentum and interest among partners, sometimes resulting in partner drop out. Informants and sponsors suggested that there be a "fast track" procedure for obtaining funding between the developmental and pilot phases, for projects that are already running. The issue of waning partner interest was more salient when community organizations were involved, given the unstable nature of their internal funding and staff resources.