Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

This section examines the impacts of the OLT program by examining project sustainability and the impact of the program on project sponsors and partners, end-users and the community. Once again it should be noted, however, that it is difficult to determine precisely to what extent the impacts would not have occurred without the OLT because of the lack of program safeguards to limit the risk that projects and leveraged funds were not incremental (as discussed in Section 5.4.2).

9.1 Project Sustainability

The evidence suggests that many OLT-funded projects would continue in some form after OLT support ends, or produced outputs/products that will be used after OLT support ends. An assessment of project sustainability for the ten case study projects indicates that:

• Two projects either have continued or plan to continue;
• Five projects involved one-time developmental activities (e.g. a pilot test, testing a new learning model, designing a technology-enhanced health and safety program) had either just terminated or plan to terminate when the final deliverable is complete, although the resulting products are either in use, or expected to be used after project completion; and
• Three projects were too new to determine whether they will be sustainable.

The surveyed project sponsors were asked to indicate the extent to which their current project would continue once OLT funding expired. As highlighted in Chart 9-1, more than three-quarters (78%) of project sponsors felt that their project would continue in some form after their OLT support ends.

Chart 9-1 also indicates that a significant proportion of the surveyed project sponsors (54%) noted that they expected the proportion of their project that would continue would constitute more than fifty percent of the activity funded by OLT. About a third (35%) indicated that fifty to ninety percent of their project would continue and nineteen percent (19%) indicated that one hundred percent or the entire project was expected to continue.

Many of the key informants felt that it was too early in the life of the projects to know whether they would be sustainable. However, some believed that it was already evident that some projects are sustainable in the long term, while others are not. Key informants also noted numerous "offshoots" have led to additional projects that are currently in place without OLT funding.

For many informants the biggest issue surrounding sustainability was funding. Necessary for sustainability is the need for project sponsors to assume some of the project cost currently funded through OLT, or continued involvement by OLT or government, as there are insufficient resources at the local level to sustain projects.

Key informants suggested that the OLT could further sustain projects by funding projects for a longer durations and incorporating the notion of sustainability as a condition of funding, as a way of addressing the notion held by some project sponsors that OLT funds "pilots" and that completing the pilot means that the project is over.

9.2 Impact on Project Sponsors and Partners

Most project sponsors indicated that their project would not have gone ahead without OLT funding or would have proceeded on a more limited scale. As noted in Section 6.4, the majority (85%) of project sponsors surveyed noted that they would not have undertaken the same project without OLT funding. For example:

• Some sponsors noted that they would have had fewer partners had they not received OLT funds;
• Some partners indicated that their project would have been of shorter duration had they not received OLT funding; and
• Several project sponsors noted that the scope of their project would have been considerably reduced had they not received OLT funds.

Several project sponsors noted that the provision of OLT funds helped ensure that adequate evaluations were completed during the project to identify lessons learned, as well as to provide direction for required changes in program delivery.

Project sponsors indicated that interaction with the OLT had increased their knowledge of learning technologies. In addition to the impact of the funding, project sponsors were asked to comment on the impact of working with the OLT for their own organization. Project sponsors were positive in terms of the information acquired through their interaction with OLT. For example, interaction with OLT increased sponsors knowledge of:

• New approaches in learning technologies (88%);
• Learning technologies information (80%); and
• Trends in the use of learning technologies (59%).

9.3 Impact on End-Users

There is a need for better outcome tracking at the end-user level. The evaluation of program impacts on end-users was limited because the program never required the collection of end-user client data from the funded projects and end-user/client information was unavailable from OLT administrative databases. The requirement for end-user data collection will be included in the new RMAF. It will be important, however, to test the new tracking methods in a variety of end-user settings to ensure that the new methods can be easily applied without undue impact on project administration or on program reach.

The available evidence indicates that OLT projects had beneficial impacts for end-users. In order to partially compensate for the unavailability of end-user data, focus groups were conducted with three of the four CLN case studies. Each focus group consisted of nine end-users and identified the following positive impacts on end-users:

• Vancouver Community Network: Project targeted on individuals with low income and literacy skills. The focus group was conducted with seniors. The major impact on senior end-users was decreased social isolation through learning Internet and e-mail skills, increases direct social interaction at a drop-in computer center developed and staffed through the project, and increased interest and capacity to use learning technologies;
• Monashee Learning and Training Centre: Project targets on increasing access to lifelong learning and to increase the presence of on-line businesses. Benefits to end-users were increased confidence in learning and a reduced aversion to using computers. Some project participants are teaching at the center; and
• Creative Retirement Manitoba: Project aims to provide seniors with necessary government and community information by on-line access, combined with courses and mentoring programs to improve seniors' computer skills. Benefits to seniors include reduced social isolation, intellectual stimulation and a feeling of independence and accomplishment.

As indicated in Table 9-2, the surveyed project sponsored and partners indicated that OLT funding had achieved the following:

• Helped workers take advantage of technology (82%);
• Facilitated lifelong learning in the community (81%); and
• Helped develop a more skilled workforce (77%).

Site visits conducted for the case studies also included interviews with the project sponsors and partners, and these interviews cited further examples as to how OLT funded projects benefited end-users. As noted in Section 6.3:

• The Alberta Forest Products Association (AFPA)'s Teaching Literacy as a Safety Initiative in the Forest Industry project is developing a CD-ROM to provide health and safety training to forestry workers for low literacy skills in remote locations. When the deliverable is completed, the project sponsors and partners maintain that the overall impacts will include new safety, literacy and computer skills for these employees.
• The Office of Partnerships for Advanced Skills (OPAS) Worker Online Re-skilling Centre (WORC) is providing Technology Mediated Learning (TML) to working adults (especially mid-career, older persons) in need of skills upgrading. The sponsor and partners felt that the use of TML in the culture/art sector (their first targeted sector) will be effective as a low-cost alternative to on-site training, reach rural and remote groups, and provide opportunities for end-users to develop broader networks.

9.4 Impact on the Community

The available evidence indicates that OLT projects have beneficial impacts on the community. The surveyed project sponsors and partners also cited community-level benefits associated with OLT funded projects. Project sponsors and partners agreed that funded projects have:

• Broadened access to learning opportunities (95%);
• Supported community capacity building (82%);
• Raised awareness of learning opportunities (78%);
• Facilitated the development of community expertise in learning technologies (74%); and
• Helped develop the communities' technological infrastructure (66%).

The OLT program contributed to improved accessibility to learning technologies. Almost half (44%) of all CLN project were directly linked, through the project partner or sponsor, to a CAP site. Additionally, by funding CLN projects facilitated by community colleges or institutes (22%) and not-for-profit organizations (38%) that serve the community, OLT has further increased accessibility to learning technologies. With respect to the many objectives of CLN funded projects, most were aimed at increasing learning skills (23%), reducing barriers (23%) or increasing accessibility to technologies (27%).

The case study evidence provides some specific examples of OLT funded projects that increased accessibility to learning technologies by providing learning opportunities through the use of various technologies.

• Accessible Adult Learning in the Health Professions Program (Dalhousie University):Through the Faculty of Health Professions, the project provided distance education courses to students at the undergraduate and graduate level using teleconferencing, email, and Internet or web-based learning.
• Computer-Based Adult Learning Technologies (Fanshawe College): The project utilized a learning assessment and recognition (PLAR) strategy and various technologies (e.g. Internet, conferencing software and audio-tactile network) to assist equity groups to learn new technical skills. End-user groups included:
  • unemployed and underemployed individuals;
  • women and single parents;
  • learners with disabilities;
  • individuals with low levels of literacy and English proficiency; and
  • individuals in remote or rural areas.