HRDC - Summative Evaluation of the Office of Learning Technologies

Directorates

· Director General's Office

· Data Development

· Strategic Evaluations

· Partnership and Continuous Evaluations

Publications

· Evaluation Reports

· Evaluation Briefs

· Labour Market Development Agreement Series

· Lessons Learned Series

· Evaluation Tool Kit Series

· Evaluation Forum Newsletter

· From Theory To Practice

Appendix B Summary of Case Studies

This appendix provides an overview of the key issues/findings drawn form the ten OLT projects sites visited as part of the evaluation. It should be noted that due to differences in the sites (i.e. some sites have been operational for 1-2 years, while other sites had been operational for only a few months, or the project had ended) it was not possible to identify outcomes/barriers for all projects.

The following issues are presented:

Project overview;
Target population/client group;
Client needs (objectives of the project);
Extent to which client needs/project objectives were met;
Identification of barriers/factors that affect achievement of the objectives; and,
Other issues.

Table B-1 Case Study Summary: Learning Technology (LTI) Projects
Project/Issue	Dalhousie University — Accessible Adult Learning in the Health Profession's: Interactive Use of Technologies	Fanshawe College — Computer- Based Adult Learning Technologies	Royal Roads University — Community Service Learning
Project Overview	distance learning for health professional from four schools/one program invested in technology systems to support distance education to identify strengths and weakness of e-learning for health professionals	to pilot test and review the effectiveness of computer mediated learning (CML) for groups that typically encounter barriers or have limited access to learning technologies to develop capacity/expertise in the community to deliver computer-based training to disadvantaged populations ("train the trainers")	distance learning for individuals in the community service (i.e. Non-profit sector) to earn a graduate (Masters level) degree targetted both the student and the not-for-profit organization involved a significant degree of mentoring (student and agencies) to allow students/mentors from across Canada to share knowledge and experiences
Target Population/Client Groups	health professionals located outside of Halifax 38% of students were in rural locations	unemployed/under employed women/single parents learners with disabilities individuals with low literacy levels individuals in rural areas that lacked computer/technology infrastructure	individuals working for/with not-forprofit organizations community-based learning (distance education program)
Client Needs/Project Objectives	to reduce the need for health professionals to move to either upgrade/enhance skills to allow individuals in rural/remote regions access to the program to enable clients in centers outside of Halifax to participate in the program	to provide groups that typically do not have access to computers/learning technologies with computer assisted training/upgrading to reduce barriers and/or increase the comfort levels with the use of computer-based learning technologies to allow individuals to upgrade/acquire skills at a pace/time convenient to them (flexible training model)	this project was developed to test the efficacy of distance learning for individuals in the not-for-profit sector. Project was designed and delivered in 1996 — was considered to be a pilot project for the use of distance learning models for Royal Roads University
Degree to Which Project Objectives Were Met	only limited end-user information was available, but evaluation reports indicate a high level of student/staff support for the project enrollment data indicates that a high proportion (38%) of students were in rural/remote locations	the evaluation that was completed for the project noted that the program was generally successful in terms of attracting end users to the centres and providing "marginalized" groups with an alternative avenue to acquire additional skills/upgrading	as a new pilot project at a new University (Royal Roads had only recently become a public university in 1996) the project was characterized by numerous "teething" problems including lack of structure and limited ability for students to interact with faculty.
	there were some difficulties in terms of student/staff comfort with technologies	the partnering associated with the project resulted in considerable synergies with other community-based learning organizations (the scope of the project was seen to be very wide) there was anecdotal evidence of student success including those students who went on to further education/those who decided on a career path and those that found work as a result of their training	information obtained during the case study suggested that while the project was positively viewed by some students, other students were disappointed with the program. Some of the shortcomings of the program could be attributed to the limited technology in place in 1996 (high speed internet connections were not common — contributing to slow access times to web-based information) as well as the lack of experience in delivering on-line instruction by the faculty. it was noted that the program contributed to information-sharing among students in the not-for-profit sector and generated "partnerships" among mentors and students
Barriers/Factors Affecting Achievement of Objectives	lack of faculty/student comfort with the technology student/faculty ratio's were originally too high (1 to 25-30 but should have been 1 to 18-20) health professionals still require some "hand's on" instructions, needed for faster turnaround on some issues	technology was not available/developed/appropriate for those with a disability (deaf/blind) individuals with low levels of computer proficiency required considerable "hands-on" support (technology could not be provided without considerable support especially for first time users) the project sponsor noted that the partnerships required considerable time/effort to establish and maintain	technology was not highly developed faculty was new to using distance learning approaches program lacked processes to allow for more "one on one" time between faculty and students
Other Issues	OLT funded project demonstrated the ability for programs to co-operate in the e-learning field project was noted to serve as a catalyst for other distance education projects in the university project continued without OLT funding, but in a weaker form, and partners are looking for other sources of funding	overall, it was felt that the program developed self-esteem and career goals for the end users it was noted that the technology should not be viewed as a replacement for traditional learning, but rather, as a way to complement traditional learning models (some face to face instruction/assistance would still be required for the end users that the project targetted (i.e. Unemployed, single parents, individuals with low literacy levels etc.). The technology did, however, allow learners to progress at their own pace and not force learners to follow rigid learning schedules/structures	students felt that learning through this "non-conventional" approach (in 1996) was valuable in itself (students had to learn self-discipline, team problem-solving in a distance environment). Relationships developed during the program translated into support networks after program completion program was viewed as an important learning project for the institution, changes were incorporated into subsequent e-learning/distance education programs that reflected the lessons learned through the Community Service Learning project outcomes of the project included replication of the new distance learning model in other continuing education initiatives of the university — and use of the model by other interested universities and learning institutions

Table B-2 Case Study Summary: Learning Technology in the Workplace (LTW) Projects
Project/Issue	Alberta Forest Products Association — Teaching Workplace Literacy as a Safety Initiative in the Forestry Industry	Office of Partnerships for Advanced Skills (OPAS) — Worker Online Reskilling Center (WORC)	Conference Board of Canada — Learning Technologies in the Workplace
Project Overview	design a CD ROM to teach a health and safety program to Alberta forestry workers, while at the same time addressing functional and computer literacy issues among the targeted end-users the project developed partnerships with colleges (for their experience in adult learning/literacy issues), a private communications company (to develop the CD), and key industry representatives (provided technical and content knowledge)	design a CD ROM to teach a health and safety program to Alberta forestry workers, while at the same time addressing functional and computer literacy issues among the targeted end-users the project developed partnerships with colleges (for their experience in adult learning/literacy issues), a private communications company (to develop the CD), and key industry representatives (provided technical and content knowledge)	to identify technologies, operational processes and strategies that would increase and improve learning in the workplace to identify management processes/actions that would promote the investment in workplace-based training/learning project was in the development stage at the time of the case study this was a research project that was designed to identify the barriers to workplace training and develop solutions/consensus as to what changes could be made to enhance the incidence of workplace-based training
Target Population/Client Groups	employers and employees of the forest industry in Alberta employer companies range from large multi-national companies to small independent business	employees (to date course content only developed for one industry sector visual arts)	as part of the research project, the primary target populations were employers, although it was noted that through the research it would be hoped that strategies would emerge that would include government, unions, workers and employers
Client Needs/Project Objectives	workplaces include remote, mobile work camps with limited access to formal classroom training — project objects: increase access to safety training employee groups include workers with poor literacy skills/English as a second language, among which accident rates tend to be higher — project objectives reduce accident rates and increase literacy skills	project objectives include developing increased familiarity of TML among working adults, increase working adults accessibility to skill-upgrading, creation of knowledgeable trainers (in a work place) to demonstrate the advantages of TML in the workplace, translate content of «face-to-face» training to on-line	to identify the benefits of in-house learning programs to Canadian employers to identify the strengths and weaknesses of learning technology in the workplace (availability, best practices, constraints etc.) to identify how learning technologies could be best used in the workplace — to determine which training programs can be best used in the workplace to help employees learn and contribute to higher worker productivity.
Degree to Which Project Objectives Were Met	CD-ROM was still being developed, not yet tested with end-users	relatively new and not yet completed (training course content developed for only one sector - visual arts)	unknown at the present time as project has just started
Barriers/Factors Affecting Achievement of Objectives	project partners located in diverse locations through out Alberta, had increased project start-up time computer literacy and general literacy levels a barrier to CD-ROM use in the field among end-users small companies may have limited technical infrastructure for on-line program delivery (CD-ROM designed to address issues as it only required single work-site computer)	access to technology and the local technological infrastructure was noted to be a barrier in utilizing TML to train employees, particularly in sectors in which employees work as contractors or have limited funds the degree to which learning technology is incorporated in the existing workplace culture attitudinal barriers of members of the cultural sector to technology and the use of technology the need to develop "user-driven" content in training programs, which matches the learning styles of the targeted end-users	unknown at the present time as project has just started
Other Issues	project is expected to terminated when the final deliverable is completed, but use of CD-ROM is expected to produce immediate and long-term (e.g. safer and healthier workforce, new literacy, computer and safety skills for employees, and increased confidence with technology)	lack of experience in development and sustaining formal partnerships a barrier partnerships negatively affected by partner fatigue and delays in receipt of project approval/funding project momentum negatively affected by delay between the developmental and pilot phases	project would not have occurred without OLT because benefits from research are not immediate and , therefore, partnerships were limited and difficult to create project expected to provide more sophisticated research models for e-learning and employee productivity

Table B-3 Case Study Summary: Community Learning Networks (CLN) Projects
Project/Issue	Vancouver Community Network (VCN) — Vancouver Community Learning Network	Monashee Learning and Training Centre — Monashee Learning Community Partnership	L'@venue Inc. — The Community Portal (arrondissement.com)	Creative Retirement Manitoba — Senior Learning Network
Project Overview	provision of Internet tools/access and training at community sites (as of Nov. 1, seven Internet access sites had been established for members, clients and the public) internet tools provided include interactive service directories, neighbourhood learning exchanges and portals project partnered with CAP were possible volunteers for training provided through the Technical Volunteer Web (TVW)	establish a hub/centre of services centered around existing organizations in community services offered are to facilitate life-long learning through technology based learning programs tailored the needs to the community (high unemployment rates, poor economic diversification, rural) and encourage individuals to remain in the community	provide accessible computer information technology and training to an area of Montreal with economically disadvantaged residents develop a community portal with access to community services (mix of content and services) allow participating community groups (approx. 100) and government bodies to develop and edit their organizations message/content on the Portal disseminate local news and issues to the community, provide discussion groups and calendar of community events	provide seniors with government and community information on-line through Community Connections resource networks and access sites provide on-site personal assistance to seniors in technology through courses and mentoring programs partnering with community organizations for learning network content
Target Population/Client Groups	community residents, including seniors; multi-barrier individuals (low income & low literacy), minority groups, artists	all community members of Lumby and surrounding rural/urban areas	community residents of Mercier/Hochelaga- Maisonneuve (including, low-income, disadvantaged youth, women and seniors)	seniors and their families in Manitoba
Client Needs/Project Objectives	client needs dependent upon community and client group (i.e. senior- need to reduced social isolation; multi-barrier individuals — need for increased literacy skills, reduced fear of technology, better understand of technologies applicability to their lives; artists — need to understand the use of technology in business development) overall project objectives are to increase access, understanding and use of learning technologies among disadvantaged groups	limited access to learning opportunities given remoteness of community — project objective: increase access poorly diversified (forestry industry) local economy results in reduced economic opportunity — project objective: provide new learning opportunities in additional fields of employment project objective to reduce poor connectivity, address fear and mistrust of technology and increase of understanding of the application and usefulness of learning technology in community	economically disadvantaged community, citizens can not afford to access technology — project objective increase access community members have limited knowledge/access to information affecting them directly (i.e. health information, tenants' rights, community events) — portal developed to increase access to information	client group has poor technological skills, fear of technology, few resources to access technology, can be socially isolated and have limited access to important information that impacts their quality of life objectives of the program are to collect and organize e-information in a senior friendly useable format, develop training tools for seniors, provided on-site instruction and technology courses for seniors
Degree to Which Project Objectives Were Met	project partnered with community groups to identify and tailor technology delivery to specific needs of client groups partnering allowed access to multi-barrier/marginalized groups focus groups with senior endusers indicated program helped decrease social isolation (by teaching them how to e-mail family and friends), increased direct social interaction at Internet-access sites, and increased interest in and capacity to use learning technology	project partners/ participants indicated program addressed travel barriers to continuing education provided end-users training opportunities in new fields of work and updated technological skills end-users indicated program increased confidence in learning, reduced fear of technology and increased individual connectivity (through center technical infrastructure)	portal not yet launched, impossible to determine whether objectives met with end-users project has developed numerous partnerships (approx. 100 community and 10 government) that will provide content for the portal	learning network currently being developed, some community organizations developed ability to post modify information project partnered with seniors groups were instruction and technology courses tailored to seniors were being provided seniors in end-user focus group indicated courses had improved their technological skills, increased their confidence and ability to access information on the net and the learning network
Barriers/Factors Affecting Achievement of Objectives	partner resource levels (lack of resources for infrastructure; staff) client fear/mistrust of technology inability to access specific isolated target groups (women, First Nations) lack of resources to address physical handicaps, provide access to one-to-one training, training in literacy/numeracy	center's lack of experience in development and sustaining formal partnerships a barrier	low level of community access to high speed internet	seniors learning needs not addressed by current provincial government, limits funding available to seniors community organizations income level a barrier to seniors access technology (i.e. computers/Internet) fear of technology and lack of understanding of use/relevancy barrier to senior use of technology
Other Issues	lack of experience in development and sustaining formal partnerships among community organizations a barrier lack of tracking of end-users partnerships negatively affected by delays in receipt of project approval/funding (between developmental and pilot phases)	lack of end-users tracking program start-up time longer than expected sponsors did not feel that project was ready to become sustainable, but it might be sustainable by end of OLT funding	too early to assess impacts on end-users	CAP site often take on the specific direction of host organization and therefore do not fulfill larger community need and difficult to use in current project staff looking for ways to sustain project

Last Modified: 2004-04-30	Important Notices