Evaluation Study of the
Community Access Program (CAP)

Audit and Evaluation Branch
Industry Canada

January 16, 2004

Executive Summary

Background

Industry Canada’s Connecting Canadians Initiative aims to make Canada the most Internet-connected nation in the world. Connecting Canadians encompasses many programs and services designed for public schools and libraries, First Nations schools, the voluntary sector, rural and remote communities, small businesses and recent graduates, all to encourage Canadians’ use of the Internet.

The Community Access Program (CAP) is a cornerstone of the Connecting Canadians Initiative. It is primarily concerned with the provision of affordable public Internet access to Canadians, as well as the skills necessary to use the Internet effectively. This is accomplished through the use of public locations across Canada (e.g., schools, libraries) as "on-ramps" to the Information Highway, and sources of computer support and training. The program is particularly focused on closing the “digital divide” — the gap between individuals, households, businesses and geographic areas at different socio-economic levels with regard both to their opportunities to access information and communication technologies (ICT) and to their use of the Internet.

Canada’s connectedness goals are achieved through a focus on activities, programs and policies related to the three pillars of a networked nation: Infrastructure, Use and Content. CAP is an important component of the first pillar — Infrastructure: Canadians connected to each other and to the world in a way that is affordable and accessible.

The formal objectives of CAP are to:

• Create affordable public access to the Internet;
• Promote public awareness of the benefits and opportunities of using information and communications technologies (ICT);
• Help citizens become better informed through the exchange of ideas and coach individuals in the use of information technologies;
• Support online delivery of government programs and services;
• Facilitate business activities such as electronic commerce;
• Foster online Canadian content; and
• Support e-learning.

CAP first began in 1994, with a focus on rural and remote communities with populations of less than 50,000. Following a successful pilot, it was expanded to include urban communities in December 1999. Industry Canada’s goal was to have 10,000 sites established in Canada by March 2001. As of March 2002, approximately 9,200 CAP sites had been established, although 1,200 of them have since closed so the actual number of active CAP sites is currently approximately 8,000. Most sites (98 per cent) are organized into CAP networks, or groupings of CAP sites that share a common interest and purpose and are committed to work together in pursuit of common objectives with other partners.

Evaluation Objectives and Issues

The overall objective of the present evaluation study was to provide input to guide decision-making on the upcoming renewal and extension of funding for CAP and, more broadly, on the development of the next generation of the Connecting Canadians Initiative. CAP is currently in the process of developing policy and program options to more effectively meet the needs of Canadians still lacking Internet infrastructure/skills in the changing ICT environment. In preparation for the possible renewal of CAP, the findings and recommendations of previous evaluations and reviews as well as the principles/requirements of the program’s results-based management and accountability framework (RMAF) must also be considered.

The primary focus of the evaluation was on digital divide CAP sites (both Urban and Rural). A number of issues were examined in this evaluation, including questions related to: the continued relevance of CAP; program delivery and implementation; the success of the program; and the cost-effectiveness of CAP and lessons learned.

Methodology

The methodology for the CAP evaluation study consisted of the following components:

• interviews with 30 key informants — including 10 representatives of Industry Canada, Information Highway Applications Branch (IHAB) and Operations Sector at both national and regional offices, eight provincial/territorial government officials and partners, and 12 CAP network coordinators;
• telephone survey of 503 CAP site representatives; and
• incorporation of key results from an online survey of 7,004 CAP site users, conducted by IHAB.

To the extent possible within the available budget and time frame, the evaluation methodology incorporated multiple methods and data from different primary and secondary sources in order to ensure that the findings were valid and captured key points of view on the Community Access Program (i.e., federal government CAP management, provincial/territorial government CAP partners, CAP site representatives and users, and CAP network coordinators). Key informants were carefully selected to ensure that they were knowledgeable and could provide an informed view on the program, though it was beyond the scope of the evaluation to consult independent key informants with no vested interest in the program. Moreover, the site survey findings (n=503) are reliable — results are accurate within ± 4.2 percentage points, 19 times out of 20. The fact that the findings from different lines of evidence were quite consistent lends support to the validity of the evaluation results.

Findings and Conclusions

A. Continued Relevance

On balance, the present evaluation findings indicate that CAP is a unique program that continues to be needed and relevant because there is still a digital divide in Canada and CAP has been having success at bridging this gap in public Internet access and capability. There is a consensus that the formal program objectives continue to be relevant, with some qualifications. First, the basic objectives of raising public awareness of ICT and helping to provide affordable public Internet access are now less relevant (for some segments of the population) than they were at the outset of the program because much progress has been made in many communities. Still, there do appear to be significant outstanding needs for many digital divide areas and groups (e.g., underprivileged Canadians with comparatively low incomes and little education, those living in rural, remote and northern areas). Second, it may not be realistic, nor is funding adequate, for the program to fully achieve some of the broader objectives on its own, in particular, fostering online Canadian content and facilitating business activities such as e-commerce (though this latter objective may become more prominent particularly if CAP moves forward in areas/sites where users have higher-level skills). In addition, it must be acknowledged that it is not reasonable to hold the program solely accountable for achieving all of its formal objectives because individual citizens as well as program partners in province/territories and communities clearly have key roles to play.

The continuing need for the program — to address the “have-not” areas and groups in the digital divide — is supported by a number of findings:

• Key informants believe that there is a continuing need for CAP, particularly to close the remaining digital divide in Canada by reaching segments of the population that have not “bought into” the benefits of ICT and by providing affordable Internet access to citizens who do not have it.
• Supporting this view, results of the Statistics Canada Household Internet User Survey (HIUS) indicate that in 2002, 62 per cent of Canadian households had at least one regular Internet user who accessed it from any location, though just 51 per cent accessed the Internet from home. Home Internet access is much less common for Canadians with lower incomes — for 25 per cent of those in the lowest income quartile compared to 78 per cent in the highest quartile — and in certain regions of the country (e.g., less than 40 per cent in some Atlantic provinces). In related findings, results from the 2000 General Social Survey (GSS) indicate that the greatest barriers to using the Internet among non-users (who had not used the Internet over the previous 12 months) were cost and lack of computer/Internet access.
• Most site representatives (74 per cent) perceive that their sites are used to a large extent by people without Internet access at home.
• In the online user survey, 49 per cent of respondents indicate that they do not own a computer (and 23 per cent do not expect to soon) and the majority (78 per cent) indicate that they use the CAP site because it is free or inexpensive — in line with the program objective to provide affordable public Internet access. Although some duplication is suggested by the finding that some of these respondents also have Internet access at home (40 per cent), from friends (31 per cent), at work (29 per cent), school (21 per cent) or other sources (20 per cent), access at work or school cannot be regarded as equivalent to access at home or a CAP site because there would presumably be much less opportunity to pursue personal interests, communications, learning, etc. at the former locations. Moreover, the online survey would more likely have captured the views of motivated CAP users who are comfortable operating a computer, not those of citizens in the digital divide who lack basic computer skills.

B. Delivery and Implementation

For the most part, CAP is being delivered as intended, is reaching its intended target groups (e.g., people without Internet access at home) and generally serving the digital divide. Still, there are groups that need to be better reached, including persons with disabilities, First nations and northern communities, the homeless and underprivileged, and hard-to-reach senior citizens and new immigrants. French-speaking Canadians also appear to be underrepresented among CAP users. Most sites have some services/equipment in place, such as wheelchair ramps, to help ensure that they are accessible to persons with disabilities but there is still a need for improvement.

Visitors to CAP sites are using them for a range of purposes in line with the program’s objectives (e.g., e-mail, learning and training, job searches, accessing government services and information), and many CAP sites have been innovative in delivering these services. For instance, there are mobile CAP sites that utilize laptops to reach users who might have difficulty getting to a site and CAP sites have been used to facilitate the efficient, integrated delivery of a range of social services. In addition, CAP networks are praised as an innovative and cost-effective delivery model that contributes to community capacity, though one drawback of this approach is that regional IC/CAP representatives do not always have direct communications with CAP sites because they deal directly with the network coordinators. Effective partnerships have also been established — federal-provincial/territorial partnerships as well as site-level partnerships with government, private and community organizations.

C. Success

CAP is perceived to have had considerable success at contributing to its objectives, providing a range of benefits for users and communities, and reducing the digital divide (though there is still more to do in this regard). For example, major perceived benefits include increased knowledge about, comfort with and use of the Internet and ICT; exchange of information and ideas among citizens; social/cultural development and better integration of users into the community (e.g., through opportunities to meet or communicate); and even some improvement in the economic situation of users (e.g., development of job skills, assistance with job search, selling locally produced goods over the Internet). Users are very satisfied with all aspects of the service at the sites, with the exception of the speed of Internet connection. Key informants believe that program progress and success are facilitated by strong community support for CAP sites, partnerships and networks but impeded by some lack of funding (e.g., to keep up to date with changing technology, meet specialized needs and pay staff at the sites) and a shortage of human resources at sites, in particular, volunteers. Volunteer burnout is widely regarded as a key challenge for CAP sites.

The program is viewed as having incremental impacts — one-third of site representatives claim that their site would cease to exist if there were no CAP funding (in particular, for Atlantic, Quebec and digital divide sites), and one-half believe that they would need to offer fewer services if the CAP funding ended. From the perspective of site sustainability, however, this finding may warrant further attention from CAP as the program refines its priorities.

D. Cost-Effectiveness and Alternatives

Although a thorough cost-effectiveness analysis was well beyond the scope of this evaluation, the available evidence does indicate that CAP is widely viewed as a cost-effective program, providing numerous benefits for a small investment in sites (an average of approximately $4,412 per site). With the federal funding, many sites have been able to leverage considerable financial and in-kind resources from other sources/partners (e.g., local and provincial/territorial government). The CAP network model — which allows bulk buying of equipment and sharing of best practices — also contributes to cost-effective delivery, as does the heavy reliance on volunteers at sites (though this benefit is diminished as volunteers burn out and need to be replaced and retrained). Moreover, approximately 60 per cent of the sites surveyed have taken at least some steps to support their sustainability, in particular, searching for alternate funding sources. Still, many sites suggest that more or longer-term funding would be beneficial (e.g., for new or upgraded computer equipment and high speed Internet connection) and, as noted above, not having some CAP funding would apparently have adverse consequences for approximately 80 per cent of sites (i.e., having to offer fewer services or close the site).

Recommendations

On the basis of the evaluation findings, the following recommendations are made to Industry Canada:

1. Refine and refocus the program’s strategic priorities (e.g., through a strategic planning exercise).

• Place the most emphasis on digital divide sites, serving communities most in need and addressing remaining gaps in Internet access in Canada. In these areas, continue to raise public awareness, provide affordable public access to the Internet, and coach community members in the use of information and communications technology (ICT).



• At digital divide sites where users have sufficient capability, focus on higher-level applications such as supporting e-learning and online delivery of government programs and services, facilitating e-commerce, and applying higher-end technology.

2. Improve the marketing of the program and its benefits/potential applications.

• Promote CAP to other government departments (e.g., to facilitate on-line delivery of government services at more CAP sites).
• Promote CAP to “hard to reach” and “have not” target groups, such as First Nations and northern communities, the homeless and underprivileged, persons with disabilities, seniors and new immigrants. Focus promotional efforts on people/communities who may not yet fully understand the benefits of ICT (e.g., underprivileged people who tend to be preoccupied with very basic needs such as food and shelter).

3. Continue to improve the accessibility of sites for persons with disabilities. For example:

• Conduct a proper assessment of these users’ needs to ensure that all features of sites are accessible (e.g., provide not only a ramp for wheelchairs, but also desks that are a suitable height for wheelchairs).
• Offer equipment/technology suitable for these users, e.g., the option of a track ball rather than a mouse.

4. Continue to utilize CAP partnerships and networks, and provide opportunities for network coordinators to share lessons learned and best practices (e.g., through mentoring activities, websites or workshops).

5. If feasible, provide multi-year funding for a renewed CAP and for CAP sites.

6. Assess the feasibility of increasing the funding amount to selected CAP sites (e.g., to strengthen/expand particularly busy or innovative sites; to assist sites with special needs and fewer opportunities to raise funds). This may involve the re-allocation of funds from sites that are more self-reliant to those with more need for IC funding. More funding would enable sites to:

• Purchase new computers.
• Upgrade existing computers and provide high speed Internet (broadband) connection.
• Improve the accessibility of the site, if needed.
• Pay for qualified staff — which would also help to overcome the problem of volunteer burnout/lack of volunteers.
• Remain fully operational with a range of needed services.

7. Establish (or review existing) service standards for CAP sites (e.g., minimum number of qualified staff, services offered, hours of operation) to ensure that levels of service are reasonably consistent across the country and compatible with evolving program priorities, while allowing some flexibility for sites to adapt to the needs of their users and community.

• In order to incorporate this flexibility and responsiveness to individual community needs, assess the feasibility of utilizing a tiered system of CAP sites offering different levels/types of service along a continuum, depending on the needs and capabilities of users. For instance: (1) basic Internet access, training and services for communities with little or no exposure to ICT; (2) intermediate services; through to (3) high-level services and application of advanced technology at sites with good Internet connectivity and more experienced users. If applicable for a given community, offer different levels of service at the same site.

8. Keep the database of CAP sites up-to-date and accurate so that the data can support program management, performance measurement and periodic evaluations.

Final Report (PDF, 656KB, 56 pages)

Management Response — January 2004 (PDF, 119KB, 6 pages)

Management Response – Update May 31, 2005 (PDF, 36KB, 8 pages)

Note: to read the PDF version, you need Adobe Acrobat Reader on your system. If the Adobe download site is not accessible to you, you can download Acrobat Reader from an accessible page. If the accessibility of PDF is a concern, you can have the file converted to HTML or ASCII text by using one of the access services provide by Adobe.

Printer-friendly Version

Date Modified: 2006-02-08 Important Notices