Francommunautés virtuelles
Program Evaluation
Final Report
October 25, 2000

Summary

Goss Gilroy Inc. was mandated by Industry Canada to evaluate the Francommunautés virtuelles program.

• The overall goals of this evaluation are to determine to what extent:



• the rationale behind the Francommunautés virtuelles program is valid;
• the program design is adequate;
• the execution and management of the program have been effective; and
• the program has reached its objectives.


• The goal of the evaluation is also to recommend changes aimed at improving the program.



• Francommunautés virtuelles is a pilot program ending in March 2001. Its aim is to expand French-language content on the Internet, to connect Francophone and Acadian communities across Canada, and to encourage French-speaking Canadians to use new information and communications technologies. To achieve these objectives, the program allocates contributions to applicants proposing projects meeting one or more of these objectives.



• The methodology used in this study included an analysis of administrative documents and data, key interviews, interviews with project leaders and a written questionnaire sent to all project leaders.

Study Results

The results of interviews and the written questionnaire can be summarised as follows.

Program rationale

• Generally, respondents agreed that the program is relevant and that there is still a need for it. Its objectives were deemed compatible with the Department's mandate and its sustainable development priority. The program contributes to the development of a knowledge-based economy while developing community life. Finally, there are no similar programs, offered by Industry Canada or any other Federal agencies, which aim at Canadian Francophones across the country.



• The program would benefit from a few changes, including an expanded budget. Additional economically-oriented projects would make the program even more relevant for the Department.

Program design

• Several respondents felt that the program should support wider-scope projects than current budgetary restrictions permit, although all agreed that current contribution limits were adequate for small-scope projects. The regional distribution of projects was debated.



• Respondents stated that the program should include the possibility of supporting projects spanning two or three years, but felt that one year was sufficient for the majority of projects.



• Project themes were considered relevant. The current selection criteria were generally deemed satisfactory, although the interpretation of criteria was sometimes problematic.

Program delivery

• Program promotion was deemed effective by the majority of respondents. However, the results of the written questionnaire suggested that a significant portion of applicants learned of the existence of the program through word of mouth, thus raising serious concerns with regard to the effectiveness of the promotional strategies.



• Some aspects of the program structure were considered overly complex. While the participation of the Advisory Committee was perceived as a strength for the program, reducing its size would be beneficial. In addition, while Public Works and Government Services Canada was perceived to be effective in managing accounts and payments, respondents felt that this role would be better served by Industry Canada.



• It was noted that streamlined applications would simplify the Selection Committee's work, as would nominating a President.



• Most project leaders were satisfied with the program services rendered, however some dissatisfaction was expressed regarding payment delays and progress report requirements.

Program results and impacts

• According to the results of the written questionnaire, for each $30,000 invested by the Department, the program simultaneously contributed to: mobilising over $100,000 from various sources to implement the projects; making available over 175 Web pages and over 100 original French documents on the Internet; training 40 people for one hour each, on average; generating over 500 visits per month on Francophone sites created through the program¹; and creating three jobs during the funding period, of which one was sustained after this period. On average, three financial partners and 16 non-financial partners participated in each project. Finally, according to most project leaders interviewed, the project results could not have occurred without the support of the program, a good indication of its added value.



• The evaluation also provided program results according to four project themes:



• Networking. Generally, increased networking was observed between Francophone associations, educational institutions, companies and individuals through the projects. This networking occurred as a result of the collaboration required to complete the projects and program financing guidelines requiring the diversification of financing sources.



• Creation of a virtual Francophone community. The close ties created reinforced a sense of belonging among the various groups involved in, and benefiting from, the projects. This feeling was reinforced through content creation projects and those fostering information exchanges among communities through Internet sites.



• Content creation. Several educational and cultural Web sites aimed at the general public were created, as well as commercial sites. These sites were effectively targeted and responded to the needs of Francophones, notably those residing in remote regions.



• Development of knowledge and skills. Numerous projects developed Internet use and Web page creation skills. On one level, projects with training objectives were effective in training people with minimal Internet skills. On another level, other projects often led to training people in Web page creation for the purposes of content projects.


• Among the strengths of the program, its unique design, its objectives and its flexibility were most noted. According to respondents and as evidenced by the high contribution demand, the program responded to a genuine need among Francophone communities across Canada. Among its weaknesses, the overall program budget was deemed too limited given its objectives.

Consultant's opinions and recommendations

• Given the observed program impacts, the needs of Francophones and the Department's mandate regarding the information highway, the consultant recommends that Industry Canada consider renewing the program and expanding its budget.



• The introduction of a two-tiered program is suggested: one for wide-scope projects, some of which may span multiple years and another for small-scope, one-year projects.



• Standardised application forms should be adopted with limited spaces and a project summary of limited length.



• Selection Committee members would benefit from more precisely defined selection criteria.



• We recommend that the regional distribution of contributions be defined in terms of minimum and maximum contribution amounts by region in order to ensure an optimal and equitable distribution across Canada.



• Economically-oriented selection criteria should be included to make the program more compatible with Industry Canada's mandate.



• Now that the program has reached a certain level of maturity, the number of community representatives from the Advisory Committee should be reduced.



• Even if the program results are modest overall, they are significant given the size of its budget. In other words, we view these results as substantial considering the contribution amounts granted by the Department. With regard to the performance of Web sites supported by the program, although the average number of visitors per month, i.e. 1,500, is a reasonable average for Francophone Web sites offering content, the program should aim for a higher average in the future.



• Improved follow-up of project results is needed and would be feasible with an expanded program budget. A standardised written questionnaire similar to the one used for this evaluation should be systematically completed by all the project leaders a few weeks after the end of the funding period. The program should set quantitative objectives (for all projects) in terms of Web content creation, the number of visitors per site, and user satisfaction levels.

Management Action Taken (February 2001)

Program management recognizes the quality of the report and has endorsed its recommendations. The report was forwarded to all members of the Francommunautés virtuelles program Advisory Committee as well as its observers for comment. The Advisory Committee found the report very informative and was generally satisfied with the results and recommendations.

The results and recommendations of the report are being taken into account in decisions on the future of this pilot program. Management is pursuing options for the renewal of the program.

¹ Not every project necessarily included a Web site or training activity. In the case of Web sites, the 500-visit figure represents an average calculated for each $30,000 portion of the overall program budget allocated to projects. According to the evaluation results, each Web site attracted an average of 1,500 visitors per month.

Adobe Acrobat Version (PDF - 258KB - 86 pages)

Note: to read the PDF version, you need Adobe Acrobat Reader on your system. If the Adobe download site is not accessible to you, you can download Acrobat Reader from an accessible page. If the accessibility of PDF is a concern, you can have the file converted to HTML or ASCII text by using one of the access services provide by Adobe.