The Contributions Program is intended to create a more effective consumer voice and thereby encourage a fair, efficient and competitive marketplace. All data sources confirm that the objectives of the Program remain relevant.

All data sources also confirm that there is a continued need for the Program. Evidence suggests that the need is expanding and there is some evidence that the Program is not keeping pace with this need.

There are few if any Federal programs that overlap with the Contributions Program. A new program by the Privacy Commissioner will require future co-ordination efforts to ensure support for privacy-related research is not duplicated. Some similar research projects might be supported by Federal departments or through Provinces and Territories but these are infrequent, specific to a research topic, and not part of a general program of support.

2. Success

Recipient organizations increase their capacity to do research and to represent the consumer interest. Many are developing expertise in particular consumer issue areas. Their credibility is also enhanced. However, few recipient organizations appear to have fully capitalized on these impacts through other funded activities. Government organizations that seek the input of consumer groups are not generally willing to pay for it. Although demand for their input on consumer issues is high, consumer groups are able to meet very little of this unfunded demand. To do so would take away from other activities of the organization.

Recipient organizations report an improvement in the human capital of their employees and co-op and summer students through activities supported by research projects. The research exposure in the supported organizations acts as a rich training environment and some employees have left for other research opportunities, many involving the consumer interest, elsewhere.

The degree of dissemination of information from Research Project Contributions varies. Most organizations identify posting a summary of the report on their web site and providing the capability to download the report from their site. Most also report holding a press conference or providing a media release. Many develop and send out reports to a list of key stakeholders (including industry) they think should get the information. A few identify making the report available to their membership. A few also mention sending the report to politicians or meeting with politicians to review the study findings.

Organizations do not record contact information for those who obtain their reports, frustrating attempts to follow-up to assess an impact. Neither the OCA nor supported organizations systematically record mentions of reports in the media or quotes in peer journals or articles. A few organizations mentioned keeping a clippings file but not compiling any information from them.

However, the limited available information on use of research reports does suggest that changes are being made to improve marketplaces through the findings from supported research projects.

Development Project Contributions have resulted in a number of examples of positive impact for supported organizations. Although most impacts are internal to the organization, given the nature of the development projects, the strength they provide to the organization is likely to have spill over effects to the consumer interest.

More generally the Contributions Program was felt to add credibility to non-profit consumer and volunteer organizations. Organizations are now better able to present research-backed arguments that are the equal of arguments by industry. As a result they are more effective lobbyists and better able to protect the interests of consumers.

3. Alternatives

The Program implemented changes recommended by the previous evaluation. This evaluation obtained the suggestions of a number of key informants related to further changes. These can be summarized as:

More and more effective use of development support.
Expansion of total support through innovative measures.
Modify project selection.
Improve dissemination of results.

These suggestions for change and the findings reported earlier support the following recommendations of this evaluation.

Recommendations

1. Find a way to increase the available funds

The Contribution Program is virtually the only source of government support for the research activities of non-profit consumer groups and voluntary organizations. The evaluation identified a strong and continued need for the Program. The need appears to be expanding and the Program appears not to be keeping pace.

Organizations are unlikely to become self-supporting, as research promoting the “common good” is not generally saleable. In such circumstances, external support is required to produce research results that are of benefit to consumers, the marketplace and society.

Demand for funds for useful research activities exceeds available contributions. The Program only supports two out of every five applications and provides one out of every three dollars of support requested.

The OCA should actively lobby for additional funds for the Contributions Program.

Some of the supported research activities benefit government organizations such as those with a mandate to consult consumers.

In situations where a government organization will benefit from the research activities supported through the Contributions Program, the government organization should be encouraged to pay for or share in the costs of the research. The OCA should facilitate access to support through the benefiting government agency and thereby “free-up” Program contributions for other worthwhile activities.

2. Improve Dissemination of Information and Institute Tracking of the Impacts of Information

Information is of no value unless those who can benefit from the information use it. The degree of communication of project results by recipient organizations ranges from a little to a lot. No organizations follow-up to assess the impact of project results. Organizations could do more to disseminate information and to help assess the value of that information.

Require all recipient organizations to produce a short summary of the project report and to post the summary, at a minimum, on the web site of the organization or on the OCA web site.

Require all recipient organizations to capture and to provide to OCA contact information for those who obtain the full report on the project (by download or in hard copy) or who participate in enhanced dissemination activities (See below). This information should be used for performance measurement and evaluation purposes

Some project results are of particular value and warrant increased dissemination efforts. This is difficult to determine at the time of the initial project application but becomes more apparent as research results become known.

When circumstances warrant special efforts to disseminate results, organizations should be eligible for a follow-on contribution agreement to provide enhanced dissemination of project results. Organizations should submit an application covering the enhanced dissemination and, after approval, should conduct and document activities and provide a full reporting of preliminary impacts and contact information for those who receive the enhanced information dissemination.

Impacts of project results may occur for a diverse group who receive the information and may take some time to occur. However, obtaining the initial intentions of those who receive the information would both identify possible impact and facilitate future tracking to identify actual impacts.

Use contact information on those who receive project reports and enhanced information dissemination to conduct follow-ups to identify intentions and actual impacts.

These changes should be reflected in the Results Based Accountability Framework.

Amend Results Based Accountability Framework to include accounts of projects disseminated, follow-up activities to information recipients, and quantification of intended and actual impacts through the information. Report on performance measures annually.

3. Institute General-Application Development Projects and Communicate their Findings Broadly

There are a number of topics of general applicability to all or most non-profit consumer and volunteer organizations. Topics might include governance structures, maintaining a membership base, how to raise funds, how to communicate research results or how to maximize the impact from a press conference. A development project involving a single organization would benefit that organization. However a project involving a number of organizations or one with a single organization but with results communicated to all organizations would provide even more benefit.

Encourage development projects that have application to multiple organizations. Share results broadly with the intent of benefiting many organizations.

One such “development”activity” could be a round-table discussion of topics of mutual interest to organizations. Topics might be lessons learned from a set of similar development projects.

Bring organizations together for a one-day session every second year to exchange ideas of common interest and share best practices toward strengthening the consumer movement.

4. Increase Transparency of Application Approval Process

The approval of applications is a highly competitive process. Funds are limited. Applications for research funding far exceed the contributions that are available. Stakes are high for organizations that rely on approval of their applications for their continuation. Although much effort is expended to make the process fair much more could be done to improve on the perception of fairness. Greater transparency is needed.

Conduct a telephone conference annually, open to all potential applicants, to explain the application process, identify changes since the previous year, and address any questions. Use the opportunity to clear up misunderstanding of eligible expenditures identified during the evaluation.

Identify the eligible points assigned to each question of the proposal form.

Provide feedback on applications by providing the score per question and overall points calculation. Identify the number of applications received and awarded for the topic area of the application.

5. Encourage Greater Self-Reliance through Development Projects

A number of consumer organizations are fragile. Some may be dependent on Research Project Contributions for their survival. Reductions in the amount of support available through Research Projects would adversely affect such organizations and the consumer movement. More needs to be done to encourage the self-sustainability of organizations and to wean organizations from dependence on Research Projects.

A high proportion of any new funding to the Contributions Program should be allocated to Development Projects and be used to increase organizational self-reliance and thereby reduce dependence on Research Project support.

Other Items of Note

Other suggestions for change are worthy of attention beyond those recommended above:

Improper use of research findings is a potential danger in any research. To help prevent such misuse, research reports and any release of findings from them should contain a clear statement of the limitations of the study methodology and guide the reader toward proper interpretation of the results.
In the future the program might consider requiring organizations to obtain liability insurance coverage. If required, premiums should be an allowable expense.
Follow-on activities to an original project may be highly cost-effective as they can build on the previous research methodology and base to confirm or help modify previous findings. But line departments, that benefit directly from the research, appear unwilling to pay claiming consumer research is not within their mandate. A perception also exists that OCA is unlikely to fund a similar study to follow-up on earlier work. As a result, potentially valuable research that could capitalize on previous research supported under the Program may not proceed through the lack of a willing funding source. This should be reviewed.
The fixed timing of the application process and research schedules which conform to fiscal year funding realities, limit the ability to be responsive to an emerging issue. Potentially OCA could consider a second, more limited, call for applications mid-fiscal year. These would have shorter time frames and focus on emerging issues only. Alternatively moving to two application windows in a fiscal year for all projects might help reduce the lags created by the current application cycle.
There may be research opportunities spanning a number of years that are not being addressed through the current annual application process. The Program might consider commitments to multi-year funding (conditional on annual applications and annual audits).

Final Report (PDF - 339KB - 83 pages)

Management Response (PDF - 19KB - 6 pages)

Note: to read the PDF version, you need Adobe Acrobat Reader on your system. If the Adobe download site is not accessible to you, you can download Acrobat Reader from an accessible page. If the accessibility of PDF is a concern, you can have the file converted to HTML or ASCII text by using one of the access services provide by Adobe.