Note: This site has been designed to be best viewed in a browser that supports web standards, the content is however still accessible to any browser. Please review our Browser Tips.

We have provided the following access keys:
Skip to search
Skip to main navigation
Skip to secondary navigation
Skip to discipline navigation
Skip to content

Peer Assessment at the Canada Council for the Arts: How the Council Makes its Grant Decisions

Introduction

The Canada Council for the Arts is Canada's national arts funding agency. It was created by an Act of Parliament in 1957 to support the study, enjoyment, and production of works in the arts. The Council's core business is providing grants to professional Canadian artists and arts organizations throughout Canada in a wide variety of artistic disciplines.

The Council's work is guided by two principles. The arm's length principle defines its relationship with government, and the peer assessment principle defines its relationship with the arts community. Together these principles are the fundamental tenets governing the Council's operations and decision-making processes.

Following a description of the two principles, this document provides a detailed account of the Council's peer assessment system - the means the Council uses to arrive at its most important decisions, namely which artists, artistic projects and arts organizations will receive grants.

The arm's length principle

The Council is at "arm's length" from the government, which means that, within its mandate from Parliament, it has full authority to establish its priorities, policies, and funding programs and make grant decisions.

When the Government of Canada began supporting the arts and culture through institutions like the Canadian Broadcasting Corporation and the Canada Council, it realized that it had to protect freedom of thought and expression or risk creating an official state culture. It was clear that if publicly financed agencies were to encourage cultural and intellectual life, they required freedom to support experimentation and creativity through work that might be critical and challenging.

The Royal Commission on National Development in the Arts, Letters and Sciences, which proposed the creation of the Canada Council in 1951, pointed out "the dangers inherent in any system of subvention by the central government to the arts and letters". To avoid those dangers, the Commission recommended that the Council be established as an independent body accountable to Parliament, not a department of government reporting to a minister of the Crown.

The essence of the arm's length relationship was expressed by then Prime Minister Louis St-Laurent when he recommended to Parliament that it create the Canada Council for the Arts. "Government," he said, "should support the cultural development of the nation but not control it." All subsequent federal governments have respected this principle.

Though the arm's length relationship was established to protect the arts from the state, it has two further consequences for the Council's work. First, it shields the Council's artistic decisions from pressures brought by other sources, whether they are colleagues and friends of applicants, partisan or special interest organizations in or outside the arts, or community groups lobbying for or denouncing a particular point of view.

A second consequence is that the Council is entrusted with responsibility for its policies, programs and disbursement of funds. It is the Council, and not the government, which is accountable for its actions. Each year the Council reports on its activities to Parliament through the Minister of Canadian Heritage, and its finances are audited by the Auditor General of Canada. Parliament also determines the amount of money the Council receives; the federal Cabinet appoints Board members for fixed terms; and Parliamentary committees may call the Council to appear to answer questions at any time. As an independent agency that dispenses public funds, the Council has a high degree of responsibility to account fully and openly for its operations.

The peer assessment principle

"Peer assessment" at the Canada Council for the Arts means the use of independent artists and other professionals working in the field to assess grant applications, advise on priorities, and make recommendations to the Council on the awarding of grants. Peers, in the Council's definition, are people who, by virtue of their experience, knowledge and open-mindedness, are capable of making a fair and informed assessment of the comparative merits of grant applications. Through peer assessment, the Council involves the arts community directly in its operations.

Of all the decisions the Council is empowered to make, its decisions about which artists, arts organizations and artistic projects will receive grants are the most sensitive, the most visible and the most likely to provoke criticism. Every year the Council receives in excess of 16,000 grant applications. Some 6,000 grants are awarded, many for less than the amount requested.

The Council welcomes spirited discussion and disagreement as a natural outcome of its intensely competitive work. At its best, the thrust and parry of democratic debate about arts grants confirms the power of the arts - their unique ability to generate strong passions and equally strong discord. The Council must therefore ensure that its grants to artists and arts organizations are dispensed with integrity, transparency and fairness and that its policies are clear and consistent.

In the Council's view, it is essential that:

  • applicants for Council grants have confidence that they have been assessed by people with the knowledge and expertise to make sound qualitative judgements in their field of the arts - even if, in failing to get a grant, they are unhappy with the outcome;

  • artists and other arts professionals who serve as members of peer assessment committees are able to attest to the credibility, honesty and fairness of the process;

  • members of the public and Parliament know that the Council's assessment system serves the public interest because it is the most equitable means available for evaluating artistic merit and is governed by policies and procedures that are clearly and consistently applied - even if, as is inevitable, they don't personally like every artistic activity the Council supports.

Peer Assessment: What does it mean? Why does the Council use it?

In its 1995 strategic plan, the Canada Council for the Arts reaffirmed its three basic operating principles:

  • its arm's length relationship with government, which allows the Council to make artistic decisions free from external pressures;

  • peer assessment as the basis for its grant decisions; and

  • comparative artistic merit as the most important criterion in its funding programs.

The cornerstone of the Council's operations is peer assessment - the use of artists and independent professionals working in the arts to evaluate the comparative merits of applications in a program competition, establish their priority order and recommend grants. Each year about 500 people serve the Council as assessors on some 120 committees. The committees in turn make use of input from another 200 independent peer assessors who have seen and evaluated specific performances or works.

In almost all its programs, the Council brings together a new assessment committee of three to seven members drawn from the applicants' discipline (or disciplines) for each competition. Their task is to provide expert advice to the Council, assessing and prioritizing applications based on the published objectives and assessment criteria of the program, and recommending which should receive support. Peer assessors are unaffiliated with the Canada Council - they are not Board members or staff. Peer review thus removes the selection of grant recipients from the control of Council personnel.

The Council's commitment to peer assessment is based on three convictions:

  • As a steward of public funds, the Council wants to make the wisest possible use of its resources. It seeks to recognize the best applicants: the most promising and original artists, the most significant artistic projects, the most capable and meritorious arts organizations. Determining who is best, before history has sifted and selected those artists of talent who will endure, is a supremely difficult task. The poet Philip Larkin once said, "Nothing . . . is harder than to form an estimate even remotely accurate of our own contemporary artists." In this role, no person - and no system - is infallible. The Council believes that the best possible advice in identifying outstanding ability in the arts comes from people who have acquired knowledge and insight through extensive professional experience in the field. Thus peer assessment helps the Council make the wisest use of the financial resources it receives from the public of Canada.

  • The Council is also a steward of freedom of thought and expression. As a matter of long-standing policy, the Council, while requiring that grant recipients report on and account for their use of grants, respects artistic freedom and does not interfere in the internal policies of organizations, projects undertaken by individual artists, or the content of works created or presented with Council support. Because a work of art or an artistic event can arouse intense feelings of admiration or disdain, the arts are highly vulnerable to shifting social currents and the political pressures they create. Peer assessment ensures that artistic quality is the major consideration in Council grant decisions and thus protects diversity of opinion and artistic freedom.

  • Peer assessment is based on collective decision-making. No applicant to the Council is judged by a single person only, and funding decisions are made by the consensus of committee members. No two assessment committees are composed of the same people. The diversity of points of view represented on the committees (actively promoted by Council policies and practices, as described below) protects artists against the dominance of any single artistic ideology and helps ensure that the process and its outcome are democratic.

In Canada and around the world, peer review is widely used by arts funding bodies, but it is not confined to the arts. It is the most common method for assessing qualitative achievement in creative and intellectual occupations. Federal granting councils in the sciences and humanities, medical and academic journals, accreditation organizations in the professions, and prizes for intellectual and creative contributions such as the Nobel and Pulitzer all base their evaluation of merit on peer assessment.

How does the Council select its peer assessors?

In selecting peer assessors, the Council looks for individuals who, by virtue of having professional experience and knowledge directly relevant to the program criteria and applicant group, are credible to both the applicants and the Council.

The Council's objective is to maximize the number of qualified people who serve on assessment committees and ensure that the committees represent a wide diversity of specializations, artistic practices, cultural backgrounds and regional perspectives. Other key considerations are breadth of vision, open-mindedness and generosity of spirit, the ability to work collaboratively and the willingness to express opinions while respecting and listening to the opinions of others.

Committees are selected with a particular concern to take equitable account of the following factors:

  1. Diversity of professional specialization (representation of people who perform a variety of different professional roles in the arts, such as creators, interpreters, administrators, directors, publishers, arts critics, arts educators, etc.);

  2. Diversity of artistic practice (representation of different artistic styles and philosophies);

  3. Language (representation of the two official languages and minority official-language communities);

  4. Region (representation from all regions of Canada);

  5. Gender (representation of men and women);

  6. Diversity of age (representation of different artistic "generations," from younger professionals to their seniors);

  7. Cultural diversity (representation of artists and arts professionals from the diverse ethnic, racial and cultural communities of Canada);

  8. Aboriginal representation (representation of Aboriginal artists and arts professionals from across Canada, including First Nations, Inuit and Métis communities); and

  9. The ability to provide aesthetic context to the discussions.

The Council recognizes that no single committee of three to seven people can represent all these characteristics, but over time it requires that its committees achieve these balances. To monitor this, the Council reviews the peer committees in each of its arts sections annually, and an annual report on the peer assessment system is made to the Council's board.

In recent years, the Council has emphasized the importance of increasing the arts community's participation in its work. Its strategic plan identified greater participation by culturally diverse artists, Aboriginal artists and artists from all regions of Canada as a major goal. As a result, the Council has enlarged its pool of potential peers, which is maintained in an electronic database, to include more experts conversant with the standards of excellent arts practice in a wide variety of communities.

In addition, the database has evolved to reflect the growth and diversification of the Canadian arts community as a whole. The range of talent in the arts has greatly expanded in recent years, not only in traditional disciplines but also in newer forms such as multi-media, interdisciplinary work and technology-based art, again leading to an expansion of the pool of assessors.

The inclusion of "new" peers, that is, people who have never before taken part in a Council assessment committee, is an especially important responsibility of the program officers. To provide more opportunities for new peers to participate, the Council has established a buffer period for service on peer committees. As a general rule, an individual can be an assessment committee member only once in 24 months. Exceptions are made for artists who have professional experience in more than one discipline. An artist may serve on a committee after receiving a grant.

Selecting assessment committee members is a major responsibility of the Council's program officers. For each competition, the officer in charge of the program chooses peers from the artistic discipline(s) most closely related to the program and its applicants, giving particular consideration to representation of the nine factors listed above. In addition to having expertise and knowledge appropriate to the program, they must be capable of providing a fair and just evaluation of a wide range of artistic styles and practices and able to work effectively in a group.

In selecting committee members, the officers consult the database. It allows them to keep track of individuals who have recently served on a committee or are in receipt of a grant. The officers are also responsible for continually researching and adding names of qualified people to the database, to build an ever-more inclusive list of assessors. The Council's Board members, professional staff and applicant organizations regularly contribute qualified names to the database.

Occasionally the Council calls on an individual assessor from the field to provide specialized expertise that supplements the work of an assessment committee, for example, to provide a written report on a performance or a script. Selection of individual assessors follows the same goals and procedures outlined above for committees.

How does the Council ensure that its peer assessment system is fair and effective?

To function effectively, any assessment system needs thoughtful and consistent policies and procedures. This is especially true for evaluations of quality such as those used in the arts, which cannot be reduced to formulas.

The Council has put in place five important tools to assist the effective functioning of peer assessment:

  • clearly articulated Council priorities;

  • comprehensible program objectives and assessment criteria based on those objectives;

  • transparent and consistent procedures for managing conflict of interest;

  • clear guidelines and practices governing confidentiality of information; and

  • ongoing training of new staff to ensure their familiarity with the policies and procedures which constitute the Council's peer assessment policy.

Information on these subjects is provided in the Guidelines for Peer Assessment Committee Members and the program documents that are sent to members in advance of the committee meeting. The peers therefore come to the meeting knowing what priorities the Council has established; what the program objectives are and what criteria they will use in evaluating applications; how they must conduct themselves if they have a potential or actual conflict of interest; and how they are obliged to treat information that is confidential.

Council Priorities

The priorities adopted by the Council in its strategic plan are:

  • The primary criterion for Council grants is professional artistic excellence or comparative artistic merit.

  • The Council's support to artists and arts organizations is concentrated on the creation, production, and distribution and dissemination of works of art, particularly Canadian works of art.

  • Specific priorities for support are Aboriginal artists and arts organizations, artists and arts organizations from diverse regions, culturally diverse communities, works directed to young audiences, dissemination within Canada, international promotion of Canadian art and artists, festivals, interdisciplinary art and new technologies.

In addition, the strategic plan confirmed the Council's continued commitment to providing grants to both individual artists and arts organizations and, within programs for organizations, to maintaining a balance of support between established organizations and new artistic developments.

Program Objectives and Assessment Criteria

Each Council program has specific objectives and assessment criteria based on the objectives. It is these criteria on which the peer assessment committee bases its evaluations and grant recommendations.

For grants to individual artists for their personal creative work, there are two fundamental criteria:

  • the comparative artistic merit of the applicant's work, and

  • the merit of the project the applicant proposes to undertake with the aid of Council funds.

For operating and multi-year grants to arts organizations and substantial project grants, there are three major assessment criteria, which are weighted, that is, their relative importance in the assessment is specified:

  • comparative artistic merit and professional excellence are the most important criteria, accounting for 60-65% of the total evaluation, depending on the program;

  • contribution to the community/public connections, including dissemination, audience development, outreach, educational activities with the public, etc., represents 15-20% of the evaluation; and

  • suitability and quality of governance and administration, including managerial and financial stability, account for 15-20%.

Managing Conflict of Interest

Since the purpose of the peer assessment system is to select qualified and knowledgeable assessors from the field, assessors may have contacts with one or more of the applicants. Where an actual or potential conflict of interest exists, it must be disclosed and the assessor must abstain from any and all participation related to it, and leave the meeting for the duration of the discussion.

Prior to the committee meeting, in the written Guidelines, the committee members receive the "Conflict of Interest Disclosure Form" (as well as examples of the kinds of conflict of interest that may occur). All members must complete this form and submit it on the first day of the committee meeting. Committee members declaring a conflict of interest must indicate the file(s) in question in the sign-off sheet.

The form specifies that conflict of interest exists or may exist if members are asked to assess applications:

  • from a full-time employer, a client or an organization where they are a board member;

  • where they have a direct financial interest in the success or failure of an application;

  • where the applicant is their spouse/partner or immediate family member;

  • where their spouse/partner or immediate family member is a senior staff member, contractor or board member with the applicant organization; or

  • where they judge that they are unable, for any other reason, to assess an application objectively.

Confidentiality of Information

Committee members are required to treat both the contents of applications that they review and the deliberations of the committee as confidential. They must not divulge information in applicant files outside the meeting. Who applied in the competition, who was recommended for a grant, and any comments made by individual committee members are also confidential matters.

As a matter of policy, the Council publishes, in a timely fashion, lists of grant recipients in each program and never reveals the identity of unsuccessful applicants.

How does the Council prepare for a peer assessment committee meeting?

When grant applications arrive at the Canada Council for the Arts, the program officer reviews them to ensure that they are complete. The officer is responsible for determining if the applicant and project meet the eligibility criteria for the program and sends only eligible applications on to the committee. Responsibility for evaluating eligibility, as well as priorities, program policies, objectives and criteria rests with the Council. The assessment committee is responsible for evaluating applications within the program objectives, policies and criteria. The officer plays no role in judging artistic quality, which is the committee's work.

Once eligibility has been determined, the officer selects the members of the committee, taking into account the desired characteristics and the nine factors described earlier, and submits the list for review by the head of section. The eligible applications are photocopied, compiled and sent in book form to the members for review in advance of the meeting. Prior to the meeting, members also receive the Guidelines for Peer Assessment Committee Members, which includes the conflict of interest form and information on the Council's priorities, and the objectives and assessment criteria for the program. In programs for arts organizations, the documentation may include profiles of the organizations that they themselves have prepared and factual background information on the program provided by the program officer.

How is the peer assessment committee meeting conducted?

When the committee convenes to discuss and determine the relative merits of the applications, the discussions are chaired by the program officer or the head of section. Committee members submit their completed conflict of interest form at the beginning of the meeting. The officer begins the meeting by formally briefing the committee on its responsibilities and how the meeting will be conducted, reviewing the Council's priorities, the strategic development outlook for the program and the program objectives and criteria.

Committee members proceed to evaluate each application against the assessment criteria and Council priorities. They also view, read or listen to support material from the applicants (e.g., slides in the case of visual artists, audio tapes in the case of composers, etc.). Where relevant, they consider written assessments prepared by independent assessors. (The following information is also provided: how assessors are nominated, how many assessments are being forwarded to the committee, and how assessments are to be considered. Assessment criteria are also available to applicants. Where staff or external assessors prepare written assessments, the basis on which these are prepared is indicated). Each application is discussed in turn, and by the conclusion of the meeting the committee has collectively ranked the applications in priority order for funding and made grant recommendations.

Depending on the program and the number of applications, committee meetings last for one to six days. Members are paid an honorarium of $250 for each day of the meeting plus reasonable expenses for travel, accommodation and meals.

During the meeting, the assessors are encouraged to work in either official language. As necessary, the officer assists in translating members' comments for other members. For many bilingual assessment committees (that is, committees in all programs except those in Writing and Publishing and Theatre, which are language-specific), a summary of application information is made available in both English and French.

The officer, as chair, must establish conditions for a serious, fair and open-minded evaluation of every application. He or she must manage the assessment process, making certain that each application is assessed against the published criteria of the program (and, where relevant, that the weighting of the criteria is respected in the assessment) and that all applicants are treated equitably. The officer must also manage conflict of interest, overseeing compliance with the guidelines. The officer encourages participation by each assessor and helps enable the group to reach final agreement on their rankings and grant recommendations. The officer's role is not to comment on artistic merit, but he or she may provide factual or background information, including information on the national and regional context of the program, if requested by the committee.

Finally, the officer records the decisions and grants recommended by the committee, ascertaining with the committee that these records accurately reflect their views and ensuring that members attest to their accuracy by signing-off on the records.

At the end of the assessment committee meeting, to protect confidentiality, members turn in all their committee books (the photocopies of the applications) for shredding.

What happens after the assessment committee meeting?

Following assessment committee meetings, the program officer prepares the necessary internal documentation to obtain approval of grants, based on the assessment committee's recommendations. Authority to approve grants belongs to the Board of the Canada Council, which, for purposes of efficiency, has delegated this responsibility to the Council's Director for grants of $60,000 and less. The decisions by the Board and the Director are based on the recommendations of the peer assessment committees. Approval may be withheld in cases of procedural impropriety or failure to observe established Council policies.

Following the approval process, the officer prepares letters and associated documentation for the applicants to inform them about the results of the competition.

In addition, peers are asked for their comments on the competition and how further improvements can be brought to the process. They are free to share any criticisms of any aspect of the peer assessment process in their responses. If they so choose, peers can address their concerns more formally to the senior management of the Canada Council in written form. In such cases, the senior management will require a period of 45 days to provide a detailed response, in consultation with the Chairman and the Board.

In communicating with arts organizations, the Council provides feedback that reflects the consensus the committee reached, based on the program criteria, in evaluating the application. This practice reflects the Council's belief that feedback should be made available when it can play a positive role in helping the organization develop or improve its work and when the Council has the resources to handle the task. Written feedback is not systematically provided to individual artists because the high volume of applications makes detailed personal critiques impossible.

Any applicant who wants further information on the competition is encouraged to contact the program officer for discussion. The officer (rather than the assessment committee) is responsible for following up with applicants and with others who wish to discuss the competition. If an applicant is concerned that there was a procedural impropriety or that Council staff failed to observe established Council policies, he or she can file a formal complaint for review by the senior management of the Council and by the Chairman on behalf of the Board. Such a process requires 45 days to provide a detailed response to the applicant.

In most Council programs, an applicant who was unsuccessful in one competition can reapply in the next. At that time, the applicant will be assessed by a new and different assessment committee in the context of a new and different group of "competitors".

The Council is subject to two related federal laws, the Access to Information Act and the Privacy Act, which have an impact on the peer assessment system:

  • Individuals and organizations have a right to request information about themselves which is contained in the Council's records. Examples include documents in the applicant's file, written artistic assessments of the applicant's work, and computerized data on the applicant.

  • If a document which an individual or organization requests contains identifiable references to other parties - for example, other artists or arts organizations or the names of independent external assessors - such references are removed before the document is provided, to protect their privacy.

Definition of an ideal peer assessor

Though the definition is subject to continuing discussion, an ideal peer assessor is considered to be an experienced artist or arts professional whose background demonstrates a profound commitment to an artistic specialty and to the development of knowledge about and opportunity for the art form.

She or he is committed to an enriched future for the artistic discipline and brings to the peer deliberations the qualities of a generous spirit, articulateness, exceptional listening skills, and a willingness to embrace change and complexity through hearing and working with different points of view.

An ideal peer assessor has extensive knowledge of the diverse artists, arts professionals and arts organizations working in her or his region and is well respected among peers in the community. He or she has an understanding of the diversity of cultural practices, critical debates, themes and issues in her or his field and the pluralistic arts community in general.

While holding strong opinions on aesthetics that he or she is prepared to defend, without causing conflict, the ideal peer assessor is respectful of the adjudication process, is open to input from fellow peers and allows for consensus even where there is some disagreement. She or he reads and analyses in a considered manner all guidelines, program criteria and materials received prior to the meeting, declares any conflicts of interest (perceived or otherwise) and maintains confidentiality at all times.

The ideal peer assessor understands how to apply program criteria within the adjudication process and will express openly any concerns about the integrity of the proceedings.

The ideal peer assessor is genuinely and deeply supportive of all artists, arts professionals and arts organizations, whatever their level of experience. He or she has the ability to understand and appreciate the contexts in which the artist-applicants work and produce art, and the role played by arts organizations and arts professionals in the development and dissemination of that art.

How can I find out about the results from the peer assessment committees?

The results of grant competitions are available on the Council's web site, as are the names of peer assessment committee members.

Canada Council for the Arts
350 Albert Street, P.O. Box 1047
Ottawa, Ontario
K1P 5V8

Telephone: (613) 566.4414 or 1.800.263.5588

 

Revised by the Board of the Canada Council for the Arts

November 2002