• Home Page
• Publications
• 2007
• 2006
• 2005
• 2004
• 2003
• 2002
• 2001
• 2000
• 1999
• 1998
• 1997
• 1996
• 1995
• Triennial Plan
• Policies
• Contact Us

Aboriginal Justice Strategy, Evaluation Framework

EXECUTIVE SUMMARY

The current Aboriginal Justice Strategy has been designed to help "establish policies and programs that will be the foundation of long term administration of justice improvements within the framework of the Canadian law for Aboriginal people." The AJS is about working within the existing Canadian justice system to build mainstream/Aboriginal co-operative partnerships which will support development of better justice system programs and policies to meet Aboriginal justice needs.

The AJS will explore innovative alternative policies and programs within the existing Canadian justice system, consistent with the implementation of the justice elements of the inherent right policy of self-government. The AJS has been developed in co-operation with other federal departments (the Solicitor General, Privy Council Office and Indian and Northern Affairs), provincial governments, and Aboriginal people, and this work will continue.

Cabinet approved the Aboriginal Justice Strategy in March 1996 with a requirement that "the Minister of Justice report to Cabinet with an evaluation of the impact and effectiveness of this proposal within five years". In December 1996, Treasury Board Ministers conditionally approved future years' funding of the Aboriginal Justice Strategy "subject to provision of the following to the Treasury Board Secretariat (TBS) before seeking the 1997-98 Supplementary Estimate: 1) an evaluation framework acceptable to the Department of Justice, TBS and partner departments". The TB Record of Decision further states: "Ministers also noted their expectation that a mid-term review of the Strategy be provided in October, 1998, and that a final evaluation be provided in November, 2000."

The Evaluation Framework lays out the long term evaluation plan for the Aboriginal Justice Strategy and identifies issues and data collection requirements for the evaluation.

The Strategy Components

There are three primary components of the Aboriginal Justice Strategy (Policy, Programs and the Aboriginal Justice Learning Network), of which the funding of community-based Aboriginal justice programs is delineated into four sub-categories (Bridge funded, on reserve, off reserve and province-wide agreements).

The overall objectives of the Aboriginal Justice Strategy are to respond to the aspiration of Aboriginal people to assume greater responsibility for the administration of justice in their communities and to help reduce the rates of incarceration and crime among Aboriginal people.

Program activity outputs and outcomes for each strategy component can be classified in terms of the audience or stakeholder involved and whether the outcome is expected to occur at an intermediate time frame (within a year) or over a longer time frame (over the five years of the program). Three basic stakeholder audiences have been defined: the Aboriginal community, the provincial/territorial government and the federal government[1]. Figure 1 outlines the key program outcomes and how each partner in the Strategy is expected to contribute to program outcomes.

The evaluation of the Aboriginal Justice Strategy will incorporate a new approach to accountability and results in socio-legal community development, called "Empowerment evaluation". This is basically an approach that fosters self-assessment, collective knowledge production, and co-operative action, in which the stakeholders in a social and economic development initiative participate substantively in the identification of the research/evaluation issues, the design of the study, the collection and analysis of data, and the action taken as a result of the study's findings. By participating in this process, the stakeholders also build their own capacity and skills to undertake research and evaluation in other areas and to promote other forms of participatory development.

As one of the goals of the Aboriginal Justice Strategy is to help Aboriginal communities meet their justice needs and their aspirations to assume greater responsibility for the administration of justice, this approach is the preferred approach of the Aboriginal Justice Directorate in the Department of Justice, Treasury Board Secretariat and the Department of Indian and Northern Affairs. When the Strategy is complete, the Aboriginal communities need to be prepared to undertake their own justice initiatives and be able to assess and modify the programs to best serve the needs of their communities. This empowering approach to evaluation is seen as the optimum vehicle to prepare Aboriginal communities to assume the inherent rights of Aboriginal people to self-government, within the justice context, with application to participatory community development in general.

Therefore, in order to properly evaluate the Aboriginal Justice Strategy, an empowerment evaluation approach is being proposed. The traditional evaluation approach in the federal government has typically involved the government establishing the plans for, and conducting the evaluation. In this case, it would have involved the imposition of very specific information reporting requirements on each of the funded communities to ensure that a national level evaluation could be carried out by the Department of Justice. However, Aboriginal communities have in the past resisted this top down approach to evaluating their programs. In order to meet the needs of each community as well as the accountability requirements of government, each community will be able to plan and conduct their own evaluation. However, a core set of performance measures reflecting the federal government's key objectives for the program will have to form part of each community's evaluation reporting. At the national level, this information will be rolled up and form part of the national evaluation required by Treasury Board for the program. This basic approach would ensure that communities develop the management expertise, culture and buy-in to evaluate the success of their own programs as well as ensuring that the federal government is accountable for evaluating the results of its expenditure programs as per TB policies.

Empowerment evaluation depends on justice personnel within the Aboriginal communities (including the members of the community evaluation committee) receiving culturally appropriate training, technical assistance, and on-going consultative support on how to implement a self-evaluation plan. Therefore, on-going consultation and support by an experienced evaluation consultant or trainer needs to be made available to the community.

Individual projects at the community level will have goals and objectives that reflect the needs of the people in the community, the problems of individuals (whose behaviour brings them in conflict with the community and the justice system) and the needs of the victims who have been injured.

Both the community and the federal and provincial/territorial funders have a vested interest in the gathering of information relevant to the measurement of the progress of the community project in relation to these universal goals, to the success of the project relative to its specific goals and objectives, and to learning "new lessons" about what works, or doesn't work, for each community and how better to achieve success. This learning at the community level is important to all stakeholders, the Aboriginal communities, the federal government and the provincial/territorial governments, in order to improve the responsiveness of the Canadian justice system in addressing Aboriginal issues and to promote Aboriginal governance.

Collaboration is called for among the federal government, the provincial/territorial government and the Aboriginal communities to facilitate the gathering of information to address these common goals and evaluation questions. At the federal level, an Interdepartmental Committee on Aboriginal Justice has been created, which includes members from the Department of Indian and Northern Affairs, Ministry of the Solicitor General, Privy Council Office (Tripartite self-government Negotiations and the Federal Interlocutor for Métis and Non-Status Indians), the Royal Canadian Mounted Police (Contract and Aboriginal Policing Services), Health Canada, Fisheries and Oceans, CMHC and Treasury Board Secretariat (Evaluation).

Throughout the five year strategy, for all aspects of the evaluation, information is to be maintained by the Aboriginal communities and shared with provincial/territorial and federal offices. This information is to be collected through interviews, focus groups or questionnaire surveys of key stakeholders within the Aboriginal community, as well as the individual projects.

AJS IMPLEMENTATION AND IMPACT AT THE ABORIGINAL COMMUNITY LEVEL

Evaluation questions pertaining to AJS management, administration and planning activities will examine the extent to which there is committed participation within the Aboriginal communities and the overall success in implementing the specific community based programs. Evaluation questions pertaining to provincial/territorial support and resources in the Aboriginal communities in their jurisdiction will review the sufficiency of training and support provided by provinces and territories.

Evaluation questions pertaining to activities of the AJ Learning Network among the Aboriginal community within each province or territory will examine the extent to which knowledge and capacity is developed within and available to Aboriginal communities; and whether consultation and collaborative partnership networks have been built which facilitate communication across the Aboriginal and mainstream justice personnel in the region.

Interviews, focus groups or questionnaire surveys will be conducted at the program mid-term and in year five with the following Aboriginal stakeholders:

• members of the Aboriginal Justice Learning Network
• key stakeholders in the administration of the regional Aboriginal justice projects
• individual program directors and senior staff
• chiefs and key community members from communities that are operating local programs
• the AJS regional co-ordinator
• senior staff in the provincial/territorial Ministry of Justice
• senior staff in other departments and with the following federal stakeholders:
• senior staff with the Aboriginal Directorate
• Directorate policy staff
• Directorate program officers
• senior staff with the Justice Department Evaluation Division
• senior staff within the Aboriginal Corrections department of the Solicitor General
• senior staff within DIAND concerned with self-government negotiations
• federal self-government negotiators

IMPACT OF AJS AT THE PROVINCIAL/TERRITORIAL LEVEL

Assessment of AJS Implementation at the Provincial/territorial Level

Evaluation questions pertaining to AJS management, administration and planning activities will include an examination of the participation and implementation of the provincial and territorial role in the program.

Activities of the AJ Learning Network at the provincial/territorial level will also be examined particularly as they relate to AJLN knowledge and capacity acquisition and collaborative Aboriginal-mainstream partnerships.

Assessment of Intermediate Outcomes at the Provincial/Territorial Level

Intermediate outcomes evaluation questions are as follows:

1. Has the province or territory developed sufficient commitment, knowledge and capacity to administer and support the AJS operational plan in their region?
2. Have new collaborative and permanent partnerships developed between mainstream and Aboriginal justice personnel within the province or territory?
3. Are Aboriginal women in the province or territory significantly involved in the planning and administration of justice programs in the region?

IMPLEMENTATION AT THE FEDERAL LEVEL

Assessment of AJS Management, Administration and Implementation

Evaluation of AJS management, administration and planning activities will be conducted.

Much of the information should be maintained by the Aboriginal Justice Directorate personnel, and submitted on a regular basis to the Evaluation Division. Other information will be obtained through interviews or questionnaire surveys of Directorate program officers and other key Justice and federal staff.

Intermediate Outcome Questions at the Federal Level

Intermediate outcomes evaluation questions are as follows:

1. Are the resources allocated to the Aboriginal Justice Strategy managed effectively and efficiently as dictated by the objectives and activities in the 1996 operational plan?
2. Has the Directorate acquired and utilised the knowledge and capacity to satisfactorily support the goals and activities of the Strategy?
3. Is federal policy on Aboriginal justice matters based on AJS lessons learned and supportive of the Aboriginal inherent right policy of self-government?
4. Have new collaborative and permanent partnerships developed between mainstream and Aboriginal justice personnel?

Required information will be obtained from Directorate personnel and through interviews or questionnaire surveys of Directorate program officers and other key Justice and federal staff.

ASSESSMENT OF THE OVERALL AJS LONG TERM OUTCOMES

It is a goal of the Aboriginal Justice Strategy that the justice projects implemented at the community level will have an impact in terms of providing alternatives to incarceration for offenders, and will reduce crime and recidivism for individuals referred to the project. A reduction in reported crime rates alone in a community may not be a completely reliable measure of project impact. Rather, recidivism and crime rates will be measured for those persons referred to the project and multiple lines of evidence will be used to validate information gathered through standard crime statistics. It is also a goal that the community justice projects will contribute to the community perception of safety and will leave community members with a sense of increased fairness in the administration of justice to Aboriginal peoples.

Information that would be obtained by the community as part of a collaborative evaluation effort involving the province and the federal government, would include:

• Number of criminal cases in the community, each year, (and for the five years prior to the implementation of the AJS) by type of offence and other circumstances pertinent to the diversion decisions, the diversion decision on each case, and final outcome disposition as a referral to the project or referral to mainstream justice system.
• For each case referred and diverted through the AJS program, nature and date of the offence(s), description of the case, the circumstances, the number and type of victims, the age, sex and number of priors of the offender, intervention actions taken, community members involved in the intervention process, outcome of the intervention, and follow-up status (regarding re-offending or problem behaviours) each year for five years following.
• Number of persons from the community who have been incarcerated (by length of time in jail or penitentiaries) in each of the project years versus each of the previous five years.

Other information will be obtained through interviews, focus groups or questionnaire surveys of community members and other justice personnel in the community.

IMPLEMENTATION OF THE EVALUATION FRAMEWORK

There will be a start-up phase: (January 1997 - December 1997), which will be directed toward steps to ensure:

• inclusion of evaluation and performance measurement provisions in project agreements,
• development of self-evaluation guides and tools for use by Aboriginal communities,
• development and implementation of a training and technical assistance plan to support the project staff and local evaluation steering committees, and
• provision of front-end consultation and technical support to communities developing and implementing project management and evaluation plans.

On-going performance monitoring will ensure on-going consultation and technical support to communities, monthly information on the activities and progress of the provincial AJS implementation, and the implementation and collection of operational and performance information for the Strategy, among other things.

Each year a brief annual status report for TBS will be prepared on activities and accomplishments over the previous year.

A mid-term evaluation report (due October 1998) will include information on the activities and accomplishments of the AJS across the federal, provincial/territorial and Aboriginal level. It will describe the individual projects that have been negotiated and implemented, and the degree to which organisational and operational capacity has developed at each site. It will include formal recommendations on whether revisions should be made to the original data collection regime, to ensure adequate information will be available for the Five Year Evaluation Report. Recommendations would be implemented by March 1999 to coincide with agreement renewals.

A five year evaluation report (due November 2000) will provide summary information on the activities and accomplishments of the AJS across the federal, provincial/ territorial and Aboriginal level, and document the degree to which the activities and accomplishments of the AJS are consistent with the AJS Operational Plan and explore reasons for changes or deviations in the implementation of the AJS.

Stakeholder surveys to be administered by March 2000, will address the questions of overall AJS system level impacts. Quantitative and qualitative information will be analysed and the draft five year evaluation report will be the basis for consultations with the federal and provincial/territorial governments and concerned Aboriginal organisations about lessons learned, goals attained and the future of the Aboriginal Justice Strategy. The Final Evaluation Report will be submitted by November 2000.

[1] Included across all three stakeholders are justice professionals including judges, Crown attorneys and defense attorneys.

Previous Page | Table of Contents | Next Page