|
Evaluation of the
Employment Equity Audit Program
Management Summary - Final Report
September, 2002 1.0 Introduction This report presents the evaluation conducted on the Employment Equity Audit Program (EEAP) that was completed in April, 2002. The report presents an introduction on the program, including a program logic model, a list of the priority evaluation issues addressed in the evaluation, the evaluation methodologies employed, and the summary of findings, conclusions and recommendations related to those issues.
1.1 Project Background
The Employment Equity Act (EE Act) which came into force on October 24, 1996, gives the Canadian Human Rights Commission (CHRC) the mandate to conduct audits of employers covered by the EE Act to ensure they are complying with all aspects of the EE Act. As part of the process of periodic program evaluation, an evaluation framework for the EEAP was completed in March 2001. However, while CHRC believed that this framework would be useful in developing the evaluation approach, there were some aspects of the framework which CHRC wanted to be addressed before conducting the evaluation. Specifically, CHRC wanted a program logic model to be developed, all stakeholders to be provided with an opportunity to provide input on the evaluation issues, the evaluation issues to be prioritized and the evaluation methodologies to be revisited. Consequently, the CHRC approached Consulting and Audit Canada (CAC) to develop a supplement to the framework and this was completed in September 2001 to address these aspects. CAC was subsequently engaged to conduct the evaluation study of EEAP.
The EE Act also contains a provision that, five years after coming into force, a comprehensive review of the provisions and operation of the Act shall be undertaken by a committee of the House of Commons. At the time of the writing of this report, hearings were underway as a part of this review. It should be noted that the CHRC and other partners have submitted proposals for the continuance of the program which if accepted and implemented will change, to some extent, the way in which this program currently operates.
1.2 Objectives and Scope
The objectives of this evaluation were to assess the relevance, success and cost-effectiveness of the EEAP. Fourteen priority issues within those categories were examined. The scope of this evaluation was restricted to the EEAP as delivered by the CHRC, although CAC also examined the roles played by the Treasury Board Secretariat (TBS) and Human Resources Development Canada (HRDC) who assist the public sector and private sector respectively on how to implement Employment Equity (EE) and achieve compliance with the EE Act.
The results of this evaluation are based on the review of documentation, interviews and survey of employers. The field work of the evaluation was largely complete in January 2002. Follow-up activities were conducted up to the beginning of April 2002.
1.3 Program Logic Model
The logic model presented in this report was presented in the supplement to the evaluation framework.
1.3.1 Background
A logic model is an essential component of any evaluation framework. It is intended to show, in a succinct fashion, the logic of how a program is expected to achieve its objectives. The model is based on the premise that program activities are designed and implemented in order to produce outputs which lead to immediate outcomes that lead to intermediate outcomes which lead to ultimate outcomes which will satisfy the original objectives. The development of a model facilitates the examination of the plausibility of the various relationships and provides an invaluable tool for the on-going and future assessment of the program. Furthermore, the development of the model typically raises some issues regarding the program which may or may not become issues which are addressed in an evaluation.
1.3.2 Elements and Structure
The narrative description of the elements of the logic model for the EEAP are included in “Annex B”.
1.4 Evaluation Issues
1.4.1 Scope of Evaluation Issues
In the Evaluation Framework completed in March 2001, an extensive list of evaluation issues were identified. However, the Commission wanted to ensure that all stakeholder groups were provided an opportunity to identify issues that were relevant to their particular perspectives. Furthermore, the Commission wanted to ensure that the subsequent evaluation focuses on only the priority issues. Consequently a revised list of priority evaluation issues was identified by CAC and approved by the CHRC for this evaluation.
When determining what issues should be examined in an evaluation of the EEAP, it is important to consider the mandates of the CHRC and of HRDC provided under the EE Act and of the wider mandate of the CHRC provided under the Canadian Human Rights Act (CHRA).
The Minister of Labour is responsible for the administration of the EE Act. However, the revised EE Act of 1995 gave the CHRC the mandate to conduct audits to monitor compliance with the EE Act. While the mandate of the EE Branch of the CHRC under the EE Act is restricted to compliance audits of employers covered under the Act, the CHRC has a much wider mandate in relation to EE under the CHRA to help reduce barriers to equality in employment.
In the conduct of an evaluation of a program, the scope of the issues to be considered should be restricted to those related to the organization’s mandate under the program. Therefore, in the case of the evaluation of the EEAP, the scope of issues was to be restricted to the Commission’s mandate of conducting compliance audits. HRDC is concurrently conducting an evaluation of the Legislated Employment Equity Program which relates to its responsibilities under the EE Act in relation to private sector, federally regulated employers and Crown Corporations with 100 or more employees. HRDC also intends to conduct an evaluation related to its responsibilities under the EE Act in relation to the federal public service some time in the future.
1.4.2 Evaluation Issues
Following is the list of priority evaluation issues that CAC identified for this evaluation and which were approved by the CHRC. These issues are divided into the program evaluation issue categories of relevance, success and cost-effectiveness.
Evaluation Issues - Employment Equity Audit Program
Relevance
1. What role should the CHRC play in promotion and education in relation to the EE Act?
2. Are the audit requirements demanded by the CHRC and its auditors consistent with the mandate provided to them under the EE Act?
Success
3. Why has the compliance rate on initial audits been so much lower than expected?
4. Are employers adequately informed of the requirements to comply with the EE Act as demanded by the CHRC? (To what extent is the advice provided by CHRC, HRDC, TBS and PSC consistent?)
5. Are the audit demands on employers reasonable? (Is what is reasonable different for large, medium and small employers?)
6. Is there consistency in the demands made on employers by different auditors?
7. What impacts have the audit activities had on compliance rates of employers?
8. What impacts have the audit activities had on representation levels of employers?
9. Does CHRC place an emphasis on process and numbers rather than on qualitative actions and outcomes?
10. Is there an appropriate balance between persuasion/voluntary compliance and directive/enforcement actions?
11. How useful is the annual report?
12. Is the audit selection process effective?
Cost-Effectiveness
13. To what extent can the audits be streamlined? (Would less stringent audits be more effective?)
14. Are the current resource levels adequate to fulfill the mandate of the program?
Back to top
|
Printable Version
Email This Page
