Synopsis of the Demographic Profiles of the Internal Audit and Evaluation Communities in the Canadian Federal Public Service

Background

Results for Canadians: A Management Framework for the Government of Canada identifies the need for better-positioned and stronger internal audit and evaluation functions within the federal government. The Policy on Internal Audit and the Policy on Evaluation, approved by Treasury Board Ministers on February 1, 2001 and now in force, enable internal audit and evaluation to play a key role in managing for results. One factor in the successful implementation of the policies is engaging and retaining professionals with the appropriate skills to meet existing requirements and increased demand in the internal audit and program evaluation functions.

To assist in the development of community-wide strategies for recruitment, training and development, and retention, the Centres of Excellence for Internal Audit (CEIA) and for Evaluation (CEE) at Treasury Board Secretariat (TBS) contracted with the Research and Development Division of the Personnel Psychology Centre, Public Service Commission (hereinafter, PPC) to provide a snapshot of the two professional communities, their training and development profiles, their future training needs, and a gap analysis. This information is contained in the report "Demographic Profiles of the Internal Audit and Evaluation Communities in the Canadian Federal Public Service" prepared by PPC for the Centres. The authors of the full report are Penny Faulkner and Vince McDonald. Thus far, the Centres are utilizing the information from the demographic report to design recruitment and learning strategies for the communities. Heads of Internal Audit and Heads of Evaluation may also find the report useful in developing human resource plans as well as specific additional or complementary strategies and activities for their professional staff.

The purpose of this document is to summarize the key findings of the report and furnish some inferences made by the Centres on their implications.

Administration of the Survey

In June 2001, PPC administered a survey to the members of the Internal Audit and Evaluation communities, including both term and indeterminate employees, located in 59 departments and agencies. PPC distributed 583 surveys, 355 of which were completed and returned. The response rate of 61% is considered high.

Approach to the Analysis of the Survey Data

One objective of the survey was to identify distinct professional communities of auditors and of evaluators within the federal government. However, respondents identified the following categories when asked to choose their primary affiliation:

1. Audit
2. Audit and Review
3. Evaluation
4. Evaluation and Review
5. Evaluation and Audit
6. Evaluation, Audit and Review

The range of categories reflects the respondents' perception of the wide variety of services they provide.

DEMOGRAPHIC PROFILE

Functional Community Membership

Click here to view a full sized image

Profile of the Communities

The demographic report has used the six groups above to analyze the responses provided by respondents on general demographic information, training and development, and career planning. "Respondents" is employed in this synopsis to refer to the total population of the six groups. For the purposes of training and development needs, benchmarking exercises, and demographic forecasting, respondents were reconstituted into two larger groupings: Evaluation combined with Evaluation and Review, and Audit combined with Audit and Review. These groupings will be referred to as the Evaluation Community and the Audit Community. The Evaluation Community represents 32.3% of total respondents, while the Audit Community represents 37% of total respondents¹.

The PPC carried out selected analyses and benchmarking in order to compare the respondents with similar groups in the Public Service. The results are incorporated in the following profile of the respondents.

¹The "Combined Audit Community" and "Combined Evaluation Community", as referenced in the actual study, refers to Audit combined with Audit and Review, and Evaluation combined with Evaluation and Review.

Employment Equity and Language

Among the survey respondents, the overall representation of employment equity group members that include women, persons with disabilities, aboriginal peoples, and members of visible minority groups was below TBS published estimates for the Public Service. Only 37.1% of the survey respondents were women compared to the TBS statistic of 51.4%. Only 2% identified themselves as persons with disabilities compared to the TBS estimate of 4.7%. There were no respondents who self-identified as aboriginal, while the TBS statistic for this group is 3.3%. Only 4.6% were members of visible minority groups compared to the TBS statistic of 5.5%.

The respondents were primarily Anglophones with about 70% identifying as English speaking. This is similar to the official representation of Anglophones in the Federal Public Service. The predominant language of work was English (71.5%). Only 2.6% reported that the primary language at work was French while the remaining 25.9% used both French and English on the job. It is interesting to note, however, that 30% of respondents identified their first official language as French.

These results bear consideration by Heads of Internal Audit and of Evaluation, as well as the Centres of Excellence for Internal Audit and Evaluation, in developing strategies for recruiting staff and optimizing the work environment.

Occupational Groups and Levels

DEMOGRAPHIC PROFILE

Occupational Groups

A considerable mix of occupational groups and levels was represented in the respondent sample, with the principal groups being ES and AS. For the most part, the ESs were evaluators, while the ASs were auditors.

For the Evaluation Community, the predominant occupational groups and levels were ES6 (16.4%) and ES5 (12.7%), followed by AS7 (11.8%) and AS6 (9.1%). The Audit Community comprised respondents primarily in the AS6 (29.4%) group and level followed by AS7 (17.5%).

To provide more precision regarding the seniority level of survey participants within a particular work unit, nine departments were selected from respondent organizations of 10 or more Audit, Evaluation, or Review employees. Respondents were divided into categories of junior, intermediate and senior, based on classification group and level. There was a prevalence of senior employees across all selected departments. For three departments, percentages of senior employees ranged from 78% to 90% of the total complement of respondents. That is, at the time of the survey, it appears that these departments had virtually no individuals at the junior or intermediate levels in Audit, Evaluation, or Review. Overall, the report suggests a need for concerted, systematic succession planning to replace senior employees and to prepare those in the wings adequately.

Education

The survey respondents reported a relatively high level of education overall. Approximately 42% had graduate degrees (MA and PhD) and 36% had at least a bachelor level degree. The Evaluation Community, in particular, had significantly more respondents with MA and PhD degrees. Most of the respondents had graduated almost two decades ago. All respondents reported a relatively high need for training to meet future work requirements.

Work Experience

DEMOGRAPHIC PROFILE

Years of Service in Government

Approximately 30% of respondents reported having been in the federal government for more than 25 years. The communities seem mature in terms of the dates of their completed education and the length of their work experience, suggesting the importance of succession planning to rejuvenate the communities and maintain their knowledge and experience base.

Time in Position

The survey respondents overall were career public servants having on average 17.8 years of government service. On average the respondents were found to have been in their current position almost a year longer than the norm of 3.5 years. This suggests that, historically, there has been relatively low mobility among the respondents. The report noted, however, that the overall statistics were heavily weighted by a group of long-term employees. A further analysis by the PPC showed that 40.2% of the Evaluation Community had moved into their current position within the last year, a rate about twelve times the feeder group norm. The rates for the Internal Audit Community were also high at 28.7% or eight times the norm. This indicates some recent mobility within the communities.

Eligibility to Retire

DEMOGRAPHIC PROFILE

Eligible for Retirement

Almost one third of the respondents indicated that they were eligible for retirement within the next 5 years. When compared with Public Service retirement statistics, the respondents' potential retirement rate is four times the norm.

While employees may be eligible to retire, it is equally important to know when they plan to do it, given the current flexibilities in the age of retirement. In that regard, 8% of respondents indicated that they intended to leave the government over the next 3 years, while 15.8% answered they might leave or stay and another 76.2% indicated they plan to stay. By analyzing these data by respondent group, the demographic report projects that the Internal Audit Community would have to increase its strength by about 32% and the Evaluation Community by about 23% in order to maintain their current sizes over the next three years. These rates may seem inflated since some who are eligible to retire may decide to stay. However, as the report notes, the eligibility to retire rate rises by 17% from year 2 (when it is 14%) to year 5 (for a five year total of 31%), underscoring the importance of systematic human resource planning for both recruitment and succession.

DEMOGRAPHIC PROFILE

Future Plans - Next 3 Years

The report notes, however, that there is no direct university stream from which to draw new talent for either audit or evaluation professionals and on-the-job training may not fill the anticipated demographic gaps rapidly enough to maintain community stability and, at the same time, retain corporate memory. The Audit Community has been successfully using the Internal Auditor Recruitment and Development program (IARD) to fill its junior vacancies and provide for a career path. CEIA will continue to promote this program. CEE is currently working with the Evaluation community to determine options for a recruitment and internship program. In addition, CEE is encouraging the Heads of Evaluation to make use of existing programs such as Post-Secondary Recruitment, Co-op Internships, and the Management Trainee Program to address entry-level recruitment needs.

Training and Development Needs

The use of structured training programs would address the demonstrated interest by both the Evaluation Community and the Audit Community, not only in professional methodological training, but also in learning how to manage and in understanding more about how government works.

Both communities reported a relatively high need for training and development to meet future work requirements. Many identified deficits in the hands-on types of competencies related to their functional areas and identified needs such as "refreshers in the use of specific tools" and "customized training".

Respondents from the Evaluation Community identified a significant need for training in management and research techniques such as quantitative and qualitative analysis. In addition, these respondents identified the need for skill training in the interpersonal aspects of their work with clients.

The Audit Community was also concerned with the acquisition of skills of an interpersonal nature, such as negotiation and facilitation and, in addition, identified a significant need for training in information technology. The report noted that auditors appeared not to have been able to avail themselves of this training, suggesting a fairly significant skill gap in the technology area.

Both the Internal Audit and Evaluation Communities desired training in knowing their clients, understanding how government works, and in promoting their work within departments. The expressions of this opinion were stronger from the Evaluation Community.

Most respondents reported that they had attended public sector conferences, workshops, and seminars and ranked these as the best source of training they had received. Training in Results-based Management and Accountability Frameworks (RMAF) and general training in Finance were identified as recent government sponsored training by Evaluators and Auditors respectively.

Respondents also indicated a desire to receive more training from professional associations. In particular, they specified the Canadian Evaluation Society (CES) and the Institute of Internal Auditors (IIA) as the more common sources of recent professional learning and also as preferred sources of continuous learning. This suggests that the Centres should liaise, and as appropriate, partner with these organizations as part of future plans for overall development of the Internal Audit and the Evaluation Communities.

Both Evaluators (about 50%) and Internal Auditors (about 44%) reported that 60% or more of their work could be learned on the job. These results will have implications for training and development plans and for establishing the right mix between learning on the job and more formal approaches to training.

Feedback for the TBS Centres of Excellence

Almost the entire respondent population (95%) was aware of the TB policies and initiatives impacting on the renewal of their communities. Respondents overwhelmingly identified the need for TBS Centres to build capacity. While there was some need acknowledged for the Centres to provide training in specific competencies such as risk assessment and RMAF training, there was also an interest simply in having the Centres be general course providers.

The respondents felt that the Centres should focus on professional issues and, in particular, on the development and monitoring of standards and practices. The communities were looking for a centralized source of leadership that included an educational role in providing specific knowledge for use by the communities and also in educating clients as to the nature and implications of audit and evaluation work. All respondents identified recruitment and career progression as priorities for the Centres.

For More Details

For more details and related analyses on the communities, the full report "Demographic Profiles of the Internal Audit and Evaluation Communities in the Canadian Federal Public Service", prepared by the Personnel Psychology Centre of the Public Service Commission, is available on request from the Centre of Excellence in Evaluation. .