A Facilitated Control Assessment Pilot
for Risk Management Planning
– Canadian Intellectual Property Office (CIPO)
May 1999
Report
Introduction
The Audit and Evaluation Branch (AEBADM), Operations Sector, to develop a risk management framework for the use of senior management.
In January, 1999 AEB defined and outlined the benefits of a risk management framework, briefly described three approaches, and recommended the PricewaterhouseCoopers (PWC) methodology over several others to the ADM, Operations Sector.
The ADM, Operations Sector agreed to a pilot, but stated that it should meet two objectives:
- firstly, a risk management assessment should be integrated within the planning initiative at the Canadian Intellectual Property Office (CIPO), a Special Operating Agency (SOA) within the Operations Sector.
- secondly, the lessons learned from the CIPO pilot should be used to develop a risk management methodology to be used throughout the Operations Sector beginning with SOAs.
The ADM, Operations Sector identified a representative from CIPO and the Operations Sector to work with AEB to select the methodology and consultants. Staff from the Audit and Evaluation Branch and CIPO worked together to complete the pilot.
The PricewaterhouseCoopers (PWC) risk control framework was selected for the pilot. As part of the framework, two half-day facilitated workshops were conducted for each high risk area identified by the ADM, Operations Sector. These high risk areas were: revenue and information technology management. The workshops were facilitated by PWC and AEB presented the introductory segment during the second workshop.
The action plan developed from the workshops was included in the CIPO business plan for the fiscal years of 1999-2000 to 2003-2004.
The balance of the report will cover the following:
- process used in the methodology;
- feedback received from the participants;
- lessons learned; and
- next steps.
Process
The phases of the facilitated control assessment pilot included:
- Senior management commitment/ Steering Committee
- Planning and foundation activities
- Facilitated risk control assessment workshops
- Reporting
- Monitoring of action plan
Senior Management Commitment/Steering Committee
Senior Management Commitment — CIPO
After PWC made a presentation of their methodology to the CIPO Senior Management Committee, the pilot was accepted and the Chief Executive Officer agreed to participate in the project. The attendance of the Chief Executive Officer is key to the success of a risk management project covering strategic risk areas.
Steering Committee
A steering committee was formed. It included representatives from AEB, Operations Sector and CIPO. The mandate of the steering committee was to select a risk management methodology and plan the project. The Committee selected PricewaterhouseCoopers (PWC) methodology.
Planning and Foundation Activities
Planning
AEB, CIPO and PWC agreed on the timing, workshop formats, documents and, most importantly, the selection of participants. It is important that senior management and stakeholders involved in the process attend the interviews and workshops.
The pilot began on April 6. Two half-day workshops were conducted for each risk area. On May 10, 1999 a report generated from the groupware used in the workshop was sent to AEB and CIPO.
Foundation Activities
During this phase, review of documents and interviews were completed prior to the workshops.
The purpose of the interviews was to:
- provide background to the study and explain concepts used in the project such as risk, control, etc.;
- clarify the business process that is the responsibility of the participant;
- obtain a description of the roles and responsibilities of the participant;
- discuss the outputs, risks, performance measurement systems, and required improvements to reduce/mitigate risks relating to the responsibilities of the participant; and
- explain and ask the participant to complete an electronic survey that requires 30 minutes to complete. The survey covers five control areas: control environment, risk assessment, information and communication, internal control and monitoring. Under each control area, questions were asked relating to issues that could lead to risks.
Facilitated Risk Control Assessment Workshops
Two half-day workshops were held for each issue. PWC facilitated the sessions using their groupware. Each participant used a laptop computer to provide their anonymous views and votes. All laptop computers were connected using groupware software that allowed instantaneous information to be projected on a screen. The information, contributed by the participants via the laptop, was then shared with others for discussion and refinement.
The first four-hour workshop covered the following:
- presenting of the project overview and approach;
- reviewing of CIPO strategic objectives to develop objectives that would align with the CIPO objectives for the high risk areas of revenue and information technology. The key objectives articulated for each risk area were developed by discussion and analysis of projected information contributed anonymously by participants;
- analysing and discussing the electronic survey results to be used when identifying key risks and controls later in the workshop; and
- identifying of risks for the key objectives developed for each high risk area. This was achieved by forming small groups and then using the technique of brainstorming to generate ideas by recording information in the groupware software.
The second four-hour workshop consisted of the following:
- rating risks identified during the first workshop by evaluating the business impact versus the probability of occurrence for each risk. Each participant voted anonymously;
- identifying controls needed to mitigate against the risks selected. This was achieved by forming small groups and then using the technique of brainstorming to generate ideas by recording information in the groupware software;
- rating controls by evaluating the effectiveness and efficiency of each control and then voting; and
- completing an action plan that highlighted key objectives, associated risks and controls, responsible person, targeted completion date and measurement criteria.
Reporting
A groupware report was printed for each issue and provided the following documents:
- an action plan covering the key objectives, associated risks and controls, responsible person, targeted completion date and measurement criteria;
- a control system evaluation survey and results;
- the identification of critical survey issues;
- a review and definition of the business objectives related to each issue;
- the identification and assessment of risks affecting the objectives of each issue;
- a list of risks and then voting on these risks based on the highest business impact and probability;
- the identification, evaluation of existing controls, and voting on these controls by evaluating their efficiency and effectiveness; and
- some notes on the development of the action plan.
Monitoring of Action Plan
One key to the successful implementation of a risk management framework is monitoring the action plan by senior management to ensure that it is implemented.
Feedback
Overall, the CIPO senior managers and participants agreed that the PWC risk management framework worked well and added benefit to the planning exercise.
The CIPO Chief Executive Officer described the process as a good one. He stated that it is "a standard brainstorming discipline with dedicated software for risk assessment. It works very well. The use of the software is a big plus since the anonymity is appreciated and remarks appearing on the desktop screen are projected anonymously on the front screen for discussion purposes. In addition, the facilitators were good." He further stated that some improvement could be made to the process. His suggestions are incorporated in the "lessons learned" section of this report.
Participants provided feedback anonymously at the end of the workshops, through discussions after the sessions, and by statements made during the introductory remarks from participants that had attended the first workshop. Their comments mentioned that the process:
- helped articulate key revenue and information technology management objectives that align with the overall CIPO strategic objectives;
- identified the associated risk and controls for the key objectives developed;
- completed an action plan to manage risks which was included in the CIPO business plan;
- mitigated the amount of time that senior management need to spend as well as shortens the overall chronological time necessary to get agreement on key risks and actions; and
- helped communication among the functional groups as they were able to understand the problems each encountered.
The participants indicated that they felt rushed at times and that additional explanation of concepts would have been beneficial.
Lessons Learned
The following are lessons learned from the pilot project:
- provide a training session of one hour prior to the workshop to:
- review the concepts used in the workshop such as risk, control, efficiency, effectiveness, business impact, alignment of objectives, etc.;
- provide practical examples to help clarify the concepts and scales to be used during the voting exercises. The handout should include a definition for the acronym "Smart", i.e. specific, measurable, actionable, relevant and timely, and should clarify that "Smart" is a good guide to use when deciding on objectives and controls;
- discuss brainstorming techniques and workshop rules such as parked item, etc.
- stress that complete sentences should be used when providing opinions in the groupware software for later discussion during the workshop; and
- distribute a handout to use during the workshop that includes all the information reviewed in the training session.
- obtain confirmation of the organization's strategic objective during the interviews to minimize discussion of these objectives. This would allow more time to develop the objectives of the processes examined, e.g. revenue and information technology;
- present a timely review of the concepts during the workshop;
- show how to use the survey results as a guideline for the risks/control exercises; and
- provide sufficient lighting to see the survey results and the information provided from the handouts given at the training session prior to the workshops.
Next Steps
The following are the "next steps" required to incorporate the risk assessment methodology in the Operations Sector:
- present a briefing of the CIPO facilitated control assessment pilot to senior management at other SOAs such as the Office of the Superintendent of Bankruptcy, Corporations Directorate and Measurement Canada as the first step to implementing risk assessment for these SOAs. This would meet the commitment made to the Department Management Committee to implement a risk assessment methodology by the Fall of 1999; and
- review the progress of the action plan in October or November, 1999 by AEB to report the status to the CIPO Executive Committee, ADM, Operations and provide copies of the report to CIPO pilot participants.
Management Action Taken
A pilot for risk management planning was held last May resulting in the production of two Risk Management Action Plans dealing with Information Technology and revenue management. The elements of the action plans were integrated into CIPO's 1999-2000 Five Year Business and Financial Plan. The majority of the action items were near term and implemented in the 1999-2000 fiscal year. Most notably the Risk Assessment provided the organization with concrete action elements that, when executed, enabled the organization to exceed its forecasted financial performance within the first year. Following the pilot, CIPO, in partnership with AEB, briefed a committee of SOAs on the benefits achieved and lessons learned from the Risk Management Pilot.
Longer term items from the Risk Management Action Plans which are scheduled to start during this fiscal year or which are currently underway include: establishing an action plan and pilot for implementation of a full costing methodology across CIPO (Fall 2000); creating the Strategic Planning and Finance Sub-Committee (Summer 2000); setting service level agreements for IT services (in progress); and linking core competencies to succession planning (in progress).
Date Created: 1999-05-31
|