Audit report

IRIS Project

October 1999

Context

The preliminary work was carried out in January and February 1998 and the audit fieldwork from May to July 1998.

This audit encompassed the IRIS component activities of the Technology Enhancement Plan. Phase 0, I, II, III and IV activities of the IRIS project were reviewed. During phase IV of the project, the audit focused on the management framework, practices and procedures. The audit fieldwork was carried out from May to July 1998. As such, the audit report is a `snapshot' of the events that took place during the audit fieldwork phase.

Phase IV was completed during the audit fieldwork, and TBS approved a $14.6 million submission for Phase V.

The audit was conducted under the guidelines of the former Treasury Board Review and Internal Audit Policy which was in effect until March 31, 2001.

Objectives

To assess the effectiveness of the management of the project in anticipation of central agency requirements.
To assess the extent to which the system provides for adequate audit trails and controls over the management of the system's operation.
To determine if the IRIS system and others are Y2K ready.
To assess the extent to which security measures have been considered.
To assess the extent to which project objectives and anticipated benefits are expected to be delivered.
To determine whether lease or buy decisions for implementation are fully supported.
To assess the readiness of passport issuing offices to implement the TEP.
To assess the extent to which proposed methods of operation comply with the Canadian Passport Order (P.C. 1981-1472, June 4, 1981).
To assess the state of readiness to support and maintain the IRIS system.
To assess the state of preparedness of regional offices and Headquarters to respond to emergencies, disasters, system breakdowns, backup needs and the higher level of traffic expected over communication lines (i.e. contingency planning and architectural testing).

Background information and audit results

In keeping with the Blueprint for Renewing Government Services Using Information Technology, the Passport Office started to conceptualize the automation of their entire production process for Canadian travel documents. This initiative was undertaken to improve services to the public and increase the efficiency of the passport issuing process. The workflow automation concept at the issuing offices was initiated in 1991 and the actual work that led to the development of a multi-year project labelled the Technology Enhancement Plan (TEP), began near the end of 1992.

Initially, the TEP project was expected to be completed in two years after the fall of 1993 at a total cost of $15 million. After several years of development, the system is scheduled for implementation in 1999. The total project cost, as approved by Treasury Board Secretariat and the Department of Foreign Affairs and International Trade, is currently estimated at $32.5 million. As the TEP project progressed, several interrelated sub-projects were undertaken and are known as the IRIS application; Office Automation; Passport Card; and the selection of new printing technology.

In our opinion, the initial time frame to complete the project in two years was too optimistic. The project involves more than automation, it affects all functional and operational areas of the Passport Office as well as the culture, strategies and policies. The program consists in restructuring the organization, introducing the empowerment concept, instituting new values, and researching and developing new technology.

In general, management of the IRIS project during phase IV was good given its complexities. The Project Director is firmly committed to the success of the IRIS project which is a basic requirement for all systems under development.

The climate at the Passport Office is favourable to the introduction of the IRIS application. On average, 71% of survey respondents `buy into' the concept that the IRIS application will provide better client services. In addition, 79% of survey respondents, on average, are comfortable with using new technology.

The current system automates the manual workflow and shifts some of the passport production activities to the front counter. To compensate for this increase in front counter "service time", Operational Directors were allocated additional front counter staff. Unless the required staff is recruited and trained, clients will have to wait longer in line as the processing time at the counters will increase in comparison with the amount of time required in the current manual system.

The benefits of the IRIS system, as it now stands, are non-monetary and will not be transparent to the public. Nevertheless, we believe that the system is a good first step to automate the current manual process and will create an infrastructure for future initiatives.

We are uncertain whether the Passport Office will be able to implement the IRIS system by the Spring of 1999. The current work plan does not allow large schedule slippage and is based on a sequential completion of major activities.

The risk of project failure before, during and immediately after implementation always exists for any system under development. Statistics have shown that 31% of all information technology projects are cancelled before they are completed; 53% of those that are completed have on average exceeded their original cost estimate by 189% and satisfied only 42% of originally- proposed features and functions; and, only 9% of the projects were completed on time and within budget.

We believe that the IRIS project could fail if the following critical actions are not taken before national implementation:

the system must be fully tested to determine the effects of an entire issuing office using IRIS. The client service time at the issuing offices must be independently calculated in this environment to ensure that clients are not subject to excessive waiting time;
the implementation period should be phased over a longer time period rather than occur over three months as scheduled. This will allow further testing of the system as a whole and provide adequate time for users to adjust to the new workflow;
the threat of the `Year 2000 Problem' must be a priority for the Passport Office. The IRIS application must be certified by the vendor and the foundation environment must be Year 2000 compliant. A formal contingency plan should be developed and include a provision to certify the CUPID system as Year 2000 compliant;
the Project Director and the Departmental Security Officer should ask the RCMP to carry out a Security Evaluation Inspection or ensure that the system is subject to Certification and Accreditation;
a comprehensive risk assessment is needed to identify all possible threats to the project. Members from the IRIS team, user group and functional areas should participate in this exercise. A continuous risk assessment methodology is recommended;
a formal communication function within the IRIS organization structure should be established to develop a plan and strategy that address the:
- communication roles and responsibilities of the issuing managers, regional directors and IRIS team; and,
- pertinent flow of information to and from the issuing offices in order to prepare staff for national implementation and collect feedback for improvements to the system;
an open communication environment should be fostered to allow users to raise their concerns. In evaluating users' concerns, criteria to segregate change management issues from essential enhancement to the system must be developed.
the roles and responsibilities of the managers at the issuing offices prior to and during national implementation should be formally developed;
a human resources plan and strategy should be developed to address training, change management, stress management and performance evaluation;
the operations of the offices that provide issuing services at the National Capital Region should be examined to determine if there is a need to modify the application or to re-engineer the work flow;
criteria to evaluate the readiness of the system for full national implementation should be established. The criteria should include not only factors relating to the development of the application but also `soft issues' needed to make an entire system successful;
a written opinion should be obtained from the legal counsel to ensure that the IRIS system complies with the Passport Order.

In addition, to improve the effectiveness and efficiency of the project management, we recommend the following actions:

a formal project charter should be developed, maintained and communicated to the project team and functional areas that provide advice and support. The project charter should include an updated organizational chart and outline the roles and responsibilities of the IRIS team and functional and operational areas affected by the system;
an independent quality assurance function should be established within the IRIS organizational structure;
more staff should be recruited to the IRIS team;
a system to formally report and monitor the project cost on a regularly basis should be developed;
the IRIS team should continue to specify contract deliverables in sufficient details and involve Passport Office staff in developing the statement of work to avoid misinterpretations;
the authority of the Project Director to make decisions should be re-established;
there should be discussions with other departments who have developed their own systems to share lessons learned;
to address change management issues and identify essential enhancements to the system:
regional directors and managers must play an active role during implementation; and,
on-site visits by the IRIS team should be conducted regularly during the testing and implementation stages.

Implementing the recommendations may delay the national implementation of the IRIS application. Strong support from the Executive Committee to implement the recommendations expediently and endorse the IRIS team will prevent undue delays.

Status of action plan

The actual system was fully tested by the IRIS improvements division and is fully functional and some work was done regarding client service time.

The implementation was spread over a longer period of time with all of the regional offices implemented in the first half of fiscal year 2001/2002.

The IRIS system was tested and certified for Year 2000 and no issues arose with the switch from 1999 to 2000.

Information Management and Technology has conducted both a threat and risk assessment and a vulnerability study in the last year or so. These studies produced a number of recommendations which were used to prepare an action plan. Some of the items in the action plan, such as Information Technology security policies are underway.

The IRIS improvements team interfaces with the users to accept and evaluate their concerns and requests. Information Management and Technology is implementing a change management process by which all system changes including those for IRIS are reviewed by the Change Advisory Board (CAB). The IRIS improvements team is represented on the CAB. Information Management and Technology has established a help desk as a single point of contact for questions, concerns and problems with all information technology systems and services.

The system was fully tested from a functionality and capacity point of view prior to full rollout. A transition plan was put into place to ensure support was available by in-house staff, and the system is almost entirely supported by Passport Office staff at this point.

The Passport Office as a whole is developing a corporate methodology for project management which includes formal documents such as project charters and feasibility studies, formal stages with sign-offs, formal project management officers, and project teams. The plan is to implement this structure for new projects but not necessarily back track for those underway.

The IRIS improvements team responsible for quality assurance testing of any new IRIS releases has been established. Such testing is to ensure that new releases function according to specification. A quality control is also implemented within software development to ensure developed code is reliable and stable.

Full- time permanent staff are recruited for both software development and testing. The IRIS improvements team is planning on recruiting more staff.

The costs for software development and support of the system from a technical point of view are now included in the Information Management and Technology base budget. This budget is subject to all the regular budget planning and monitoring procedures in place.

All functional specifications are developed by the IRIS improvement team, users and representatives from Information Management and Technology software development. This section is fully responsible for preparing the technical specifications for all new releases and delivering the same for use either by in-house staff, contractors or a combination of both.

Information Management and Technology is extensively involved with the implementation of new releases of the system and visits sites as required. Regional directors and managers are kept in the loop with respect to the direction for the system as well as for rollout of new releases. Also, as previously mentioned, Information Management and Technology is implementing a formal change management process to review and authorize system changes including those for IRIS.