12.10 In recent years, the amount spent on developing large-scale computer systems, in both the private and public sectors, has risen dramatically. Historically, many of these systems have failed to meet the needs of prospective end-users and/or have not incorporated sufficient financial and managerial controls.
12.11 Within the federal government, the Public Accounts Committee, this Office and a number of deputy ministers, through the Treasury Board Secretariat Committee on Information Management Systems, have expressed concern over the difficulties experienced by the government in implementing systems development projects.
12.12 A recent United States research study completed by The Standish Group, which included both private and public sector organizations, reported that "a staggering 31.1 percent of projects will be cancelled before they ever get completed" and that "52.7 percent of projects will cost 189 percent of their original estimates." This study noted that "on the success side, the average is only 16.2 percent for software projects that are completed on-time and on-budget." In larger companies, "only 9 percent of their projects come in on-time and on-budget."
12.13 Nevertheless, most organizations have become either largely or totally dependent on their computer systems. With current technologies, systems are becoming significantly more integrated and complex. For the remainder of this decade, emerging technologies will place even greater demands on all organizations - including the federal government - to introduce technology in a careful, well-managed way.
12.14 Within the federal government, a number of organizations have specific responsibility for portions of the systems development life cycle. Exhibit 12.1 lists these organizations and their respective responsibilities. Exhibit 12.2 provides an overview of key steps in the systems development process.
12.17 We chose these specific systems because they:
12.19 The audit also considered the impact of environmental factors, such as changing priorities and objectives of the government, new technologies, and changes in organizational structure that affect the implementation of such systems.
12.20 In addition, we examined the role of central agencies in the approval and funding of these projects, because the independent review and analysis of data provided by departments, in support of a request for approval and funding of a project, is a key control within the government's expenditure management system.
12.21 The scope of this audit did not include a review of the procurement of information technology, as this is the subject of a separate audit to be reported in 1996. In addition, we did not audit the role of the contractors in these projects, since it is the responsibility of departmental management to monitor and manage the contractor's performance.
12.23 As the basis for our audit, we conducted a risk assessment, a process through which professional judgment is applied in considering certain basic questions about virtually any project, including systems under development. These questions include:
12.27 In our current audit, we found that only one of the four systems reviewed, the Integrated Departmental Financial and Materiel Management System, is presently being managed in a way that deals effectively with the risks associated with the project. Of the remaining three:
12.30 Our risk assessments for all projects represent a snapshot in time of a project's status and the risks that project management faced at the time of the assessment.
12.31 The findings that have emerged from this audit are not new. Although we have made similar observations in past audits, the issues that have led to these observations persist and continue to impair the ability of the government to successfully introduce information technology. Although management has generally identified those risk factors that continue to cause difficulties in implementing systems, it has not been effective in reducing the risks associated with large multi-year projects.
12.32 In comparing the results of this audit and previous audits with the results of private sector research studies, the factors that influence the risks associated with systems development initiatives in the Canadian federal government are similar to those reported in the private sector.
12.33 Current approaches to developing and managing systems development projects must be changed. If current practices are continued, and private sector results are applied, a significant number of the 25 large-scale systems, with total planned expenditures of $2.1 billion, will experience serious difficulties, resulting in some cancellations and/or significant cost overruns.
12.34 We recognize that, while there are similarities in the factors that affect the risk of systems under development, there are also differences in the ways in which the private sector and the federal government deal with projects experiencing difficulties. Traditionally, there has been a tendency within the federal government to reorganize and restructure projects and apply additional resources rather than cancel projects.
12.35 Whether projects are cancelled or restructured, the impact of project cancellations and/or cost overruns is significant. In the case of the PSCS project, Public Works and Government Services Canada (PWGSC) had spent approximately $61 million, of the estimated total project cost of $119.5 million, by the time it terminated the project. These figures reflect only the Department's expenditures for purchased services and do not include the cost of departmental resources dedicated to the project, or such elements as the cost of money.
12.36 There is little doubt that PWGSC did gain improvements in its departmental infrastructure as a result of the system development work that was done; however, a significant amount has been spent for which the planned savings will not be realized. The Secretariat has noted that the government will still receive a return on its investment in the PSCS system, as the funding levels of PWGSC have already been permanently reduced by the savings that were expected as a result of PSCS implementation.
12.37 The focus of our report is to identify those variables that influence the risks associated with systems under development. Our observations and findings are presented on subsequent pages under the following headings:
12.39 We noted a number of management-related weaknesses associated with the introduction of these systems. These include:
12.42 The PSCS was designed to upgrade and modernize existing public service compensation systems within Public Works and Government Services Canada, with a view to improving service levels and reducing the cost of compensation administration within the Department.
12.43 A number of organizations play a significant role in the business processes for compensation administration. These include the Treasury Board Secretariat, which, on behalf of the Treasury Board, is responsible for developing and monitoring the human resource policy framework, under which the rules are set for payment of public servants, and Public Works and Government Services Canada (PWGSC), which administers compensation within the federal government.
12.44 As part of our audit we attempted to determine whether the approach to modernizing the compensation administration process within the federal government included sufficient analyses of the underlying business issues that influence the cost of compensation administration. We expected to find a thorough analysis of these issues by both PWGSC and the Treasury Board Secretariat.
12.45 It was recognized from the outset that the compensation environment was complex. The Secretariat also realized that the necessary legislative and collective bargaining changes needed to simplify the complexity would have taken years to complete. The Secretariat has stated that the need for a fully integrated compensation system such as that proposed by PWGSC was too great, and the significant savings to which PWGSC committed too alluring, for the project not to proceed. It appears, however, that the complexity of the compensation environment was not fully appreciated at the outset by either organization.
12.46 Public Works and Government Services Canada undertook the development of PSCS without the benefit of redesigning all components of the existing business procedures or processes. In this instance, the complexities of the compensation policy framework contributed significantly to the difficulties encountered in trying to develop this system.
12.47 The government anticipated that the cost of administering compensation would have been reduced had the PSCS initiative been successful. However, the system would not have achieved the full cost reduction potential because of the failure to analyze and resolve underlying business issues, which were a root cause of the high cost of compensation administration within the federal government.
12.48 If the government is to deal effectively with its administrative costs, it must take the current opportunities to re-engineer its business processes and consider the administrative cost of its policy framework.
12.49 We did not find any indication that the Secretariat had specified expected overall levels of performance for the government-wide cost of compensation administration.
12.50 We note that the Secretariat has recently initiated a pay and benefits re-engineering project to address some of these issues.
12.52 In our audit we found that the effectiveness of project sponsorship varied from one project to another.
12.53 The Integrated Departmental Financial and Materiel Management System (IDFS) project illustrates effective project sponsorship and commitment from senior management. At Transport Canada, the project sponsor played a strong and active role. The sponsor's involvement contributed to overcoming significant difficulties encountered shortly after the project's inception.
12.54 Soon after Transport Canada signed the contract, it was realized that the project was in severe difficulty. The contract did not accurately reflect the requirements of the project, the project had slipped in its planned schedule, and costs were expected to increase. The sponsor of the project responded to these problems by:
12.56 The Common Departmental Financial System (CDFS) project illustrates the consequences that can follow when roles and responsibilities within a project are not clearly established.
12.57 The CDFS development was initiated in 1989 to replace an existing government financial system (FINCON). CDFS was intended to meet the functional needs of a group of users, most of which were small-to medium-size departments, while satisfying the requirements of the central agencies. At the same time, the Treasury Board Secretariat initiated its Financial Information Strategy, a project that was intended to allow the Government of Canada to renew its financial system infrastructure and to develop a new set of guiding principles for financial systems and management.
12.58 In an attempt to reduce the number of financial systems within government, it was agreed that CDFS would be a preferred optional system of the Secretariat's Financial Information Strategy.
12.59 Under terms of an agreement signed by Public Works and Government Services Canada(PWGSC) and the then Office of the Comptroller General (now part of the Treasury Board Secretariat), as joint developers of CDFS, some of the respective roles and responsibilities for each organization were established. The agreement failed, however, to clearly establish a project sponsor. The Comptroller General was responsible, in concert with the system's users, for defining functional requirements (what the system must do to satisfy users' needs), setting priorities, approving plans and monitoring the delivery and performance of the system. PWGSC, as product manager, was responsible for developing, implementing and maintaining CDFS.
12.60 By not clearly establishing the respective roles and responsibilities in this instance, each of the two lead organizations or joint developers looked to the other as being responsible for the successful introduction and acceptance of the project.
12.61 Many viewed the Office of the Comptroller General (OCG) as being responsible for the CDFS project because it was initiated under the auspices of the OCG's Financial Information Strategy. This view was reinforced by actions of the OCG, which, in the initial stages of the project, initiated and funded an independent review of the project to assess project status and risks. As well, the OCG established and co-chaired a CDFS Management Board, which was intended to provide a forum through which advice and guidance could be provided to PWGSC and through which user inputs could be provided.
12.62 By the end of 1992, the CDFS project had slipped significantly behind schedule, and the functional requirements had not yet been agreed upon. Furthermore, the government's Financial Information Strategy, which CDFS supported, had still not been finalized. (We note that at the time of this audit, the Secretariat had renewed its efforts to develop the Financial Information Strategy.)
12.63 After it became apparent that difficulties and delays were being encountered in completing the project, the Office of the Comptroller General, as one of the joint developers, quietly reduced its support for CDFS at the end of 1992, and did not actively encourage departments to consider the system. However, it did so without advising either PWGSC or the federal government financial community of its decision.
12.64 In this instance, decisive action was required to either support the continued development of CDFS or to cancel it. As a consequence of the failure to establish clear roles and responsibilities for the successful implementation of the system, approximately $10 million has been spent since late 1992 on developing a system for which key elements of sponsorship are missing and for which user confidence has eroded.
12.66 The problem of lack of experienced project management is not one that is unique to government. Research studies conducted within the private sector on causes of systems development failures have cited the lack of experienced project management as a significant factor that contributes to project risks.
12.67 In all four of the projects reviewed as part of this audit, the project teams either did not, at the outset, have experience with projects of similar size and complexity, or lacked experience with the technologies associated with the project. Although the teams developed experience as the projects progressed, their lack of experience at the start contributed to increased project risks.
12.68 In two of the four projects, the Integrated Departmental Financial and Materiel Management System(IDFS) and the Common Departmental Financial System(CDFS), key personnel were replaced at key points during the project. By the time this occurred, both projects had slipped significantly behind schedule. However, in the case of IDFS, those changes helped to turn a project that was in difficulty into one that is currently being effectively managed.
12.69 While changes in personnel within the CDFS project are helping to deliver a final product, they have not offset the delays that had already occurred. The delays, together with a rapidly changing business and technological environment, and a project for which roles, responsibilities and accountabilities were not clearly established, have meant that the developers cannot build the system quickly enough to catch up with the evolving expectations of those who may eventually use it.
12.71 In our audit we noted that the involvement of users varied from project to project.
12.72 The Income Security Program Redesign(ISPR) project provides an example of effective user involvement. Management of the ISPR project has taken seriously the issue of involving users, getting their buy-in and involving them in decision making. Realizing that the project would not be successful without careful management of organizational change, extensive consultation with end users was planned. For example, to involve users from the outset, management took a number of steps such as:
12.74 In the Common Departmental Financial System (CDFS) project, the nature and extent of user involvement has fluctuated. At its inception, CDFS had two objectives. One was to replace an existing financial management system. The other was to meet all the requirements of the Financial Information Strategy being developed by the then Office of the Comptroller General. Today, because of the lack of user commitment and support, CDFS has only a small number of users. At the time of the audit, Public Works and Government Services Canada was the only department and there were three small agencies (of the approximately 20 users of the current system that CDFS was designed to replace) electing to use CDFS.
12.75 The primary forum for co-ordinating users' input and providing guidance to PWGSC for the development of the system has been the CDFS Management Board. However, Board members whom we interviewed as part of this audit do not believe that they have had any meaningful involvement in or commitment to the project. Generally, potential users remain unconvinced that CDFS will provide them with the same level of performance and functionality that they have with existing systems. Although the department that conducted the pilot test of CDFS had committed significant human and financial resources to assessing the system, it decided not to implement the system. Deficiencies cited by potential users as reasons why they chose not to use the system include concerns over functionality and performance such as:
12.77 As well, some potential users have indicated a desire to assess the outcome of the current initiative of the Treasury Board Secretariat to assess a number of commercial software packages.
12.78 Effective user involvement and commitment to the systems development project is critical to the project's success. The CDFS project illustrates two common problems. First, the rate of change of technology in today's environment invariably makes potential users more demanding. Accordingly, a project must have the flexibility to respond, where appropriate, to evolving expectations. Second, successful systems development projects and, in particular, shared systems, require continued user commitment or buy-in.
12.81 Accordingly, it is important that management has effective processes for managing systems under development. Such processes include:
12.84 We are also concerned over the time it takes to initiate corrective actions. The Income Security Program Redesign project illustrates this point. While project monitoring identified delays in the information technology component of the project in July 1994, effective corrective action, in terms of revised plans and schedules, was not completed until April 1995.
12.85 As discussed below, we noted that improved procedures are required for monitoring time used by project team members in completing tasks.
12.86 We found that the extent of tracking of project hours varied from one project to another. In most instances, Crown time devoted to a project was not tracked and, in some instances, information on contractor time was not provided to project managers. Under fixed price contracts, such as those in place for the systems we reviewed, this information does not affect the price that the Crown is required to pay. However, the nature of the contract has been used by contractors as a reason not to provide project management with full information on the time taken to complete specific tasks. In our view, such information is an essential component of sound project management.
12.87 Complete information on time used for a project provides management with information on levels of effort expended to achieve the results to date. It is also required to predict whether more or fewer resources will be needed to complete the project on time.
12.88 Such information is critical to assessing the continued relevance of the original business decisions and is a necessary first step to respond to the changing business environment.
12.89 Monitoring by Treasury Board Secretariat. While departments have primary responsibility for adequate monitoring of the progress of their projects, there is also a responsibility on the part of the Secretariat, because of its overall responsibility for the financial management of government, to monitor projects of significance and risk.
12.90 We expected that the following key areas would be monitored:
12.92 We noted that the Secretariat receives information on project status from departments, and in general, that this information is reviewed and analyzed. In some instances, the Secretariat has, either on its own or in concert with other departments, initiated corrective action. For instance, in response to concerns about delays in the CDFS project, the Secretariat initiated and funded an independent review of the status and risks associated with the project. In other instances, for example with respect to PSCS, while concerns were identified, actions were either late in coming or ineffective in resolving the concerns noted.
12.93 We also reviewed the nature and extent of the Secretariat's monitoring of the effectiveness of its information technology policy framework in contributing to the successful introduction of systems development initiatives.
12.94 We anticipated that such monitoring would consist of the tracking and analysis of information technology spending within the federal government, and an assessment of the reasons for past successes and failures. In our opinion, such data and analysis are essential for assessing the effectiveness of the information technology policy framework.
12.95 We found that while the Secretariat has data on individual departmental projects, such data are not analyzed, from a government-wide perspective, to assess the effectiveness of the policy framework. The Secretariat has not regularly captured and disseminated lessons learned from past successes or failures as a basis for future improvements.
12.96 We noted that in response to concerns raised by departmental deputy ministers, the Secretariat has begun an examination of all aspects of the management framework for large information technology projects.
12.99 One such measurement technique used in the software development industry is called Function Point Analysis (FPA). This technique provides management with a relative estimate of the size, complexity and risks associated with a project.
12.100 In FPA terms, any project containing more than 5,000 "function points" is considered complex and entails significantly more project risk. A function point can be considered to be a task that the computer software must perform. Research on FPA carried out in the United States has shown that systems development projects that have more than 10,000 function points have about a 50 percent chance of being cancelled before they are implemented.
12.101 The use of software metrics does not, by itself, guarantee success in implementing information technology. However, the use of such techniques is an integral component of sound project management, which provides management with a benchmark for measuring the size and complexity of a proposed initiative, as well as an indicator of the levels of risk that a project may face.
12.102 We noted that such measurement tools had not been used during the development of the business cases for any of the systems included in this audit. For two of the four systems, management had completed a function point count only after the projects had begun:
12.104 By comparison, some private sector organizations that use function point analysis will not, because of the associated high risk, undertake systems development initiatives that exceed 5,000 function points.
12.106 The complexity of the four systems reviewed as part of this audit varied, as did the reasons for their complexity.
12.107 For example, Transport Canada acquired a commercially available software package for the IDFS project and modified it to meet the Department's needs. In this instance, the modifications involved developing interfaces with existing departmental systems and the government's central accounting systems and customizing some of the system's reporting features.
12.108 While the acquisition and installation of a commercially available software package should have been the least complex of the systems we reviewed, the extensive modification of the software to meet the needs of the Department and to be integrated with central accounting systems significantly increased the project complexity.
12.109 As well as adding complexity, such modifications may make the implementation of new releases of the software more difficult and costly.
12.110 The use of commercially available software offers the potential for cost savings. A significant portion of the savings comes from limiting the extent of changes to the commercial software package. Within the private sector, a guideline often used for changes to commercial software is to limit them to less than 10 percent of the functionality (what the system will do for its users) of the software package.
12.111 In contrast, the remaining three systems had aspects that are, or were, significantly more complex.
12.112 For example, in the case of CDFS, the developer faced the task of building a shared system that would interface with many different departmental systems and architectures, all of which added to the general complexity of the project.
12.113 In such instances, project complexity and cost could be reduced through greater adherence, on a government-wide basis, to common standards and architectures. Such increased standardization across departments would also facilitate achievement of the government's objectives.
12.115 The impact of any uncertainty arising from imprecise requirement statements affects two key areas:
12.117 However, at the same time, the expected savings in departmental operating costs, after successful implementation of the system, rose from an initial estimate of approximately 300 person-years to 781 person-years.
12.118 The second possibility is that in order to keep project costs stable, the project requirements will be reduced to match the funds available. This means the system will perform fewer functions than were originally planned, which may increase the risk that the system will fail to meet users' expectations or needs.
12.119 The IDFS project provides an example of a project where scope was reduced to stay within the original budget. In order to maintain the $60 million ceiling for the project and to avoid an estimated $20 million increase in contract price, management decided to redefine and prioritize the original scope of work to eliminate some items, while maintaining what management considered essential functionality.
12.120 A decision to reduce a system's scope is sometimes reversed after it is discovered that the "downsized" system will not adequately meet users' needs. In these cases, subsequent contracts or enhancements can add significantly to the project cost. This happened in the case of the CDFS project. In 1992, to facilitate earlier delivery of some components of the projects, approximately one third of the system's original requirements were eliminated. While the project scope was reduced, the contract price remained constant. This functionality has since been reintroduced at an approximate additional cost of $10 million.
12.121 We also noted that estimates of project cost do not include all costs associated with implementing the system. Estimates include only the costs of obtaining goods and services from suppliers or contractors, and not the substantial departmental resources expended to develop and implement the system.
12.122 Approval process. We are concerned that failing to clearly define the scope and systems requirements for the projects that we reviewed has had implications for the approval process. From the above-noted observations, it is apparent that the inherent nature of large, multi-year projects does not allow project sponsors to predict with confidence what the ultimate cost of the project will be.
12.123 A similar concern was expressed by Treasury Board ministers at the time of the second Treasury Board submission for PSCS when estimated project cost had risen by $35 million.
12.124 Within the scope of our audit, we wanted to assess the nature and extent of Treasury Board Secretariat analysis of the business case submitted by departments in support of their project initiative. Because of the strategic importance of information technology in reducing costs and improving services, we had expected that the Secretariat's analysis of departmental submissions would include a thorough analysis of full project costs, projected savings and assessment of the risks associated with the project.
12.125 However, in our view, the analysis was not sufficient for the purposes of providing a recommendation or advice to Treasury Board ministers as to whether a project should be approved and funded at the level requested.
12.126 The analysis of the submission for PSCS illustrates this weakness. In this instance, the Secretariat did review and comment on the submission and asked the Department, PWGSC, for clarification on the draft submission. However, we found little evidence of extensive analysis of estimated project costs to assess the reasonableness of the requested funds or analysis of projected savings to assess their reasonableness.
12.127 Finally, the analysis done by departments and by Treasury Board Secretariat should be sufficient to provide as complete information as possible on which to base the system investment decision. This analysis should provide assurance that:
12.129 Key environmental factors include:
12.131 Changes to the business environment have affected Transport Canada's Integrated Departmental Financial and Materiel Management System (IDFS). When the Department initiated the system in 1990, it employed 20,000 people and included airport, aviation and coast guard operations. The size and operations of Transport Canada in 1995 and beyond differ greatly from what they were when the project started. Today, Coast Guard operations have been transferred to another department, air navigation services are proposed to be transferred to a not-for-profit organization outside government, airports are being privatized and projections reflect a Department of 3,000 to 5,000 employees. All these changes equate to a very different business environment than when work on IDFS began.
12.132 As a consequence of the many factors that have changed the operating environment of Transport Canada, the Department had significantly more licences for its commercial software than would be needed for its new operating environment. To mitigate the impact of the changing environment, the Department, in conjunction with PWGSC, has negotiated a transfer of surplus licences to four other departments, one of which is implementing IDFS.
12.133 In the case of the Income Security Program Redesign (ISPR), some of the basic assumptions that governed the decision to build the system have changed. Initially, it was assumed that the system would administer the three elements of the social security framework: Family Allowance, Old Age Security and the Canada Pension Plan. Since the project was initiated, the Family Allowance program has been replaced by the Child Tax Benefit program and is administered by Revenue Canada through the taxation system. The federal Budget of February 1995 indicated that the government would conduct a review of the public pension system and was making changes to the Old Age Security program. The results of this review may impact on the original assumptions under which ISPR began.
12.134 To mitigate the impact of a rapidly changing business environment, it is important that large systems development initiatives be broken down into smaller, more manageable components, with each component delivering an improved capability (efficiency or effectiveness). Under such an approach, if environmental changes affect systems development requirements, the government has still achieved a benefit and payback from the implementation of earlier components.
12.137 In our audit we have identified a number of risks that influence the successful implementation of systems development projects:
12.139 Many factors affect the successful development and implementation of systems. Among the most critical to the successful management of project risks are:
12.141 Such an approach provides certain key benefits:
12.143 We encourage and support the Treasury Board Secretariat's efforts to renew the current management framework for systems under development.
12.144 Future audits will monitor the progress of this initiative and will continue to examine issues that affect the successful implementation of systems development projects.
Public Works and Government Services Canada response: The Department shares the concerns of the Auditor General with respect to the risks of systems under development.
As part of its recent evaluation of Common Purpose Procurement (CPP), PWGSC conducted its own investigation into the difficulties experienced with large-scale information technology systems and how these might be overcome. The results of this work indicated that improvements could only, meaningfully, be realized through a co-operative effort between ourselves, Treasury Board Secretariat and, of course, the sponsoring departments for the projects involved.
This perspective was endorsed when our research was presented to the Treasury Board Subcommittee on Information Technology (TIMS) on April 28th of this year. At that time, a co-ordinated, interdepartmental Action Plan to address areas of major concern and from which improvement efforts are expected to result in the greatest benefits was agreed to.
Public Works and Government Services will continue to strengthen the skills our staff bring to the contract and risk management associated with information technology and systems integration (IT/SI) projects; examine the potential for providing project management expertise for such initiatives; consult with stakeholders regarding how to assess the achievement of value for money in IT/SI projects; and collaborate with Treasury Board Secretariat in examining issues concerning partnering arrangements.
Treasury Board Secretariat has undertaken to ensure the adequacy of policies, standards and tools for the management and delivery of IT/SI projects, including risk, project and contract management; to determine acceptable partnering relationships for IT/SI projects; and to collaborate with PWGSC and the broader community to resolve project management issues and to further develop the successor to CPP.
Sponsoring departments will be obliged to meet the requirements set out by Treasury Board, including the assurance that sound business cases and superior project and risk management skills are brought to major systems projects.
More specific to the systems reported in the OAG audit, the measures taken by the Department to reduce risks include:
1. Responsibility for design and construction of the system was contracted out to major systems integration firms (PSCS and CDFS).
2. Risks of unclear user requirements or overly optimistic construction estimates were mitigated by the use of competitive tenders, independent reviews of project size (PSCS and CDFS), as well as funding of the design phase to two systems integration firms (PSCS).
3. Detail reporting was required from the contractor on time and performance (PSCS).The fact that these were not successful, as expected, in reducing risks, highlights the need for the Crown to understand the industry track record as well as to manage the risk that contractors will fail to perform well on complex fixed price software development projects. With hindsight, the Crown placed excessive confidence on the industry's ability to deliver. Consequently, we agree with the Auditor General that organizing smaller projects would reduce risks.
Recent work on the CDFS product has in our view addressed any remaining concerns with the functionality of the system. Nevertheless, in light of the environment described by the Auditor General, the Department has decided to suspend active marketing of the system to other government departments and to focus its efforts on implementing CDFS within PWGSC and on supporting its committed clients. While this client base does provide adequate returns to the Crown for its investment in the system, more consistent support from the Treasury Board Secretariat would undoubtedly have helped to increase its return on the investment.
Finally, while the Department is convinced that the notion of "project sponsorship" is critical to the success of common shared financial and administration system development, current circumstances and the accountabilities of the central agencies, departments and the common service department, preclude the effective application of this concept.
In the future, the Department will continue to adhere to professional standards of project management, take a balanced approach to achieving cost reductions through a combination of organizational, procedural and technological change, and review the opportunities for smaller, and shorter duration projects.
Treasury Board Secretariat response: The Treasury Board Secretariat generally concurs with the findings of the Auditor General. These findings are consistent with a separate analysis that TBS has conducted. As both studies point out, technology is complex and ever-changing and its acquisition carries inherent risks. Nevertheless, information technology is a crucial part of the government's thrust to help improve quality service to the public while reducing the cost, and information acquisition projects must continue. While the risks cannot be eliminated, whether in the public or the private sector, we agree they can and must be reduced.
The Treasury Board Secretariat has begun the introduction of changes to the management framework for large information technology projects. These changes will potentially affect all aspects of the project environment, including project size, approvals, risk assessment, project management experience and training, procurement vehicles, change management, reporting and monitoring. We are confident that the improvements will significantly improve the success rate of government informatics projects. Treasury Board Secretariat will be monitoring high risk projects closely to ensure effective implementation of the new approach.
We thank the Auditor General's Office for its support and look forward to its continued involvement in this work.
Human Resources Development Canada response: The Department generally concurs with the findings of the Auditor General and shares the concerns expressed with regard to the risks inherent to systems development projects. The Department will continue to manage the risks associated with the ISP Redesign project, placing additional emphasis on those areas identified in the report. We would also endorse the observation that "large multi-year system development" projects are unlikely to be successful, unless the project is broken down into smaller, more manageable components with significant user involvement at all times.
Audit Team
Greg Boyd
Guy Dumas
Ira Greenblatt
Christine Kelly
Laurel Lowry
William Moeller
Gerry Montigny
Richard Quesnel
Hugh Rodgers
Marvin Schwartz
Bruce Sloan
Barbara Taylor
Peter Taylor
For further information, please contact Eric Anttila, the responsible auditor.