14 December 1999
David Rattray, FCGA
Assistant Auditor General
Mr. Chairman, I would like to thank you for your invitation to appear again before this Committee. With me today are Peter Kasurak, the Principal responsible for National Defence matters and Hugh McRoberts, the Principal responsible for the audit of Public Works and Government Services Canada. Last year we discussed our 1998 chapters on equipping and modernizing the Canadian Forces and the procurement of major capital projects. In response to these audits, and Parliament's continuing interest, the Department has begun to undertake reforms in both budgeting and management of capital.
Today, I would like to provide an overview of what the Department has achieved and bring you up to date on two of our more recent projects related to procurement: the Alternative Service Delivery audit at National Defence and the government-wide audit of sole-source contracting for professional services. I would also like to provide a brief overview of other audit projects we have completed such as our chapters on the management of hazardous materials and the proper conduct of public business.
First, let me touch on the Department's acquisition reforms. Our views are not based on audit work, but on less than formal on-going contact with the Department. We intend to do a full follow-up on capital acquisitions and report to Parliament in about a year.
The Public Accounts Committee held two hearings on Defence capital procurement and reported to the House of Commons in October 1998. The Standing Committee on Public Accounts called for DND to:
While the Government said it agreed with these recommendations, there is much to be done before reaching these objectives:
Overall, some progress has been made, but we hope to see more before we table our follow-up report in the Fall of 2000. Peter Kasurak will now take over the presentation.
Turning to our more recent work on Defence procurement, the audit of Alternative Service Delivery at National Defence repeated one of the themes of our audit of major capital projects: business cases and options analysis were often poorly done. In the capital project audit, five of the six projects we examined had not undertaken adequate options analysis. In the more recent ASD audit, five of 14 projects fell short. While this indicates that some improvement has occurred, there is still much to be done. We note in our report that the Department has already recognized this problem and has taken measures to address it, especially for very large projects such as the Supply Chain project.
From the results of our audit and the testimony before this Committee by members of the Department, we concluded that there is still a lack of experienced staff to undertake sophisticated business analysis demanded by complex ASD projects. The Department recognized this problem in 1997 and decided to concentrate on fewer projects. It responded to our audit by saying it will provide more in-service training to staff, but this problem will likely persist over the next three or four years.
This is also the case for the cost information necessary to make business decisions. Three of the 12 projects with completed business cases had inadequate baseline costing studies to support them. And there are also problems with establishing the service levels that underpin management's goals for in-house or contract services: service levels were established prior to the ASD project in only three of 14 service areas. Half the business cases we examined were deficient in this area.
Like training, both cost and performance data are being improved by the Department, but these deficiencies are likely to persist for two or three more years. Parliament can help by encouraging the Department to develop these three technical areas more quickly and by requesting the Department to provide measurable plans for accomplishing this.
The ASD audit also brought to light a significant problem regarding the contracting of major service contracts, like the $2.8 billion, 20-year NATO Flying Training in Canada project. Such a project differs from typical service contracts in two significant ways: they are very large, thus limiting the number of Canadian firms that are likely to be available to bid on them; and second, they create a "partnering" relationship with a contractor over a very long period of time during which other firms may be shut out of a major government market and during which the government may become highly dependent on an external source for a vital service. Nonetheless, they also offer very attractive possibilities for reducing costs and importing technology.
Our audit concluded that the NATO Flying Training in Canada Project had not followed government policies for sole-sourced contracts. In addition, we found that the profit markup awarded was not consistent with existing Public Works and Government Services Canada guidelines. In part, this is evidence of overall compliance problems that I will discuss later. But it is also an indication that the existing contracting framework for major service out-sourcing is not adequate. It does not provide adequate guidance to officials for addressing competition problems when there is a limited market. I believe that this is a vital component for future large ASD projects to ensure that value is received. It is also necessary to assure industry that all firms have an equal chance at competing for the government market. The existing framework also does not appear to provide sufficient guidance to officials for evaluating risk and analyzing and documenting risk transfers to the private sector. In the NFTC contract, there was a substantial transfer of risk and a significant award of profit markup because of it. Yet, we found that there were inadequate studies underlying important decisions.
The Treasury Board Secretariat and Public Works and Government Services Canada have both responded positively to our recommendation that these areas should be addressed by continuing work on procurement reform. Parliament could help by asking officials to clarify their philosophy regarding competition for these major projects and to provide a timetable for the achievement of their planned reforms.
Hugh McRoberts will now explain his work on sole-source contracting.
In our second procurement audit, the use of Advance Contract Award Notices or "ACANs" in sole-source contracts, we found significant problems that endanger value for money and other important principles. This audit was government-wide - we audited three other departments other than National Defence. We reviewed over 500 contracts let using ACANs and selected 50 for in-depth audit. ACANs are electronically-posted notices that are intended to inform suppliers that a sole-source contract is about to be let and allow a challenge to the decision.
Our sample contained 20 projects managed by National Defence. While our sample was not designed to draw a statistical conclusion about National Defence's track record vis-a-vis the other departments, National Defence's performance was not exceptional and this Committee may wish to consider this government-wide problem in its deliberations over procurement at National Defence.
We found that for these smaller service contracts the overall policy is essentially sound, but that either officials do not understand it or simply ignore it. Only 11 percent of the contracts we examined complied with government policy regarding the letting of sole-source contracts. Failure to compete government contracts is not only inequitable to the business community, but it also endangers value for money to the government.
We also found that almost half the contracts we audited had inadequate statements of work. In about one-third of the cases, the statement of work in the contract was substantially different from the way the work was described in the ACAN. Departmental managers could provide little evidence that they had received the best possible price or highest value to the government from these contracts.
Other than general non-compliance and poor business practices, our concern is that ACANs are becoming a "fifth exception" to regulations requiring competitive tendering. Officials appear to believe that, if an ACAN is posted, and no challenge is lodged, then the decision to sole-source is justified. As noted, this was not the case in nearly 90 percent of the cases we audited. We also found that challengers were not treated fairly. Challengers appeared to face a higher burden of proof than the suppliers whose selection was being appealed. There was a lack of independence in the process in that the same managers that had awarded the sole-source contracts also judged the appeals against their own decisions.
Our report makes several recommendations to the Treasury Board Secretariat on how to address these findings. The key points are the need to follow the rules and to enhance accountability for doing so, and to realize that sole-sourcing is a serious decision that deprives others of economic opportunities. There is also a need for some form of independent due process for those who wish to challenge these decisions. We are disappointed in the Secretariat's response. It flatly rejected our recommendation that sole-source decisions, which are based on claims that the services of a unique supplier are required, should be reviewed and approved by an independent senior manager. We are also discouraged by the Secretariat's refusal to support mandatory audits of a sample of contracts in departments that frequently use sole-sourcing. With all due respect to the Secretariat's concern for the use of audit resources, the tide of non-compliance is so high that it can no longer be ignored. We believe that measures must be taken now, and we do not believe that costs would outweigh benefits.
National Defence also responded by saying that in the last calendar year they had denied about one-fifth of their departmental managers' ACAN requests and had cut the number of ACANs by half. They said they had established a Contracting Advisory Committee to review "selected" sole-source decisions, analyze trends and advise managers.
Parliament can help by reaffirming its support for the rules and the principles of competition, fairness and best value that lie behind them. It can indicate that it views violations of these rules as serious and encourage the government to do the same. Specifically, there is a need for an ongoing assessment of contracting performance, a review function in departments that requires a senior official to take or concur in the decision to sole-source and to be accountable for it, and an independent dispute resolution mechanism to deal with contracts that lie outside the purview of the Canadian International Trade Tribunal.
Finally, I would like to mention the other audits we have completed since our last appearance. The hazardous materials management audit concluded that compliance with laws and policies designed to protect employees and the environment had not improved since the Department completed its own audit in 1993. Almost half the individuals at the 10 bases we audited who handled hazardous material had not received the appropriate training. In addition, although the Department says that it conforms to provincial and municipal standards "where appropriate", we found it had never defined what this meant. Air and water emissions from Canadian Forces Bases were not adequately monitored and often did not meet federal guidelines.
Our Proper Conduct of Public Business audit examined the framework supporting ethical conduct across the Department. We found that the Defence Ethics Program - intended to support good conduct - was well-designed, but not completely implemented. Compliance audit checking of financial transactions and of Base administration in general appeared to have fallen below prudent levels. Below the national headquarters level, compliance audit was not targeted with adequate risk analysis. Finally, the Department's follow-up on complaints of abuse made to National Defence headquarters was often weak. About half of the 66 cases we examined met all our criteria. Twelve of 66 cases were not adequately assessed, inadequate corrective action resulted about one-fifth of the time and 9 of 74 cases took three years or longer to resolve.
These chapters show that National Defence is capable of designing good management systems, but often does not follow through in the implementation. The pattern of repeatedly breaking rules, including contracting, financial administration, and health and safety regulations is troubling.
Mr. Chairman, that concludes our opening statement. We would now be pleased to answer the Committee's questions.