AUDIT REPORT

Management Framework of the Space Technologies Branch
PROJECT #03/04 01-01

Prepared by the
Audit, Evaluation and Review Directorate

September 2005

	Executive summary
	1.0 Introduction   1.1 Audit project rationale   1.2 Audit project objective   1.3 Scope   1.4 Methodology   1.5 The organization and its resources   1.6 Mandate and functions   1.7 Programming
	2.0 Operational planning   2.1 Linking activities to results   2.2 Nature and level of financial needs   2.3 Chart of accounts   2.4 Operating budgets
	3.0 Program delivery   3.1 Organization of work and delegated authority   3.2 Delivery and funding options   3.3 Internal control and compliance with policies and procedures   3.3.1 Certification of deliverables   3.3.2 Audits and price certification   3.3.3 Contract provisions   3.3.4 Roles and responsibilities   3.3.5 Accounting of financial transactions   3.4 Allocation of staff costs   3.5 Asset protection
	4.0 Accountability   4.1 Performance data
	Appendix A - Audit objectives and criteria
	Appendix B - Summary of operating costs
	AppendixC - Management action plan

EXECUTIVE SUMMARY

The objective of this audit project was to evaluate the extent to which elements of the Space Technologies Branch's management framework concerning governance, operations and information systems allow the Branch to fulfil its mandate, carry out its operations effectively, efficiently and economically while complying with requirements prescribed by acts, regulations and policies and protect and account for the use of resources.

In a context in which the focus is on results, we believe it is important that significant steps be taken to strengthen the management framework of the Space Technologies Branch. Many of these improvements concern fundamental elements of the Branch's management framework. Some measures will have to be implemented by the Branch itself, while others will fall to the appropriate corporate services sectors.

The planning process is aimed at ensuring that resources and staff's attention are directed at achieving the Agency's objectives. It is important that all activities and projects have a formal relationship linking them to the Agency's strategic outcomes. It is also just as important that staff be familiar with these relationships. The planning process is also conducive to the implementation of instruments for monitoring operations. Managers must be able to rely on a chart of accounts that is consistent with approved activities and projects and must translate resources approved under work plans into operating budgets that will guide them in carrying out their activities and projects. They must also comply with the government's budget structure by ensuring that the nature of their financial needs matches the nature of their operations.

In the area of program delivery, the Space Technologies Branch will have to pay more attention to aspects of management relating to effectiveness and efficiency, internal control and monitoring of operations. The delivery of certain programs should be reviewed to ensure that the mechanisms used to administer them are consistent with the nature of the operations. We would also highlight the importance of the roles and responsibilities of staff who manage contracts and agreements so that they set up appropriate legal frameworks that clearly lay out the obligations of the parties. The roles and responsibilities of staff must be discharged in a manner that meets the requirements of internal control. Moreover, given the nature of the Branch's operations, the allocation of human resources and the control of their use require the implementation of a time management system.

With regard to accountability, the final step in the management process, efforts must be undertaken to give all programs a performance measurement framework that provides periodic reports on performance in terms of the objectives pursued.

This internal audit was carried out in accordance with the Treasury Board Policy on Internal Audit and the IIA (Institute of Internal Auditors) Standards for the Professional Practice of Internal Auditing. In our professional opinion, the audit procedures followed and evidence gathered were sufficient and appropriate and support the accuracy of the conclusions in this report. The conclusions are based on a review of the situations in question using established audit criteria. The conclusions only apply to the entity examined.

DESCRIPTION OF THE MANDATE

1.0 INTRODUCTION

1.1 AUDIT PROJECT RATIONALE

This audit project is part of the 2003-2004 Audit Plan approved by the Audit Committee.

1.2 AUDIT PROJECT OBJECTIVE

The objective of this audit project is to evaluate the adequacy of the key elements of the management framework of the Space Technologies Branch (the "Branch"). The management framework allows the Branch to

• achieve the objectives under its mandate;
• carry out approved operations in an effective, efficient and economical manner;
• achieve the anticipated results established in key planning documents;
• comply with the applicable policies, regulations and guidelines set by the Canadian Space Agency (the "Agency") and central agencies; and
• account for how the resources allocated to the Agency are used.

Appendix A provides a more detailed description of the audit objectives and criteria used.

1.3 SCOPE

The audit project covered the systems and procedures that were in use during the 2003-2004 fiscal year and that are still relevant today. These systems and procedures touch on the following areas:

• operations planning;
• operations performances;
• performance monitoring and evaluation; and
• accountability.

The audit does not cover elements of the management framework relating to the administration of grants and contributions programs, as such activities were the subject of a recently conducted audit project, the results of which where published in an audit report in March 2005. Since the Canada/ESA Cooperation Agreement has currently been the subject of an audit and program evaluation, elements of the management framework specific to that program were also excluded.

1.4 METHODOLOGY

This audit engagement was carried out in accordance with audit standards set forth in the Treasury Board Secretariat (TBS) Policy on Internal Audit, which requires that audit objectives be set on the basis of audit criteria. Audit standards also require that the audit engagement be conducted in a structured manner, according to a process that includes

• a planning and preliminary review phase;
• an execution phase; and
• the reporting and disclosure of results.

Various audit procedures were used, including interviewing employees and reviewing and analysing documents, records and reports.

1.5 THE ORGANIZATION AND ITS RESOURCES

In 2003-2004, the Space Technologies Branch consisted of four directorates and one division, not including the Office of the Director General. Before 2003-2004, the Systems Engineering Directorate was part of the Space Programs Branch. As a result of the Agency-wide organizational review carried out in the 2002-2003 fiscal year, the Director General, Space Technologies now reports to the Vice-president, Space, Technology and Programs. The Space Technologies Branch was allocated a budget of approximately $114 million for its operations. TABLE 1 shows the resources allocated to each of the Branch's constituent entities.

TABLE 1 - RESOURCES ALLOCATED IN 2003-2004

1.6 MANDATE AND FUNCTIONS

The Space Technologies Branch has been given a mandate to act as the Agency's internal centre of technical expertise and to foster the development of space technologies so as to strengthen the competitiveness of Canadian industry and support Canada's space programs.

To fulfil its mandate, the Branch

• develops innovative and emerging technologies;
• helps industry to maintain its position at the forefront of international markets;
• manages commercialization and intellectual property issues;
• develops and maintains a pool of technical expertise; and
• provides technical support for Agency projects and programs.

1.7 PROGRAMMING

To carry out its mandate, the Space Technologies Branch has allocated resources to a vast array of activities and projects. TABLE 2 shows the resources allocated and used in 2003-2004 to carry out its activities and projects, broken down by program elements under the Branch's responsibility.

TABLE 2 - RESOURCES ALLOCATED IN 2003-2004, BY PROGRAM ELEMENT

A detailed summary of operating costs by program element can be found in Appendix B of this report.

AUDIT RESULTS

2.0 OPERATIONAL PLANNING

The planning process is carried out on several different levels. Strategic planning involves identifying and predicting courses of events that will affect the achievement of Agency objectives while taking into account organizational needs, changes in policy and external factors. The President, Strategic Development Directorate is responsible for this level of planning. Important events (the Auditor General's December 2002 report on the Agency and the revision of the Planning, Report and Accountability Structure [PRAS] currently under way) have made it necessary to conduct an in?depth review of strategic planning procedures, both at the level of the Agency's strategy itself and in respect of stakeholders and instruments.

Operational planning is concerned with turning strategic plans into detailed multi-year plans by selecting programs and services according to their contribution to achieving set objectives. Operational planning is the responsibility of the Program Review Advisory Board (PRAB), in line with the Management Framework for the Canadian Space Program. Programs and services under the responsibility of the Space Technologies Branch were approved before 1999, that is, before the adoption of the current Space Program, which incorporated and renewed those services and programs.

Managers from all levels of management are responsible for taking part in annual planning aimed at identifying activities and projects to be implemented in the coming fiscal year and justifying the resources to be allocated to them. Managers are also responsible for turning spending authorities into operating budgets to guide them in carrying out their activities and projects. We focussed on current activities related to annual planning in the Space Technologies Branch in order to evaluate the extent to which

• Branch activities and projects follow a structured planning process;
• operations receive the required approval;
• the nature and level of resources allocated are appropriate;
• the planning process contributes to accountability; and
• budget control is based on the planning process.

2.1 LINKING ACTIVITIES TO RESULTS

The government's spending management system is based on the concepts of "programs" and "strategic outcomes". They are the raison d'être of departments' activities and form the basis of the votes allocated to them. The government's Estimates, particularly the reports on plans and priorities and departmental reports, rely on these concepts to justify the rationale for allocating resources to departments' and agencies' operations.

To associate programs more closely with results, the Treasury Board Secretariat (TBS) revised its policy on the Planning, Reporting and Accountability Structure (PRAS). At the request of TBS, the Agency began a transition to a new Program Activity Architecture (PAA) that describes expected strategic outcomes, sets out which program activities are to be given priority and outlines how Agency activities and projects relate and contribute to those outcomes. The PRAS/PAA serves as a reference for decision making at all planning levels, be it strategic planning (directions), operational planning (range of programs) or annual planning (selection and implementation of activities and projects).

Staff of the Corporate Management Directorate have made considerable efforts to link programs with strategic outcomes. For example, resource planning and allocation documentation used for the annual work plan drafting exercise were revised in order to incorporate strategic outcomes. The financial coding structure was also reviewed to ensure that classification by "object" gives an accounting of resources used in relation to results.

The Space Technologies Branch is also taking steps in this direction. Program management manuals were recently developed for a number of important programs the Branch is responsible for (STDP, EOADP and GRIP [Government Related Initiatives Program]). The manuals describe program objectives and set out the main aspects of their respective management frameworks. More specifically, they establish a results-based approach to program management and outline performance measurement frameworks. The managers' guide to the STDP states that management of that program is based on the Results-based Management Accountability Framework (RMAF). It even includes a logic model depicting the relationships between activities and outcomes. Despite the increased importance ascribed to results-based management, we were surprised to find that staff members at every level generally have a poor understanding of the structure of Agency programs. Many have never even heard of the PRAS/PAA and/or are not aware of the relationship between the activities and projects they are responsible for or assigned to and the Agency's final outcomes (strategic outcomes).

This situation can be explained by the long time that has elapsed since certain program elements were approved and implemented, the year-after-year repetition of the same activities and staff turnover. Staff will have difficulty gaining a perspective of how activities and projects contribute to strategic outcomes if the relationships linking them have never been set out and/or formally communicated. Losing sight of final outcomes could even affect decision making and the execution of activities and projects

The Corporate Management Directorate is responsible for coordinating the implementation of the PRAS/PAA and ensuring that the relevant elements of the Agency's management framework are adapted to it. However, it is up to the operational sectors to develop programs, deliver them and evaluate their performance in line with the approved program structure.

RECOMMENDATIONS

SPACE TECHNOLOGIES BRANCH

• i) Ensure that all activities and projects under the Branch's responsibility are linked to the Agency's strategic outcomes. Developing logic models patterned after those established for results-based management accountability frameworks (RMAFs) would be a good way of establishing and communicating this linkage.
• ii) Ensure that staff are familiar with the relationship that links their activities and projects to strategic outcomes.

2.2 NATURE AND LEVEL OF FINANCIAL NEEDS

The government's budget process is designed to compel departments and agencies to establish their financial needs so that votes can be approved by Parliament. The structure of the Estimates is such that it allows departments and agencies to make use of several different types of votes, depending on what they are to be used for:

• program expenditures votes;
• operating expenditures votes;
• capital expenditures votes;
• grants and contributions votes;
• non-budgetary votes; and
• special appropriations.

When an appropriation act is passed, the wording of each vote and the spending authority associated with it establish the terms and conditions under which these expenditures will be made.

The TBS coordinates government-wide budget planning activities leading to the preparation of appropriation acts; within the Agency, this responsibility is handled by the Corporate Management Directorate. However, it is up to the managers of the operational sectors to identify the appropriate nature and level of resources needed to carry out activities and projects undertaken in order to deliver the program elements under their responsibility.

The Agency relies on several different types of votes to fulfil its mandate. TABLE 3 shows the votes allocated to and used by the Space Technologies Branch in the 2003-2004 fiscal year.

TABLE 4 provides additional details by presenting a breakdown of the use of parliamentary votes by the program elements to which they were allocated. We found that capital votes were largely allocated to program elements that would normally have required operating expenditure votes. A review of a sample of expenditures (see Section 3.3) confirmed that the votes allocated and used were not consistent with the nature of the activities and projects carried out. Thus, nearly $30 million in capital votes were used to cover the costs of feasibility studies (Projects in Phase O/A) and research and development activities related to space technologies and Earth observation applications.

TABLE 4 - USE OF VOTES, 2003-2004

This situation not only fails to comply with the applicable requirements and compromises the quality of financial data in the Agency's and the federal government's financial statements, it also leads to complications in terms of monitoring and controlling spending authorities. Even if the rationale for votes is not always consistent with their actual use, every effort is made to control them so that the authorities received are not exceeded. There are, however, circumstances that require special monitoring and control measures, such as when different types of votes are allocated to the same initiative. Such situations occur when, in the course of the fiscal year, surpluses arise and are reallocated to ongoing activities, or when resources are allocated to new initiatives and available votes are not in keeping with the nature of these new initiatives.

We have already reported in a previous audit report (Financial Management Information - December 2003) that there are inconsistencies between the parliamentary votes requested and the uses to which they are put. Even though we had not yet followed up on the implementation of the Management Action Plan at the time this audit report was being drafted, we noted that the Corporate Management Directorate had already taken some steps to revise practices.

Nevertheless, we would point out that, with respect to planning, it is up to managers who have been given operational responsibilities to establish the nature and level of funding required for the activities and projects they intend to carry out and to record them in the appropriate documents required by decision-making authorities (PRAB, Executive Committee, TBS). When undertaking financial planning exercises (ARLU, Estimates, Supplementary Estimates, work plans, etc), managers must ensure that instruments reflect the nature and level of resources required. They must therefore have a thorough grasp of the fundamental principles of financial management as set out in the policies of central agencies and in the Agency's own policies and procedures. Staff in the Corporate Management Directorate must not take the place of the managers to whom these responsibilities are assigned; rather, they should be providing assistance and coordination.

RECOMMENDATIONS

SPACE TECHNOLOGIES BRANCH

• i) As part of approval and planning exercises, managers should ensure that the nature of their financial needs matches that of proposed expenditures, in line with applicable rules.

2.3 CHART OF ACCOUNTS

As part of the planning process, the appropriate decision-making authorities are expected to approve programs and services, authorize activities and projects and allocate resources to them. Another expectation is that a chart of accounts setting out an adequate coding structure will be used to provide managers at all levels with the information they need to manage their operations and assume their responsibilities.

The 2003-2004 Work Plan for the Space Technologies Branch gives an overview of all the activities and projects deployed to address approved program elements. We were expecting to find a coding structure modelled on the authorized activities and projects to which human and financial resources have been allocated. We found that the chart of accounts in use was poorly suited to monitoring the finances of several of those authorized activities and projects. Each activity and project described in TABLE 5 did not have a distinct, appropriate coding that would allow the corporate financial system to exercise proper budgetary control over those activities and projects.

Without an appropriate coding structure, it is impossible to account for and total up operating costs, control the use of resources or give an accounting of results versus authorities obtained. For example, when we read the monthly Financial Management Report (FMR), we found that there was no mention of the financial situations for each activity and project described in TABLE 5. Such information had instead been amalgamated and presented under other headings.

Sound management principles require that managers with operational responsibility for a financial delegation must be given means that are commensurate with their responsibilities. Since the Agency's information management systems are based on the chart of accounts, there is no disagreement on the necessity of ensuring that the chart of accounts meets expectations in this regard.

Naturally, the resource approval and allocation processes must themselves include special procedures for identifying the relevant elements of the chart of accounts at the outset to ensure that expenditures are accounted for only in respect of approved operations. We have already pointed out that efforts have been made to make work plans more effective. For example, provisions have been made to link activities and projects in the 2004-2005 Work Plan with the Integrated Planning System (IPS). We believe that similar provisions should be made to link approved activities and projects with the chart of accounts. Thus, project approval documents (PADs) and submissions to the Executive Committee or the Program Review Advisory Board should also include provisions relating to accounting.

RECOMMENDATIONS

SPACE TECHNOLOGIES BRANCH

• i) Ensure that managers are familiar with the coding structure associated with their operations.

CORPORATE MANAGEMENT DIRECTORATE

• ii) Ensure that the instruments used in resource approval and allocation processes include provisions aimed at identifying the relevant elements of the chart of accounts.
• iii) Ensure that the chart of accounts is consistent with approved activities and projects.
• iv) Ensure that corporate systems permit real financial monitoring of Agency activities and projects.

2.4 OPERATING BUDGETS

An effective planning process should normally include operating budgets that put work plans into detailed financial terms. Work plans are actually management agreements that contain the objectives of responsible managers, expected outcomes and the levels of resources allocated to the activities and projects they plan to carry out. Work plans are management documents (for resource planning and allocation), as opposed to financial documents.

Currently, once work plans have been approved by the appropriate authorities, staff from Financial Services then distribute spending authorities and record them in greater or less detail in the corporate financial system. With an accounting of commitments and expenditures, the corporate financial system can produce a variety of financial reports. Given the weaknesses highlighted by the internal audit report Financial Management Information - December 2003, the information at managers' disposal allows them to conduct limited financial monitoring of their activities and projects.

Moreover, after meeting with managers, we found that they attached little importance to preparing and using operating budgets. Some claimed that the nature or scope of their operations was such that there was no point in using sophisticated management tools. This explains in large part why many managers never consult the financial reports sent to them on a monthly basis by staff from Financial Services, preferring instead to rely on their own monitoring systems, which offer a better reading of their financial situation. For many, work plans are the final step in the planning process.

However, our analyses and discussions showed that some managers have an erroneous perception of the state of their operations and the relative magnitude of resources they assign to those operations. For example, some were under the impression that they spent about 10% of their budget on program administration when they actually were spending closer to 16% or, as in one case, as much as 32%, or $2 million instead of $650,000. Furthermore, other analyses showed that expenditures on the books for certain activities were not representative of actual costs, because certain expenditures were accounted for incorrectly. Without operating budgets, variance analyses or forecasts, it becomes difficult to account for results compared with resources used (see Section 3.4).

The operating budget is a spending plan. It is used to identify planned expenditures and determine at exactly what time payouts will be made. These considerations will guide the actions to be taken and decisions to be made so as to ensure that all terms and conditions are met, thereby making it possible to carry out the planned expenditures. Many activities carried out by the Branch pose management challenges in this regard. For example, several program elements are delivered through contracts awarded to businesses and organizations through calls for tenders. This approach involves multiple stakeholders, and procedures for drafting calls for tenders, evaluating bids and drawing up contracts require a great deal of time and effort. Moreover, when the work is being carried out, businesses and organizations frequently cannot meet the scheduled deadlines which are often set as a precondition of payment. Such situations necessarily require that action be taken to manage cash flow as effectively as possible.

Managers responsible for program delivery must be very rigorous in their administration of the resources entrusted to them. Operating budgets are tools that allow them to discharge their financial management responsibilities while keeping more senior levels of management informed of developments in situations under their responsibility.

RECOMMENDATIONS

SPACE TECHNOLOGIES BRANCH

• i) Make managers aware of the advantages of using operating budgets as a tool for managing their operations.

CORPORATE MANAGEMENT DIRECTORATE

• ii) Help managers take full charge of their responsibilities by furnishing them with the appropriate management tools.

3.0 PROGRAM DELIVERY

The planning process is aimed at identifying and selecting the activities and projects that are most likely to lead to the achievement of an organization's objectives and allocating the resources needed to carry them out. It is then up to managers to implement the appropriate means for ensuring that their operations are performed and controlling the spending authorities they receive. The following measures should be considered:

• organization of work, distribution of responsibilities and delegated authorities;
• direction, support and means provided to staff;
• controls for ensuring compliance with acts, policies and regulations;
• management controls for measuring achievements with respect to objectives; and
• asset protection.

3.1 ORGANIZATION OF WORK AND DELEGATED AUTHORITIES

The Space Technologies Branch is responsible for a large number of programs and services and an even greater number of activities and projects associated with them. Effective administration of program elements under the Branch's responsibility requires the following:

• an organizational structure that reflects an effective organization of work;
• well-defined responsibilities for activities and projects;
• clearly defined chains of command, set out in an up-to-date organization chart;
• relevant task descriptions; and
• delegations of authority that are consistent with the responsibilities delegated.

Generally speaking, the organization of work, the sharing of responsibilities and the delegation of powers currently in place contribute to allowing the Space Technologies Branch to effectively administer the program elements under its responsibility.

In the 2003-2004 fiscal year, program elements were divided among the Office of the Director General and the Branch's four directorates and one division. The responsibilities of each of these organizations with respect to program elements were clearly assigned. Spending authorities and financial delegations were commensurate with their responsibilities.

3.2 DELIVERY AND FUNDING OPTIONS

The Agency has been given a mission to use the powers, duties and functions attributed to it under its enabling act to establish a set of programs and services allowing it to fulfil its responsibilities. Depending on the objectives pursued and anticipated outcomes, program delivery may be assigned to government bodies, third parties or a partnership of the two. The approach chosen will determine which funding options and administrative mechanisms are used to deliver programs and services.

Delivery and financing options for programs under the responsibility of the Space Technologies Branch were established upon their approval, which in many cases came before 1999, the year when they were renewed with the adoption of the current Space Program. Since then, submissions to the Program Review Advisory Board regarding these options have been essentially aimed at obtaining approval for financing or for specific projects under program elements that have already been approved. In this context, despite the fact that the submissions include a section in which the procurement strategy is to be outlined (contract, memorandum of understanding, grants and contributions), delivery options did not appear to have been questioned. At any rate, we did not attempt to determine if accepted delivery options were the most effective ones. Instead, we concentrated on evaluating the extent to which the funding options and administrative mechanisms used were consistent with accepted delivery options.

TABLE 7 summarizes the Space Technologies Branch's program delivery approach.

TABLE 7 - DELIVERY AND FUNDING OPTIONS

We found that several program elements delivered by third parties (STDP, EOADP and TDP) were based on service contracts (TABLE 8). Our analyses and discussions revealed that some of the characteristics of these contracts made them more like financial assistance to third parties than true procurements of goods and services. In the case of activities and projects under GRIP, other departments failed to report back to us about the delivery and financing options used.

TABLE 8 - FUNDING OPTIONS AND ADMINISTRATIVE INSTRUMENTS

It is highly recommended that transfer payments be used when

• the department's mandate permits it;
• the government is not receiving goods or services in exchange for payment;
• third parties are in a better position than the government to provide specific services or carry out a particular task;
• transfer payments are the most effective means of achieving departmental program objectives; or
• the objective of the departmental program is to promote economic development.

In administrative terms, transfer payments and service contracts are subject to different management frameworks. The Policy on Transfer Payments sets out requirements in that regard, while service contracts are subject to the Government Contracts Regulations, trade agreements and the Contracting Policy.

Furthermore, our discussions with staff responsible for the day-to-day administration of the EOADP, STDP and TDP revealed that they had a poor understanding of the Policy on Transfer Payments.

RECOMMENDATIONS

SPACE TECHNOLOGIES BRANCH

• i) Review delivery of the EOADP, STDP and TDP to ensure that the administrative mechanisms used are consistent with the nature of operations, such that they are subject to the appropriate management framework.
• ii) Ensure that programs are financed using the appropriate parliamentary votes.
• iii) Ensure that staff are familiar with the Policy on Transfer Payments.

3.3 INTERNAL CONTROL AND COMPLIANCE WITH POLICIES AND PROCEDURES

Managers are responsible for administering human, physical and financial resources to make sure that public funds are used for authorized purposes. They must carry out their duties in compliance with the applicable acts, policies and regulations in force and implement internal control mechanisms and management practices to ensure that

• expenditures are approved by the appropriate authorities;
• expenditures are made through the appropriate procurement instruments;
• payments are contingent on certifications to the effect that the goods/services have been delivered or the work has been carried out;
• expenditures are accounted for under the appropriate accounts; and
• spending authorities are controlled.

The nature of activities and projects carried out is such that the Space Technologies Branch generates a very large number of individual transactions. We analysed expenditures to evaluate the extent to which current practices contribute to the sound management of public funds. Our review looked at a sample of payouts relating to the main program elements delivered by the Branch's four directorates and one division.

We found several situations that require action to bring operations under more rigorous control.

3.3.1 CERTIFICATION OF DELIVERABLES

A fundamental part of internal control is certifying that goods have been received, services have been rendered or work has been carried out as agreed in the contract documents. The importance of this element of internal control led Parliament to enshrine it in the Financial Administration Act (FAA). We refer to the FAA regularly, particularly Section 34.

Our review revealed that several payments were made on the strength of certifications issued by the appropriate authorities even though the goods described in the contract documents were never actually received. All such payments were related to personal service contracts for program management services. The deliverables in question were monthly work progress reports that were supposed to accompany claims for progress payments.

Moreover, this type of contract is similar in many respects to an employer-employee relationship. This appearance was reinforced by the fact that the manager responsible did not feel it necessary to require work progress reports to evaluate the nature and scope of work carried out.

RECOMMENDATIONS

SPACE TECHNOLOGIES BRANCH

• i) Those with authority to confirm contract performance and costs should ensure that the expected deliverables have been received before requesting that they be paid for.
• ii) Service contracts that could be interpreted as creating an employer-employee relationship should be avoided.

3.3.2 AUDITS AND PRICE CERTIFICATION

The certification required under s. 34 of the FAA also applies to the price asked by the supplier. In this regard, we noted that, even though certifications were issued, several payments had been made on a basis other than those set out in the contracts. We found that, in some cases, holdbacks were applied against claims when they were not required, and the reimbursement rates and/or eligible costs used to calculate payments were different from the prescribed ones. Our discussions with staff revealed that the basis of payment for contracts did not necessarily reflect what the Agency had agreed to pay (see Section 3.3.3 on this subject).

The terms and conditions of payment agreed upon by the parties are spelled out in the contracts and thus serve as the reference for establishing payments to suppliers. All stakeholders in the administrative process leading to duly executed certification must rely on these terms and conditions. If these terms and conditions are no longer representative of the situation, they must be changed by amending the contract.

Our discussions also revealed that stakeholders had a poor understanding of certain contract provisions, particularly those limiting payments to 90% of the ceiling price.

Moreover, we noted that the nature and scope of the audits carried out by staff of Sector Financial Services were not adequately documented. Claims should include a more formal certification to this effect to make work easier for those with authority to issue certifications under s. 34 of the FAA and to establish an acceptable audit trail.

RECOMMENDATIONS

SPACE TECHNOLOGIES BRANCH
CORPORATE MANAGEMENT DIRECTORATE

• i) Ensure that staff involved in the price certification process confirm prices in accordance with the relevant contract provisions.
• ii) The nature and scope of audits carried out should be adequately documented.

3.3.3 CONTRACT PROVISIONS

An analysis of payouts led us to look at a number of contracts and agreements to corroborate the audits and certifications carried out. In this regard, contract provisions setting out the basis of payment and the method of payment are particularly important. Some of the terms and conditions of payment we looked at had special characteristics that complicate the administration of contracts and agreements.

Earth Observation Applications Development Program - EOADP

The Earth Observation Applications Development Program is essentially delivered through contracts. The Program is geared to private businesses, but participating businesses are encouraged to form partnerships with end users. The Agency's participation in projects takes into account the level of maturity of EO applications, contractors' contributions and the contributions of potential end users. The program is also designed to allow contractors to make contributions of goods or services (in-kind contributions).

We found that the bases of payment agreed upon under this type of contract were complicated and difficult to administer. In some cases, they did not accurately describe what the Agency agreed to pay, which according to program representatives obliged them to establish amounts payable using different bases from one claim to the next. Furthermore, staff responsible for managing these contracts did not always systematically apply the conditions provided for, especially with respect to contractors' submissions relating to total project costs, which form the basis for calculating the Agency's contribution.

We also noted that, on one occasion, the contract included contradictory clauses regarding the application of holdbacks to the contractor's monthly claims.

Technology Diffusion Program - TDP

Like the EOADP, the Technology Diffusion Program is delivered though contracts. It allows contractors to acquire specialized services aimed at expanding space technologies to non-space applications. Again, the program is set up such that the private sector is required to contribute, but this contribution is established in accordance with the number of employees a business has. The Agency's contribution is not to exceed $50,000.

The bases of payment for this type of contract do not allow the Agency to exercise adequate control over payments. Eligible costs are poorly defined and do not permit adequate checks of contractors' contributions.

Earth observation - Other federal departments

Significant payouts are made to other federal departments, especially under GRIP, an Earth observation program delivered in cooperation with other federal departments. Essentially, the agreements used (Official Memoranda of Agreement, or MOAs) consist of a one-page document, plus an appendix presenting the description of work, which normally contains a project description drawn up by the other department. With regard to the financial provisions, the agreements state the amount of the Agency's contribution for a given fiscal year, which in most cases, depending on the statement of work, is presented as an advance. The agreements used under this program do not state the rights and obligation of the parties, which would be especially important in respect of deliverables, allowing the Agency to evaluate the outcomes of and rationale for its contribution. Furthermore, as mentioned in Section 3.3.5, the agreements do not address the parties' obligations with respect to accounting for payouts.

When the Agency and one or more other parties agree to work together toward a common goal, there should be

• a clear definition of the parties' powers and responsibilities;
• joint investment of resources;
• a sharing of risk; and
• mutual or complementary advantages.

Given the absolute necessity of holding parties to account, the Agency has consequently chosen to establish a policy and procedures regarding the drafting of cooperation agreements.

RECOMMENDATIONS

SPACE TECHNOLOGIES BRANCH
CORPORATE MANAGEMENT DIRECTORATE - PROCUREMENT AND CONTRACT ADMINISTRATION DIVISION

• i) Ensure that contractual agreements include appropriate provisions that accurately reflect the obligations of each of the parties and that contribute to the effective administration of agreements, particularly with respect to the basis of payment and the method of payment.

SPACE TECHNOLOGIES BRANCH

• ii) Ensure that agreements with other federal departments and agencies are made official through properly drafted written agreements that are in line with the principles set out in the Agency's policy on this subject.

3.3.4 ROLES AND RESPONSIBILITIES

The administrative process for issuing official certification under s. 34 of the FAA may require action on the part of several individuals, depending on the nature and complexity of the statement of work and the terms and conditions of payment agreed upon. The roles and responsibilities of the contracting authority, scientific/technical authority and program authority are normally described in the contract. When the Agency acts as contracting authority, staff of the Sector Financial Operations must check to see if claims comply with the terms and conditions of payment. Heavy workloads and complex contracts often require the program authority to call upon additional staff to assist in the day-to-day management of contracts.

In the course of our analysis, we were able to determine that certifications and audits were carried out by the appropriate authorities. However, we noted that, in some cases, they were not issued in a logical order that safeguards the integrity of the process. Under the process currently applied to the EOADP, claims are checked by Sector Financial Operations staff only after the program authority has already issued certification. Those responsible for the EOADP explained that this process was designed to spare Financial Operations staff from having to perform unnecessary work, as they very often had to amend the amounts claimed on bases different from those stated in the contracts. Moreover, Sector Financial Operations had not asked them to comply with the current process. However, the principles of internal control specifically require that administrative processes be designed to avoid such practices.

Stakeholders in the certification process must take action in their area of expertise to assist the manager, who ultimately exercises the delegated power to certify that work has been performed, pursuant to s. 34 of the FAA.

The process must be designed to meet operational needs in terms of effectiveness and efficiency so that payments are made to suppliers in a timely manner consistent with business standards. The process must also incorporate internal control mechanisms to guarantee the appropriate use of public funds.

RECOMMENDATIONS

SPACE TECHNOLOGIES BRANCH
CORPORATE MANAGEMENT DIRECTORATE

• i) Design an administrative process for issuing duly executed certification under s. 34 of the FAA that is effective and incorporates the principles of internal control.
• ii) Ensure that the staff involved understands the nature and scope of the responsibilities delegated to them.

3.3.5 ACCOUNTING OF FINANCIAL TRANSACTIONS

We place a great deal of importance on the accounting of financial transactions, because it forms the foundation of the Agency's information management systems. We believe that all staff should adopt this same outlook when issuing payments to suppliers and reconciling financial transactions with their operating budgets.

Our analysis of the accounting treatment applied to a sampling of expenditures revealed discrepancies between the votes allocated and the nature of activities and projects carried out (see Section 2.2 on this subject), particularly with respect to classification by authority.

With regard to classification by object, which serves to identify the nature of expenditures, we found errors had been made in several respects in the accounting of several payments made to other federal departments. The payments analysed relate to the procurement of services or to Agency contributions to joint activities and projects. Some purchases of services were accounted for under the account 1470 - Scientific consultants/Earth and space sciences (in the General Ledger, or G/L) even though they concerned other types of services that should have been accounted for under different accounts in the General Ledger.

Payments made as contributions to joint activities and projects (which total several million dollars) were also accounted for under 1470 - Scientific consultants/Earth and space sciences. The accounting of transactions between federal department that contribute to the achievement of joint objectives does not comply with prescribed practices. Essentially, the Agency must first account for the payouts as advances and then, to justify expenditures incurred by other departments, account for these expenditures in the Agency's own financial statements under the appropriate account according to the nature of those expenditures. In general, according to prescribed practices, payouts are supposed to be charged against the credits for which they were allocated, and financial data must be accurate on a government-wide basis.

Upon reading the MOAs under which these payments were made, we could find no specific provisions setting out the parties' responsibilities in terms of accounting.

RECOMMENDATIONS

SPACE TECHNOLOGIES BRANCH
CORPORATE MANAGEMENT DIRECTORATE

• i) Ensure that payments to other federal departments and agencies are subject to the appropriate practices.
• ii) Ensure that expenditures incurred by the Agency or other federal departments and agencies are accounted for under the appropriate accounts according to the nature of those expenditures.

3.4 ALLOCATION OF STAFF COSTS

The annual process for drafting work plans is aimed at approving resources and allocating them to activities and projects that are planned to be carried out in the coming fiscal year. Sound management practices require managers to continuously monitor the status of each activity and project under their responsibility. They rely on periodic procedures and systems that allow them to evaluate performance and the actual use of resources compared with estimates.

Work plans for the 2003-2004 fiscal year for the Space Technologies Branch set out 73 specific activities and projects that require attention from management. We have already reported certain discrepancies in accounting (Section 2.3) and a lack of real operating budgets (Section 2.4), which together compromise the monitoring of operations. Another important condition to be met in order to ensure effective monitoring of operations is the recording of expenditures incurred for the activities and projects to which they relate. In this regard, the allocation of labour costs is an indispensable exercise.

Currently, the Salary Management System (SMS) is used to allocate labour costs. The SMS is adequate for situations in which a staff member is assigned to a few activities on an ongoing basis and actual costs vary little from the projected ones. However, we noted that the allocations did not always reflect reality, thereby compromising the quality of financial data. For example, costs of the IP and Commercialization program element, as shown in Table 9, are overvalued to the detriment of those for the EOADP, given that the salary costs associated with the Manager, Commercialization and the Administrative Assistant are charged to IP and Commercialization activities, even though they spend most of their time on EO program activities. This highlights the importance of regularly revising allocations recorded in the SMS to ensure that they are still appropriate.

The SMS is not a time-management system. Consequently, its limits in terms of allocation are reached when staff members are asked to take part in multiple activities and projects, such as is the case with the STRP and matrix and technical support activities. To help meet these needs, an Excel-based time management system was developed. In the beginning, the system was designed to plan the allocation of staff (Agency employees, students, post-doctoral researchers and contractors) and the expenditures required to carry out projects under the STRP. The system was also designed so that staff could record the time spent on these projects, allowing the actual costs to be established and compared with estimates.

The system is still used for STRP projects and matrix and technical support activities. It appears to be effective at meeting financial and human resource planning needs. However, in our view, the time card system used by staff, which allows time to be reported by activity or project, is unreliable and ineffective. There is no assurance that all relevant data have been entered and are kept up to date. Furthermore, this in-house system has no way of interfacing with corporate systems (SMS and SAP) so as to establish labour costs and record actual versus approved activity/project expenditures in a way that makes effective financial monitoring possible.

In light of the weaknesses in accounting and the lack of an effective time management system, it is currently impossible to effectively administer the allocation and use of staff and establish the operating costs of each approved activity/project. Since the Agency has adopted a matrix management approach, and given that the Space Technologies Branch is an important centre of expertise, it is essential that managers have an effective time management tool at their disposal.

RECOMMENDATIONS

SPACE TECHNOLOGIES BRANCH

• i) Make managers aware of the importance of ensuring that staff assignments recorded in corporate systems reflect the activities and projects to which staff are actually assigned.
• ii) Implement a time management system that permits effective management of the allocation and use of human resources.

3.5 ASSET PROTECTION

In support of program and service delivery, the Space Technologies Branch acquires materiel and equipment and places it at the disposal of its staff. Managers' responsibilities do not end with the evaluation of the relevance of an acquisition. They are responsible for ensuring that physical resources are kept and used for authorized purposes only throughout the life cycle of those resources. Managers must ensure that physical resources are inventoried in corporate systems and account for them periodically.

According to information extracted from the corporate system for managing physical resources (the Automated Materiel Management Information System, or AMMIS), the Space Technologies Branch is responsible for more than 3,000 items representing a total acquisition cost of $17.4 million. This inventory includes a wide range of items, such as computers, furniture, office equipment and workshop/laboratory equipment. These items, whose individual values can range from a few hundred dollars to $2 million, are mostly found in Agency work places but can also be found outside the organization.

When data are entered into the AMMIS, responsibility is established by associating each item with a responsibility centre. We found that responsibilities were not identified correctly. Moreover, we had no assurances that the data were up-to-date. There are no periodic, systematic procedures for validating the integrity and accuracy of data entered into the AMMIS. We do not have any assurance that all physical resources and all equipment produced under certain R&D; contracts have been entered.

The Administration Directorate is responsible for administering the AMMIS, which supports managers in the discharge of their duties in respect of the security and control of physical resources in their custody.

RECOMMENDATIONS

SPACE TECHNOLOGIES BRANCH

• i) Remind managers of their physical resource management responsibilities.
• ii) Validate inventory records with the help of the Administration Directorate.

CORPORATE MANAGEMENT DIRECTORATE

• iii) Ensure that the corporate materiel management system is up to date and accurately reflects the assigned responsibilities for custody of physical resources.

4.0 ACCOUNTABILITY

The government has adopted a management framework that focuses on results. This management framework requires managers to go beyond activities and outputs and concentrate on actual outcomes, that is, on the consequences and effects of their programs. When practicing results-based management, outcomes must be given constant attention at every stage of a program. Managers must clearly define the outcomes to be achieved, implement the program/service, measure and evaluate performance and, if required, make adjustments to improve efficiency and effectiveness.

This type of management thus requires managers to give periodic reports on the performance of their programs and services.

4.1 PERFORMANCE DATA

Management relies on several different accountability mechanisms, some more formal than others:

• Verbal reports: ongoing dialogue between staff and management, which are fostered by their proximity.
• Weekly reports: brief narrative reports made by responsibility centres to higher levels of management.
• Project reports: a mid-year report for each STRP project.
• Work plan progress reports: twice-yearly reports outlining accomplishments in terms of results indicators defined and accepted at the beginning of the year, when work plans are approved.
• VP-STP program review: periodic review that uses a pre-set format to present a summary of the situation of a program/project in terms of costs, schedules, programming and budget. Where required, it also presents significant accomplishments, upcoming activities and issues.

All of these instruments are essentially aimed at reporting on the status of activities and projects. They are not designed to report on the performance of these activities and projects in terms of the objectives pursued by implementing them.

We reported in Section 2.1 that some programs under the responsibility of the Space Technologies Branch included a performance management framework. In this regard, the STDP has the most developed framework and would appear to be a good example to follow. Performance indicators were defined for each component of the logic model (outputs, immediate outcomes, intermediate outcomes, final outcomes), and information sources and/or a data collection methods were identified. Furthermore, the database used for program management facilitates the storage and analysis of data collected. This rigorous and highly structured approach should facilitate performance evaluation throughout the entire life cycle of a program.

However, we found that management did not generally give formal reports on program performance despite the existence, in some cases, of a performance measurement framework.

An ongoing reporting system is needed to permit monitoring of the progress of programs in terms of objectives so that corrective action can be taken, if necessary.

However, we found that management did not generally give formal reports on program performance despite the existence, in some cases, of a performance measurement framework.

An ongoing reporting system is needed to permit monitoring of the progress of programs in terms of objectives so that corrective action can be taken, if necessary.

RECOMMENDATIONS

SPACE TECHNOLOGIES BRANCH

• i) Develop a performance measurement framework for each program, based on the framework developed for the STDP.
• ii) Collect data throughout the entire life cycle of programs.
• iii) Periodically report on program performance in terms of the objectives pursued.

APPENDIX A - AUDIT OBJECTIVES AND CRITERIA

The overall objective of this audit project was to evaluate the adequacy of the key elements of the management framework of the Space Technologies Branch.

More specifically, the following objectives were set for the audit:

Objective 1: Ensure that operations are well planned.

Objective 2: Ensure that operations and resource use are controlled.

Objective 3: Ensure that managers account for outcomes and means used.

APPENDIX B - SUMMARY OF OPERATING COSTS

SPACE TECHNOLOGIES BRANCH
OPERATING COSTS
BY PROGRAM ELEMENT, 2003-2004

APPENDIX C - MANAGEMENT ACTION PLAN