Operational Audit

Canadian Firearms Program

December 2001

Internal Audit Division

Table Of Contents

Executive Summary

The Canadian Firearms Program, hereafter referred to as the Program, provides for the universal licensing of firearms owners and the registration of firearms. The Canada Firearms Centre (CFC) is responsible for the overall implementation of the Program. The Firearms Act (Bill C-68) mandates a licensing deadline, which was reached on January 1, 2001, and a registration deadline of January 1, 2003. The Program operates through the CFC and various federal, provincial, and territorial partners.

This audit is the last in a series of audits that were initiated to assess the operations of the CFC and to review progress to date on the implementation of the Program. It is based on work carried out between August 2000 and March 31, 2001. The audit was segregated into two snapshots-immediately prior to the January 1, 2001 licensing deadline and just after that date. The first snapshot focused on how the Program was dealing with the approaching deadline, and the second addressed how management was handling the processing of licence applications and how it planned to meet the 2003 registration deadline.

During the audit period management initiated a new strategic direction for the Program that has changed the impact of this audit to a significant degree. Although information on this "New Vision" is evolving, its scope includes three major components: a replacement computer system, delivery of all related business processes, and support in achieving steady-state operations. The tight time frame for putting this vision into place raises the risk that implementation delays will have significant impacts. It should be noted that this new business vision has the potential to significantly alter the shape of the organization. As such, the results of this audit may be overtaken by the scope and velocity of future change.

This audit report covers four key themes: the operation of the Canada Firearms Centre in program delivery including its relationship with the federal, provincial, and territorial partners, program performance, financial performance, and computer system performance.

Operation of the Canada Firearms Centre

We find that CFC management has made real progress in creating an operational organization out of the development team that existed when the Program first initiated operations in December 1998. However, as a result of a change in management team and the pending change in management direction, organizational uncertainties within the CFC remain. During the audit period, the Centre had evidenced low staff and contractor morale due to fears of possible downsizing, a lack of formal strategic planning, and significant change pervading the organization. Actions set in motion to address such strategic concerns have been sidelined by the changes in management direction. During the initial phase of the audit, inconsistent communications between the CFC and its operational partners had deteriorated leading to misunderstandings over roles and responsibilities. However, by the end of the audit period (March 2001) CFC and partner efforts had helped to dissipate many of the concerns regarding the quality of communications.

Program Performance

Program performance continues to be affected by various factors. At a strategic level, the concern about the low rate of applications by the public and the operational inability to handle the workload has been addressed. However, designing a national workload management process is still only at an elementary stage and will be affected by options being considered for the new business vision. Overall, licensing and registration processing has been improved as previous problems with the Central Processing Site have been addressed. The operation of the Outsourced Processing Site has had a positive impact. This facility was set up as a temporary measure to specifically address the huge volume caused by the rush to meet the licensing deadline. Such a facility is being considered to assist in processing registration applications.

Work processes in the field are now more robust as the partners have become more adept at using the system, processing the various transactions, and getting the required information out of the system. A combination of temporary licensing and an Outreach Program to help the public with completing applications has been very successful, although management problems will require attention if this approach is used for registration. Some other problems also remain: partner interactions with the CFC can be inconsistent; the operational agreements under which the partners operate require review; and partner expense forecasts are at times delayed making the creation of Payables at Year End (PAYE) problematic.

Financial Performance

The CFC continues to have difficulty managing revenue and ensuring financial control over the movement of moneys. Although cash received can be tracked in most cases, problems occur in reconciling revenue, since the computer system-the Canadian Firearms Information System (CFIS)-lacks full accounting functions. Although the system does capture financial data related to the applications processed, it does not control how that data is subsequently altered. For example, the CFIS cannot deal with refunds. This function must be handled manually in a highly work-intensive process. As a result, it is not possible to reconcile revenue received with the number of licences issued.

System Performance

Although much improved from over a year ago, certain aspects of system performance are still a concern. For example, the system slows as the rate of processing increases. Changes to the system have a significant impact on processing operations and system downtime after a new release is still common. Future strategic decisions concerning information technology will have an impact on what-if anything-is done to the CFIS with regard to further system development changes. No further improvement in the overall capacity of the CFIS can be made without massive change and, as a consequence, significant investment.

Conclusion

By the end of fiscal year 2000-2001, the Program will have cost $527 million. Current management directions that involve outsourcing the steady-state operation, outsourcing the completion of registration, and reorganizing the CFC are aimed at streamlining operations and reducing costs. However, care must be taken in this period of renewed development activity not to repeat past mistakes that increased costs. In addition, CFC management must ensure that a financial capability has been sufficiently defined as part of any new business processes to avoid current financial control problems.

The management response to the recommendations contained in this report was provided by R. Landry, Director, Corporate Services, Canada Firearms Centre on April 25, 2002. An additional response was provided to section 4.3 in July 2002.

1. Introduction

1.1 The Canadian Firearms Program

The Firearms Legislation Task Force, set up within the Department of Justice in 1992, was instrumental in developing the Firearms Act (Bill C-68), which was proclaimed in December 1995. This Act is the underlying legislation for the establishment of the Canada Firearms Centre (CFC). Formally set up in 1996, the CFC is the central headquarters responsible for the overall implementation of the Canadian Firearms Program, hereafter referred to as the Program.

The Program provides for the licensing of firearm owners and the subsequent registration of their firearms. In order to be licensed or to register a firearm, individuals must meet certain criteria, complete an application, and submit the appropriate fee. The Program became operational on December 1, 1998 when the CFC started accepting and processing licensing applications from individuals. The Firearms Act mandates a licensing deadline of January 1, 2001 and a registration deadline of January 1, 2003.

A coalition of federal, provincial, and territorial partners deliver the Program throughout the country. The CFC has central responsibility for overall delivery; the other jurisdictions involved have more discrete roles. Two roles are crucial to the Program-the Chief Firearms Officer (CFO) that oversees licensing individuals and the Registrar who specifically oversees firearms registration.

Initially each provincial and territorial jurisdiction involved had a CFO role to play. Since the outset, however, several jurisdictions have opted out of this aspect of Program delivery leaving the federal government to name federal CFOs and to run the Program in those locations. The Canadian Firearms Registry, hereafter referred to as the Registry, is the responsibility of the RCMP as a key federal partner. The role of the Registrar has been assigned to the RCMP and is filled by a uniformed officer. Other important federal partners include the Canadian Customs and Revenue Agency, the Department of Foreign Affairs and International Trade, and the information technology area of the RCMP as a crucial support element in the electronic delivery of computer system functions.

As part of its mandate to implement administrative and functional services to deliver the Program, the CFC developed a complex computer system, the Canadian Firearms Information System (CFIS). During the audit time frame this system dealt primarily with licensing, but was designed to meet registration requirements as well.

At the request of the partners and to facilitate the processing of applications, the Federal Government established a Central Processing Site (CPS) in Miramichi, New Brunswick. This location acts as the funnel for all application processing. A second processing site was established in Montreal, Quebec (QPS) to handle all Quebec-based applications. Once the workload was more fully understood, a third location-the Outsourced Processing Site (OPS) in Ottawa, Ontario-was set up as a temporary facility to handle the mounting volume.

2. Scope

This audit is the last in a series of audits that were initiated to assess the development and operations of the CFC and to review progress to date on the implementation of the Program. This report is based on audit work largely carried out between August 2000 and March 31, 2001.

2.1 Purpose and Objectives

The purpose of this audit was to review and build on the recommendations from previous audits and other management reviews, to assess the changes being implemented or initiated for optimizing program operations, and to determine whether the management framework can support the drive to meet the legislative deadlines for licensing and registration.

Auditors were tasked with the specific objectives of:

• Reviewing the operational efficiency of the workload delivery mechanism country-wide, the quality of partner interactions, and the overall capacity to process transactions;
• Assessing the risks the CFC faces in meeting the legislative deadlines with regard to adapting its management framework to promote efficient and effective operations, ensuring its effective management of the interfaces between partners, the interaction with the public in promoting compliance with the Act, and the error level in current operations, and particularly with regard to potential future strategic decisions.
• Following-up on operational efficiency initiatives and overall operational improvement.

2.2 Approach and Methodology

The approach and methodology used to carry out the post-operational audit of the Program is consistent with accepted practices, procedures, and standards as defined by Treasury Board's Manual of Review, the Institute of Internal Auditors, and the Canadian Comprehensive Auditing Foundation.

The audit examined four key themes:

• Operation of the Canada Firearms Centre-the role of the CFC in program delivery and its relationship with the various partners;
• Program Performance-the management of work processes, partner operations, and public adoption initiatives;
• Financial Performance-the management of revenue at all levels for the various licences and registrations as well as the CFC function of expenditure management and budget allocation;
• System Performance-the ability of the CFIS system to handle the volume of transactions to successfully meet both legislative deadlines.

To pursue these themes, the audit reviewed: the operation of the processing sites in Miramichi, New Brunswick and in Ottawa, Ontario; the various provincial/territorial Chief Firearms Officers (CFOs) that deliver the program, including those operated by the Federal Government; the federal partners that have responsibility for discrete elements of program delivery; and the management of the CFC itself.

2.3 New Management Vision

The audit was segregated into two snapshots-just prior to the licensing deadline of January 1, 2001 and just after that deadline. The first snapshot focused on how the Program was dealing with the approaching deadline. The second snapshot reviewed the Program from December 2000 to March 31, 2001 and addressed how management was handling the volume of licence applications and how it planned to meet the registration deadline of January 1, 2003.

Both assessments followed the same basic format as discussed under Approach and Methodology. Detailed findings are set out below. However, given the increasing pace of change within the Canada Firearms Centre, care must be taken in reading the conclusions contained here. During the audit period the CFC experienced significant changes at the senior management level as part of a management renewal process. This new management team has recently developed a fresh initiative or "New Vision" to establish a new strategic direction for the Program. This new business vision has changed the impact of the audit to a significant degree by proposing extensive changes to program delivery organizations and processes.

During the audit period, information on the New Vision for the Canadian Firearms Program was limited. However, the scope of the vision includes three major components: a replacement computer system, delivery of all related business processes, and support in achieving steady-state operations. The proposed schedule for implementing a new automated product is tight; the final product is expected to be operational in April 2002, at which time the CFIS will be decommissioned.

3. Detailed Findings

3.1 Operation of the Canada Firearms Centre

CFC management has made real progress in creating an operational organization out of the development team since the Program initiated operations in December 1998. However, even before the deadline date of January 1, 2001 strategic directions were being reconsidered by management. As a result, many changes were being halted, rethought, or even superseded.

Change at the most senior levels of management provided a much needed impetus for renewal throughout the organization. The main focus for that change has been the strategic redirection of the Program. However, as a result of the pending change in management direction, other organizational uncertainties within the CFC are evident. For example, at the beginning of the audit period CFC management was forecasting possible downsizing as a means to address the high operational cost of the Program. The resulting uncertainty had a negative impact on both staff and contractor morale. Managers worried that, rather than cope with future employment instability, experienced individuals would seek other positions. By the completion of the audit period, it had become clear that to reduce operational costs a policy of attrition was necessary. As a result, the organizational structure of the CFC is still very much in a state of development.

At the start of the audit we observed that the quality of communications between the CFC and the various levels of operations could be improved. Partners felt communications from the CFC were neither sufficient nor timely, especially concerning any potential new management direction. In particular, management changes at the CFC were not well communicated resulting in misunderstandings regarding respective roles and responsibilities. However, both the CFC and the partners made efforts during the interim between the two audit visits, and by the end of the audit period, partners' concerns about the new management direction had begun to dissipate.

For some time, the organizational uncertainty was evident from a lack of strategic planning and a significant level or change. Some duplication of function was evident. In particular, quality management and quality assurance have been assigned to several different units. Although CFC management chose this approach to better achieve local accountability, standard operating procedures such as workload reporting was not evident across all processing sites. On the other hand, some functions do not appear to be represented in the organization. For example, staffing of a business process analysis unit was planned but has not occurred.

As well as improvements in organizational structure, some key business processes are still needed for the CFC to move fully into an operational mode. For example, some processes are crucial to ensure that business requirements are: captured at source, used to define system and procedural changes, and managed into operation. The CFC has been trying to develop and implement such processes since beginning operations in 1998. A selection of an overall business process methodology-Rational Unified Process or RUP-has been made. However, the decision to proceed with RUP has been sidelined by the changes in management direction.

Despite the potential impact of strategic changes, administrative support procedures are needed. Until the scope of the new direction is solidified, many of the previously identified management priorities have been delayed.

3.2 Program Performance

3.2.1 Workflow Management

Managing the licensing workflow required new and changing directions to deal with the existing volume and the projected deluge by January 1, 2001. Achievements included: improving the Central Processing Site (CPS), creating the Outsourced Processing Site (OPS), improving application forms, opening a mail handling site, and creating a virtual call centre where incoming calls were forwarded to British Columbia, Alberta, Quebec, and Ontario, as well as to the Registrar, for response by staff.

These efforts greatly increased incoming work volumes, but not without attendant problems. The Outreach Program used a new application form among other things. (For a more detailed discussion of the Outreach Program, see Public adoption initiatives section of this report.) Nonetheless many errors still occurred despite the new forms. In particular, the Date of Birth field was often ignored when the mailing address portion of the form was not used. Advertising also had an impact that was not always fully anticipated. For example, the organization was not ready to deal with the increase in applications generated by the advertising campaign during the 2000 Summer Olympics. Interest was so high that the call centre was losing many calls. In order to cope, the Central Processing Site had to move processing staff to call centre duties with a corresponding impact on applications processing.

A national workload management process does not yet exist. Such a process would provide a means to balance workload across the entire operation; produce comparable workload statistics regardless of where the process was carried out; and provide management with sufficient high-level information to support strategic decisions. Overall, we found that workload management has improved, although planning problems and suspected poor communications have caused problems. The Chief Operating Officer (COO)-responsible for all operations of the CFC-manages the national workload, but is not provided with supporting staff. When the strategic direction is finalized, it will affect how a national workload management process is implemented. Thus, no recommendation is made on this topic, although it is suggested that this form part of the new business vision.

3.2.2 Processing Centres

Applications for licences and registrations are initially processed at several processing sites. The Central Processing Site (CPS) in New Brunswick was created through a Memorandum of Understanding (MOU) with Human Resources Development Canada (HRDC), as that department has the operational experience for this kind of work and could provide staff support and training. The Quebec Processing Site is the responsibility of the Chief Firearms Officer (CFO) in that jurisdiction.

CFC management has addressed the need for increased throughput in applications processing by improving and expanding on existing operations. First of all, by resolving some long-standing problems concerning internal communications, staff relations, and the need to meet changing management directions from the CFC, the operational capability of the CPS has been significantly improved. The MOU between the Department of Justice and HRDC has been set aside giving the Director of the Central Processing Site full decision-making authority.

Daily operations at the CPS seem to be more fluid and better focused than during the first audit snapshot. Specifically, general operation is much improved as a result of better computer system response time and removal of the problematic process gap between initial processing and full application processing. By the time of the auditors' second visit, larger volumes of applications were being processed and the call centre was handling the call volume with much less difficulty.

However, supporting work processes still require attention to maximize the operational capability of the Central Processing Site. For example, site management finds that system releases do not always include corresponding business process changes. At the time of the audit visit in November 2000, the CPS was still adjusting to the most recent release of the system (Release 4.1). Partially to compensate for this problem, the CPS took over, in practice, the role of business process design-a role not intended for that organization and for which specialized staff was not available. Since processes necessarily evolve over time, the local staff at the CPS were trying to align current processes with legislative and regulatory requirements. This role was more fitting for the CFC headquarters. Although performance of the CPS would be enhanced through the use of business process specialists, the new business vision may have an impact that could preclude further changes for now.

As a second alternative to address processing capability, the Outsourced Processing Site (OPS) was set up to improve overall Program effectiveness in dealing with licence applications. Internal quality assurance reports have resulted in management improvements. The process originated for the OPS was well thought out and complete. The process has been well enough received that the Central Processing Site has copied it.

The OPS was conceived to improve overall capability for processing licensing applications in the short term. We noted several weaknesses in handling applications errors and exceptions, archiving paperwork, scanning paper applications to generate computer images, and in the design of interior office space. While none was significant enough to seriously hamper licence application processing, should the OPS be considered for handling firearms registrations, management improvements in these areas would increase efficiency.

3.2.3 Partner Operations

Partner operations represent an essential element of program delivery. Overall, partner operations have definitely improved. In particular, work processes-the specific steps followed in processing applications-have shown dramatic improvement.

The operation of the central call centre (CPS) has greatly improved, and the ability of partners to handle registrations and transfers has increased. Reporting on the safety training courses that are required for applicants has improved, and business processes are operating well. Computer system stability is generally better, although partners still experience system downtime. Partners are making an effort to keep work processes similar, but some partners are more independent than others. One area where jurisdictional differences exist is in conducting safety training courses-every province has different requirements. However, partners are meeting to develop a plan to ensure adequate similarity and acceptance of results across the country regardless of the province or territory of origin.

Partner interaction with the CFC has proven inconsistent. During the first audit snapshot, the CFC had made some unilateral decisions even in apparent opposition to partner interests. Partner concerns related to potential new management directions in the CFC were evident. Current operations between the CFC and its partners are governed by a set of operational agreements that outline staffing levels, estimated work throughput, and so on. These agreements were to expire in March 2001. The funding of local operations was being directly affected by the new management direction in the CFC but with little input from the partners. Because the CFC has not made any movement toward renegotiation, the CFOs felt they are being marginalized by the CFC with regard to program management and development. However, since the first audit snapshot, CFC management has made a significant effort to improve communications between themselves and the other partners. This effort has included providing information on the new management direction and has resulted in the CFOs becoming more involved.

These operational agreements are problematic on several fronts. They need to be reviewed in order to deal with the new management vision and existing business requirements. Also, the CFIS computer system was not implemented when the CFC negotiated the agreements, so staffing levels were based on anticipated volumes as opposed to real numbers. In the future, CFC management will be better equipped to determine what is needed with real numbers and the acquired knowledge of how the system works. These agreements also allow for audits of partner expense claims. However, the claims have not been coming in, and as a result, few audits of this expenditure have been conducted.

3.2.4 Public Adoption Initiatives

The Program has always been under the pressure of the looming legislative deadlines for licensing owners and registering firearms. In the effort to meet the first of these deadlines-licensing by January 1, 2001-CFC management set up some initiatives to improve public understanding of the legislation and to increase the number of applications being submitted.

In order to achieve overall targets for licensing, CFC management created the Outreach Program to assist the public in finding, completing, and submitting applications. Both fixed and mobile teams met the public, assisted in completing applications, took pictures of clients, validated the information, and submitted the validated packages. The Outreach Program resulted in increased numbers of applications received as well as much improved client communications. However, it did experience some planning and management difficulties.

An idea rejected by management at one time as too expensive, Outreach was called upon to gear up quickly, sustain itself for a period of time, and generate substantial numbers of applications. Despite doing a good overall job, the program was a victim of its own success as sites ran out of resources such as applications and Polaroid film. In addition, management experienced difficulty in getting the payroll system to pay outreach staff appropriately and to provide reimbursements of expenses in a timely fashion.

The Outreach Program provided face-to-face contact with clients. This aspect benefited the licensing portion of the Canadian Firearms Program since the actual client population was not known. However, for firearms registration that population is not only identified, but location information is also known from the data provided for licensing. Therefore, if outreach is to be used for registration, the program will require significant reworking to deal with known management problems, such as those mentioned above, and a clearly identifiable client population.

In the event that the Program might not be able to process applications or that operations might not be able to issue licences fast enough, a contingency of issuing temporary licences based on initial processing was set up. This initial processing involved such things as confirming an applicant's name and address and carrying out a basic criminal record check. If no concerns were raised, then a temporary licence would be delivered with full application processing for the permanent licence to follow later. However at the time, the CFOs were concerned that their role under the legislation left them responsible for licences regardless of how these licences were produced. If a mistake occurred in initial processing at either the operational or system level and a temporary licence was issued that should not have been, it would be the CFO who would shoulder the resulting impact, not CFC management who had made the decision to issue the licence in the first place. The CFOs were also concerned about the volume of application processing and the impact on their operations should applications arrive at a faster rate closer to the deadline.

As it turned out, although several errors occurred, including duplicate licences being issued, temporary licences issued without CFO involvement, and some licences that should not have been issued needing to be revoked, the overall picture is very much a successful one with a large number of applications handled-over 1.2 million since the fall of 2000. However, continued management attention will be required to ensure all entitled temporary document holders receive permanent licences in a timely fashion.

The operations of the Canadian Firearms Registry have had to change in order to accommodate CFC management's decision to give highest priority to meeting the licensing deadline. The organization of the Registry is structured to handle the expected work processes surrounding the identification of firearms, the managing of exceptions, and the final approval of firearms registrations. Initially, the Central Processing Site was responsible for capturing application information; any errors were sent to the Registry. When CFC management decided to concentrate all efforts on licensing, a backlog of registration applications was created. In July 2000, the Registry started accepting and processing registrations. As was the case for licensing, the process was separated into two steps: initial processing-capturing basic information such as name and address, acknowledging receipt of the application fee, and so on-and full application processing. Initial processing on 120,000 registration applications was completed by September 2000. The Registrar received another 34,000 envelopes in December 2000 representing the October to December volume. The decision to support the overall CFC licensing drive by proceeding with initial processing only for registration must eventually add to the overall work queue for the Registry, because full application processing has yet to be undertaken.

The Registry felt that 150,000 applications might mean 450,000 firearms to be registered (single owners with multiple firearms). Thus, the Registry would have a large number to finish. More applications could also be expected after licensing was completed. By the time the licensing workload began to decrease, the registration workload would be immense, as no processing would have been done on application processing for some time. If these projections prove accurate, the Program will again face the risk of having to deal with out-of-date payment instruments (cheques, money orders, etc.), which accompany each application. To ensure that the licensing volume had been effectively handled, no work on registrations was anticipated until June 2001.

At the time of the second audit snapshot (December 2000 - March 31, 2001) the CFC was developing the Registration Readiness Initiative (RRI) to prepare for an anticipated push for registration-similar to that for licensing-before the next legislative deadline (January 1, 2003). To date a very low number of firearms have been registered in compliance with the legislation, and some doubt exists that the Program goals can be accomplished in the time remaining. The Registrar has estimated that under the current system, registration could take up to six years to complete.

Accuracy of data is a key element of the registration system. An analysis in 1995 of the existing Restricted Weapon Registration System found old data to be largely incorrect and incomplete. However, trying to keep tight control over new registrations proved to have an impact on operations in that many applications were rejected since they contained inaccurate information-applicants often did not know or did not accurately express the required data on their firearms. Management changed the firearms description requirement and now the Registration Readiness Initiative is looking at the problem of meeting legislative deadlines while ensuring sufficient detail. As a result, the Registry has prepared alternatives for managing registration as a function within the CFC and has redesigned the application forms to simplify the process.

These issues and challenges facing the CFC in terms of the approaching registration deadline point to the need to make registration a key priority. For this reason meeting registration requirements has been a significant factor in the decision to develop a new business vision.

3.3 Financial Performance

3.3.1 Revenue Management

Although cash received can be tracked in most cases, problems occur in reconciling revenue as the design of the CFIS lacks full accounting functions. The system does capture financial data related to applications processed, but it does not control how that data is subsequently altered. For example, the current system does not deal with refunds. This function must be handled manually in a highly work-intensive process. If a client decides to change from a Non-Restricted Possession and Acquisition Licence (PAL) to a Possession Only Licence (POL) requiring a refund of $50, clerks would open a new file with a 0 balance to reflect the actual licence issued. The revenue would be recorded in the old file-now inaccurate-and a refund would be processed for the balance. If that old file were then closed, the impact on revenue would be unknown. In addition, the file balance reverts to 0 when a licence is issued-even if there is a positive balance that must be refunded or a negative balance caused by a change in application or a Not Sufficient Funds (NSF) payment. As a result, it is not possible to reconcile the revenue received with the number of licences issued.

Furthermore, Public Works and Government Services Canada (PWGSC) is not receiving accurately reconciled nor complete financial information on a timely basis. CFC Finance and Administration Directorate receives transactions from PWGSC for deposits and tries to balance with the CFIS. CFC Finance must factor in refunds and VISA reversals as well as NSF cheques. The timely distribution of revenue is affected by difficulties in reconciling the accounts.

The Reconciliation and Refund Clerk conducts verification of refunds through the use of Excel spreadsheets designed and maintained through the departmental network by CFC Finance in order to capture all relevant information related to: cash refunds (cheque issue), credit card refunds (maintains combined as well as separate VISA/Mastercard spreadsheets), chargebacks (combined VISA/Mastercard spreadsheet), and bulk payments (maintains separate spreadsheet by payment method). Refund requests are compared with previous refunds issued from all sources in order to prevent duplicates from being issued. Most often the supporting documentation for each refund request is incomplete and must be compared with current data in the CFIS. This work is very human-resource intensive.

CFC Finance designed and maintains these Excel spreadsheets, because the information required for purposes of verification may not have been captured in the CFIS. In any event, the financial limitations of the CFIS do not permit finance staff to replace these spreadsheets with a more efficient control system.

A second problem area concerns deposits. Jurisdictions are required to transmit deposit information to CFC Finance. The format for reporting varies between jurisdictions and may contain deposit information of one or more days. The CFIS-generated deposit number that appears on the "CFIS Current Account Deposit Summary" is transcribed by a jurisdiction onto the financial institution deposit form. We found from the "List of Deposits" that, although the deposit numbers are sequentially assigned in the CFIS, there appear to be breaks in the assignment of these numbers. CFC Finance is unaware of any CFIS report that accounts for sequence breaks or the assignment of duplicate numbers. Furthermore, only one employee in a jurisdiction signs a deposit transcribed onto a financial institution deposit form. While it was noted that two employees are required to sign the CFIS "Current Account Deposit Summary," this requirement does not appear to be applied to the preparation of a financial institution deposit form.

The Reconciliation and Refund Clerk reconciles the deposit information contained in the transmission from a jurisdiction to the Receiver General deposit information. Adjustments to a deposit are made under the same CFIS deposit number assigned to the original deposit. The Reconciliation and Refund Clerk maintains a list of unreconciled deposits that have been reported on the Receiver General daily deposit information and not found on the weekly CFIS report "List of Deposits." A review of this list between the period April 20, 2000 and November 24, 2000 showed several VISA and Mastercard deposits as well as a number of "No Such Deposit in Records" situations.

The third area of concern pertains to NSF cheques. The Receiver General returns NSF cheques to CFC Finance. These cheques have no specific identification to allow tracking, and CFC Finance staff must go back to the regions to obtain documentation from the local financial institutions. Often this process is slow, which adds to the timeliness concern, while the impact is felt on reconciliation and distribution. The CFC may hold back hundreds of thousands of dollars in revenue in order to account for hundreds of dollars in NSF cheques. Also, Treasury Board Secretariat (TBS) has regulations for charging a fine for NSF instruments. Yet, at the time of audit, this fee had not been charged. CFC Finance is currently developing a policy to charge such a fee.

3.3.2 Program Funding

Program funding has been an ongoing management concern since the outset. By the end of Fiscal Year 2000/2001, the project will have cost $527 million, representing six years' expenditures. Concern over growth in Program costs has been a significant factor in the New Vision direction recently undertaken. As a result, real decreases in budget are forecast.

3.4 System Performance

A major component of Program delivery has always been the Canadian Firearms Information System (CFIS). This complex integrated system was built in-house to meet the requirements as defined during the system development period. However, subject to the demands of a constantly changing environment, the resulting product is now described as overly complicated, too reliant on paper processes, and no longer suited to the business purpose of the Program.

3.4.1 System Capacity

The overall capacity of the CFIS has evolved to the point where further improvement will require massive change and, as a consequence, significant investment. This realization, added to the change in software requirements in order to achieve steady state, has prompted the management review of strategic alternatives.

Although much improved from over a year ago, certain aspects of system performance are still a concern. The system slows as processing throughput increases-the Program is up to 1200 users countrywide now. The system slows down in the afternoon when all users are working. In the past network capability was a bottleneck. However, technology improvements have since caused a significant reduction in traffic. Adjustments made to the way the system manages work queues improved throughput dramatically, as did improving how the Canadian Police Information Centre (CPIC) shares information on criminal record checks with the CFIS.

3.4.2 System Development Processes

System development changes have become a lower priority since the arrival of the new management team. For example, decisions were taken to move toward temporary licensing and to move exception handling to the CFOs. Although now considered a lower priority, system concerns still requiring attention include eliminating temporary manual workarounds and fixing discovered and documented errors-new ones continue to arise with each new system release.

Despite extra time taken in the management of version releases, high priority changes to the CFIS still cause problems for operations. Lack of specific training for the Central Processing Site indicates that the CFC does not appreciate the impact of system changes on processing operations. We found initially that system downtime after a new release was still common, although by the end of the audit period it had become less of a problem. Changes made to system releases between design and delivery are often not well communicated.

Finding a way to improve system capability while reducing overall program cost has become a crucial objective of the new business vision. Therefore, a replacement computer system is one of its major components. Strategic directions concerning information technology and the overall support of any system for the CFP will have an impact on what-if anything-is done to the existing system. Nonetheless some uncertainty remains. It is not yet clear what will be outsourced under the new plan, nor whether the optimistic implementation schedule for the new product can be met.

4. Conclusions

The pace of change for the Canadian Firearms Program and the Centre has increased to the point that there has been an impact for both the audit and the recommendations arising out of previous audit reports. The recent evolution of the New Business Vision also means that the utility of new recommendations on organizational, procedural and systemic risks and concerns is limited. As a result, we are focusing the recommendations on areas where management attention is required regardless of the strategic direction taken.

4.1 New Management Vision

Most recommendations from past reports are now of a "lessons learned" variety. Although these recommendations were created during the development phase of the Program and as such do not easily relate to current operations, the lessons offered should not be ignored.

The audit has offered recommendations in years past in a number of crucial areas, including:

• Governance (program and partnership structure, strategic planning, and administrative processes),
• Program development and management,
• Financial management and control,
• System development and implementation.

There is obvious evidence of learning through experience. However, some crucial lessons can be applied today. For example, during the previous period of CFIS development, the Canadian Firearms Program focused on program delivery. However, this focus was combined with unrealistic budgetary and delivery expectations on the part of the stakeholders. A history of massive changes resulted in problems controlling scope and spending.

Current management feels that in undertaking the development of the CFIS, the Program committed to a solution that was more extensive than was required by legislation. But since management is once again looking for a technical solution, care must be taken to ensure similar problems do not ensnare this initiative.

In developing a strategic direction under the new business vision, management will need to consider the reorganization of the CFC, the operation of the processing sites, the roles of the partners, and the development and implementation of a new steady-state computer system. However, the full impact and cost of this business vision cannot be fully estimated until the outsourcing process gets underway and proposals can be assessed.

Management seems well aware that a crucial risk to realizing the New Vision for the program is the very tight time line. An organization is in place to manage the transition. However, this audit would like to further emphasize the critical importance of monitoring the time frame.

Recommendations and Management Response

1. It is recommended that CFC management ensure an appropriate monitoring approach be implemented to maximize mitigation of the risk of delay in product delivery.

Agreed. CFC will implement a management monitoring approach to mitigate risk in product delivery. An Alternate Service Delivery (ASD) team is being established and Public Works and Government Services Canada (PWGSC) is proceeding to seek advance Treasury Board approval for an Alternate Service Delivery Contract for our program. The objective is to have the Preferred Proponent commence exploratory activities in April, assuring the Canada Firearms Centre (CFC) of a new "steady state" operational system by January 2003.

4.2 Revenue Management

Overall financial control is adversely affected by the difficulty involved in reconciling the flow of financial information. CFC management must ensure that a financial capability has been sufficiently defined as part of any new business processes in order to avoid current financial control problems.

Recommendations and Management Response

2. It is recommended that:

1. CFC management ensure the development of a complete financial capability for revenue as part of the move to the new business vision.

Agreed. Financial capability for revenue is being developed as part of the ASD contract. Such factors as refund capacity, eliminating offline fee capture and financial system data grouping are being addressed.

2. As part of this, CFC should:

Fee reductions are offset in IFMS against either current year revenue or refunds to previous years revenue account depending upon what fiscal year the corresponding revenue was received.

Non-recording of client refunds affects the current revenue reports produced by CFIS. Currently an external database is maintained by CFC finance and the revenue reports are offset accordingly. The newly proposed ASD provider was requested to provide refund capability in their new system, which would rectify this problem.

• Undertake a review by jurisdiction of the deposit experience related to overages/shortages by financial institutions to determine the materiality of the differences and whether further analysis is warranted.

Review of deposit overages and shortages is monitored on an ongoing basis as part of the revenue reconciliation process. When deposit inconsistencies arise they are investigated to ensure deposit procedures are being followed. A review of the deposit, reconciliation and cash management processes of the Outsourcing Processing Site was conducted January 12, 2001 to investigate such deposit discrepancies and corrective actions were taken where necessary.

1. Ensure the amount recorded on a financial institution deposit form is confirmed with the make-up of the deposit and with the CFIS "Current Account Deposit Summary;"

CFC has in place policies and procedures with regards to the creation and confirmation of deposits. If discrepancies arise between amounts actually deposited and the CFIS deposit information they are investigated by CFC finance in Ottawa.

2. Ensure the deposit form is signed by the preparing employee and by a financial officer whose signature will attest that the deposit has been verified;

Where possible CFC will implement this recommendation to ensure more than one employee signs the deposit forms. Some jurisdictional offices are so small that only one signature is practical. The volume of deposits in these jurisdictions is usually relatively small and therefore any inconsistencies are easily tracked.

3. Develop system reports of unassigned/duplicate deposit numbers for purposes of deposit control and reconciliation;

System reports of unassigned deposit numbers are being developed for the new ASD contract. The CFC reports and queries team cannot currently produce such a report because the missing numbers are not recorded in the database and therefore a report cannot be developed.

4. Review the cost of matching low-value, same-denomination cheques to the Receiver General reports with Department of Justice Finance in order to set an acceptable, cost-effective cheque-matching policy;

A review was conducted of the matching of low-value same denomination cheques to the Receiver General reports. Subsequently the CFC reconciliation policy was changed to increase jurisdictional involvement in the reconciliation process. This greatly increased the accuracy and efficiency of the returned cheque reconciliation. Discussions with DOJ finance are currently underway to restructure the CFC bank accounts in a more centralized structure to allow for more practical reconciliation of Receiver General accounts.

5. Ensure the follow-up and collection of charge back and NSF fees that have not been collected from applicants.

There are currently procedures in place to ensure the follow-up and collection of charge back and NSF fees that have not been collected from applicants. First, notices are sent to the clients to request payment. If payment is still not received the licence is withdrawn from the system. If the client received a charge back after their licence was issued then a comment is made in CFIS and the client will be required to pay fees in arrears before licence renewal is possible. This is necessary because legally non-payment of fees is not grounds for licence revocation.

4.3 Current Process Improvements

The new business vision will significantly change the organization and thus the scope and direction of processes now in place. As part of the new re-direction it is important to pay special attention to the contracting process, partner agreements and records management activities. As well, there is a need for enhanced strategic planning and workload management.

Recommendations and Management Response

4. It is recommended that CFC management review the following areas when assessing how to implement the new vision:

1. Improve contracting. There should be adequate separation of duties between raising a request for goods, authorizing the acquisition, verifying receipt of goods, and paying the invoice. An invoice should not be paid without matching Request for Goods/Delivery Slip/Invoice.

Internal procedures have been revised to ensure that all necessary documents are provided before the contract is issued. Request for Goods/Delivery Slip/Invoice are matched before payment is authorized.

2. Improve records management. Given the amount of paperwork involved in licensing and registration, consistent records management should be considered in all processing locations.

Records management training has been provided at processing locations involved in licensing and registration to ensure that staff are aware of proper records management procedures.

3. Re-negotiate the current partner agreements. Ensure the CFOs' expense claims are received and do the required audits to ensure accuracy and reasonability.

New agreements have been negotiated with all of the provincial opt-in jurisdictions and should be signed by late summer 2002 (Note: Quebec's agreement may be signed at a later date). Expense claims are verified for accuracy and reasonability before processing and for the first time will have annual budget ceilings. New MOUs with federal partners (RCMP and CCRA) are presently being negotiated and drafted. Discussions are still ongoing.

4. Improve national workload management. In order to efficiently handle current operational requirements and deal with potential future strategic changes to those requirements, management should consider improving the overall co-ordination of quality management.

A continuous improvement cycle with CFC's provincial partners has begun to be implemented and service standards will become effective with the signing of the new agreements. The new ASD vendor will follow established ISO standards that adhere closely with TB standards.

5. Implement a strategic planning process. For the past year or more much of the management focus has been concentrated on strategic issues. However, the lack of a formal plan to address these strategic issues will ultimately become problematic.

A strategic planning process has been initiated with a strategic plan covering the next two years having been developed.