EVALUATION OF TRANSPORT CANADA'S FUNDING TO THE TRANSPORTATION ASSOCIATION OF CANADA (TAC) - June 2005

Background

Departmental Evaluation Services (DES) conducted an evaluation of Transport Canada’s (TC) funding to the Transportation Association of Canada (TAC) between April 1, 2001 and March 31, 2004. During this period, TC provided TAC with a total of $1,077,782 in funding through: membership fees, a contribution program and sponsored projects.

Key Findings

Relevance

TAC directly supports and advances the mandate and strategic objectives of TC and is consistent with the federal governments priorities.

There is an on-going need for TC’s financial contribution, as the association provides a forum for continued dialogue and consensus building with provincial and territorial governments.

Success and Impacts

TC’s annual funding to TAC is small but worthwhile as the association provides the following benefits to the department:

A forum for transportation professionals to share perspectives and identify issues of mutual consent;
A forum for continued dialogue and consensus building with the provincial governments;
A forum for cooperative funding arrangements for developing, discussing, advancing and showcasing technical knowledge; and
A network to establish national standards and guidelines for roads.

The evaluation found that reporting and monitoring of TC’s funding to TAC needs to be strengthened to improve accountability and results measurement.

Cost-effectiveness

Given the previous stated benefits, the evaluation found that funding to TAC is a low cost investment that allows the sharing of project costs with other jurisdictions.

Recommendations

The logic model and a performance framework for the contribution program needs to be updated and implemented to improve program monitoring.

In order to keep track of TC’s funding relationship with TAC, the department needs to implement a simple process where the program manager responsible for the project maintains a summary highlighting the objectives of the project, total costs and, at the end of the project, results achieved. The department may want to consider keeping these summaries in a central location to facilitate reporting on TC’s overall funding relationship with TAC.

[top]

1.0 INTRODUCTION

This report provides a summary of the results of an evaluation of Transport Canada’s (TC) funding to the Transportation Association of Canada (TAC) through a contribution program, sponsored projects and membership fees. TC has provided funding to TAC since 1978 and, in recent years, has been the largest contributor to TAC.

The evaluation was conducted to meet the requirements of the Treasury Board Secretariat’s (TBS) Policy On Transfer Payments and focuses on the relevance, success and cost-effectiveness of funding to TAC.

1.1 Profile of TAC

The Transportation Association of Canada (TAC)

TAC is a non-profit association of more than 550 voluntary corporate members, which include federal, provincial and territorial transportation departments, municipalities, academic institutions and private sector firms with an interest in roadway infrastructure and urban transportation issues. TAC’s aim is to promote the provision of safe, efficient, effective and environmentally and financially sustainable transportation services in support of Canada’s social and economic goals.[1]

TAC is a neutral forum for gathering or exchanging ideas, information and knowledge on technical guidelines and best practices, primarily with respect to roadways. It receives its funding from a wide variety of sources, including membership fees, the sale of publications, the delivery of training courses, the conduct of sponsored projects, and a contribution program from TC.

1.2 Logic Model – TC’s Funding to TAC

Table 1 shows the main activities and outcomes for TC’s funding to TAC.

Table 1: Logic Model -TC’s Funding to TAC

Activities

Immediate Outcomes

Intermediate Outcomes

Ultimate Outcomes

TC provides funding to TAC for:

Contribution Program

Scholarships to promote and develop transportation skills
Task Force on Vehicle Weights and Dimensions
Councils

1.3 Profile of TC’s Funding to TAC

1.3.1 Contribution Program

In 1991, TC obtained the authority from the TBS to establish a contribution program with TAC to fund councils and transportation safety related projects approved by TAC Board of Directors, which includes a representative from TC. It has been renewing the contribution program annually with the approval of the Minister of Transport Canada, who is authorized to approve payments of contributions and related terms and conditions for up to $250,000 to non-governmental organizations and individuals.

Between April 1, 2001 and March 31, 2004, TC funded a number of projects and councils under the contribution program for a total of $398,797. These are listed in Table 2. Projects are undertaken in collaboration with other governments who share costs. Funding share for both projects and councils is based on a formula with the federal share ranging from 10 to 35 percent, and provincial and territorial share is based on population size. For example, for the Task Force on Vehicle Weights and Dimensions (VWD), TC pays 33.33% and provinces and territories pay 66.67%, with the split by jurisdiction calculated by population size. The total annual project costs for the VWD for 2002/03 was $102,500. The provincial and territorial governments contributed a total of $68,337 and TC contributed $33,825. Results to-date include amendments to the federal, provincial and territorial Memorandum of Understanding (MOU) in order to improve regulatory harmonization, and the creation of a national forum and resource centre for VWD issues. The Task Force has provided an ongoing focus and continuity for deliberations on VWD issues, both between government agencies and with motor carriers, shippers and other stakeholders.

Table 2 - Funding given to TAC through the contribution program to projects and councils between April 1, 2001 and March 31, 2004

Projects	Description of Project	2001/2002	2002/2003	2003/2004	Total
Scholarship Program	A multi-jurisdictional shared fund to promote and develop Canadian transportation skills by financially assisting studies of promising post-graduate students and professionals. TAC Scholarship Program has provided scholarships to post graduate university students for several decades. TC’s annual contribution to TAC’s Scholarship Program was combined with equivalent contributions from P/T jurisdictions to form an annual F/P/T scholarship of $3,000. TAC scholarship program has now been transferred to the TAC Foundation, a new organization set up as a charity at arms length from TAC. The new foundation is currently seeking to increase the scholarship significantly. TC has indicated a continued interest in participating in the Scholarship Program under the new organization, but has not yet committed to a funding level.	$250	$250	$250	$750
Task Force on Vehicle Weights and Dimensions	A multi-jurisdictional task force reporting to the Council of Deputy Ministers for Transportation and Highway Safety. It is a national coordination mechanism for matters related to highway vehicle weight and dimension limits in Canada, including assessing the desirability of more uniform provincial policies, regulations and practises, and seeking consensus on Canada’s position with regard to harmonization of regulations under NAFTA.	$35,743	$33,825	$29,370	$98,938
Conference Board of Canada Study on Governance and Infrastructure Investment	A study proposed by the Conference Board of Canada that was to examine issues related to governance and funding of transportation infrastructure investment. Other sponsors of the study included Air Canada, the Railway Association of Canada, VIA Rail, and the Greater Vancouver Regional District. The total cost of the study is $107,000 which is being shared by the participating jurisdictions and TC.	$0	$0	$20,000	$20,000
Councils	Description of Council
Council of Deputy Ministers (CoDM)	The CoDM is made up of the deputy ministers of the sixteen-federal/provincial/territorial government departments responsible for transportation and highway safety in Canada. Collectively, they are responsible for approving the annual budgets and work plans developed by Policy and Planning Support Committee (PPSC) and submitted by the Secretariat, including proposals for collaborative projects.	$71,606	$57,376	$63,778	$192,760
Chief Engineers Council	A multi-jurisdictional technical council under the auspices of TAC aimed at researching, developing and recommending national guidelines for roadways in Canada. Costs are now absorbed in membership fees.	$74,346	$12,003	$0	$86,349
Total Contribution		$181,945	$103,454	$113,398	$398,797

Table 3: TC Sponsored Projects Funded between 2001 and 2004

TC Sponsored Projects between 2001/2004	Description of Project	Total funding 2001/2004	Year Funded
In-service Road Safety Guide	The objective of this project was to prepare a guide which aims to prevent collisions through the review of road designs at the pre-construction stage. The project resulted in a publication titled In-Service Road Safety Guide as well as a training course on the subject matter.	$12,000	2001/2002
Roadway Illumination Guide Update	The objective of this project was to rewrite TAC's Guide for the Design of Roadway Lighting. The project will utilize the expertise and experience from Canadian as well as international sources in formulating the contents of the new Guide for the Design of Roadway Lighting.	$50,000	2002/2003
Advance Warning Flashers (AWF)	The first phase of the AWF project was undertaken to standardize applications and practices with regard to the use of amber advance warning flashers in Canada. The project produced the Advanced Warning Flashers Application and Installation Guide, which synthesizes the evaluation of the benefits of advanced warning flashers, justification and recommendations for application criteria and options where application is not recommended. The second phase of the project is currently underway to conduct further research to improve the understanding of AWF performance and benefits. The objective is to develop a research and data collection program to support a before-and-after analysis to assess the implication of the variance between posted and operational speeds and effectiveness by application criteria.	$10,000	2002/2003
Salt Management Syntheses of Practice	The guide for Salt Management Syntheses of Practise was updated to provide guidance on locating, designing and managing snow storage and disposal sites. The work resulted in the production of technical briefs on salt management.	$10,000	2002/2003
Performance Measures for Evaluation of Road Networks	This project examined the use of performance measures for the evaluation of road networks. The first phase of the project included a survey of performance measures used by provincial and territorial highway agencies. Work on the project is on going.	$16,585	2002/2003
AASHTO DG Calibration (Pavement Structures)	This project will develop guidelines to adapt and calibrate the US Guide for Mechanistic-Empirical Design of New and Rehabilitated Pavement Structures for Canadian conditions. Phase 1 of the project is currently underway. This phase will identify short and long term requirements and resources needed to implement the new guide, assess Canadian agencies’ needs in support of the design procedure, and develop a road map for subsequent phases of the project. TC is considering funding future phases.	$0	2003/2004
TC Total Funding 2001/2004	$98,585

1.3.3 Membership Fee

All TAC members pay an annual fee. Membership fees cover core services that have been established by the Board of Directors as essential in order for TAC to continue to exist. These include: rent, utilities, telecommunications, database support systems and management of and financial support for secretariat services offered to the four councils and the Board of Directors.

Between April 1, 2001 and March 31, 2004, TC paid $525,000 in membership fees to TAC. See Table 4.

Table 4: TC Membership Fee to TAC

Year	2001/2002	2002/2003	2003/2004	Total
TC’s Membership Fee to TAC	$175,000	$175,000	$175,000	$525,000

1.3.4 Strategic Highway Research Program (C-SHRP)

The C-SHRP was launched in 1987 as a research program in partnership with the Strategic Highway Research Program (SHRP) in the United States. The program ended on March 31^st, 2004 after 15 years. This program was initiated in response to the continuing deterioration of highway infrastructure with the intention of making significant advances in highway engineering and technology. The goal of both C-SHRP and SHRP was to conduct research to improve the performance and durability of highways and to make them safer for motorists and highway workers. SHRP and C-SHRP research contributed 138 products, in the form of new equipment, processes, test methods, manuals and specifications for the design, maintenance and operation of highways. These products and new technologies were implemented and used by various agencies.

Between April 1, 2001 and March 31, 2004, TC contributed $55,400 to TAC for C-SHRP as listed in Table 5.

Table 5: TC Funding for C-SHRP between 2001/2004

Program	2001/2002	2002/2003	2003/2004	Total
Canadian Strategic Highway Research Program (C-SHRP)	$20,200	$20,200	$15,000	$55,400

1.4 Roles and Responsibilities

The Director of Intergovernmental Affairs & Accessibility (ACCF) Branch manages the contribution program and is responsible for ongoing monitoring of the program and individual projects including monitoring milestones. He is in regular contact with TAC management.

TC’s individual directorates that have chosen to conduct a sponsored project with TAC are responsible for the management of the project.

1.5 Study Rationale

The study was conducted to provide input for future decision-making and to assess the relevance, success and cost-effectiveness of TC’s funding to TAC.

1.6 Evaluation Issues

The evaluation study assessed the following questions:

A. Program Relevance

1) Is the TC funding consistent with federal government and TC strategic objectives and priorities?

2) Is there an on-going need for TC financial contribution to TAC?

B. Success and Impacts

3) What is the impact of TC’s funding to TAC?

C. Cost-effectiveness

4) Is the funding provided by TC the most cost-effective means to meet departmental objectives?

A safe and secure transportation system that contributes to Canada’s social development and security objectives;
An efficient transportation system that contributes to Canada’s economic growth and trade objectives; and
An environmentally responsible transportation system that contributes to Canada’s sustainable development objectives.

The funding provided by TC is also consistent with current government priorities as outlined in the most recent Speech from the Throne (October 4, 2004) which outlined a “five‑point strategy to build an even more globally competitive and sustainable economy.” The second element of the strategy is aimed at strengthening Canada’s ability to generate and apply new ideas – a commitment by the government to build a strong foundation in basic science and technology. Given the type of research projects and activities supported by TC’s funding to TAC, the evaluation concludes that this funding does, in a small way, contribute to innovation, research and development and the design of new technology.

2) Is there an on-going need for TC’s financial contribution to TAC?

Finding: There is an on-going need for TC’s financial contribution, as the association provides a forum for continued dialogue and consensus building with provincial and territorial governments.

The provincial and territorial governments have direct jurisdiction over roads and highways. By providing funding to TAC, TC has the opportunity to offer a leadership role in conducting research, developing new road safety projects and influencing transportation decisions.

TAC offers a neutral forum for networking and consulting with provinces, territories and municipalities. The organization’s neutrality provides a structure for federal/provincial relations and allows all levels of government an equal share in ownership and funding of projects that establish national guidelines and regulations.

[top]

3.0 SUCESS AND IMPACTS OF TC’s Funding to TAC

3) What is the impact of TC’s funding to TAC?

Finding: The funding provided to TAC contributes to road safety by financially supporting individual projects and councils that conduct research, develop technical publications and construct and maintain guidelines for road related design.

Interview respondents and project reviews confirm that by providing funding to TAC, TC receives the following benefits:

A forum for transportation professionals to share perspectives and identify issues of mutual concern;
A forum for continued dialogue and consensus building with the provincial governments;
A forum for cooperative funding arrangements for developing, discussing, advancing and showcasing technical knowledge; and
A network to establish national standards and guidelines for roads.

The evaluation found that TAC publishes technical reports on projects that provide tangible advice on engineering practices and the development of national guidelines for road and road related design development. Results of sponsored projects are listed in Annex 2.

For example, the first phase of the Advance Warning Flashers (AWF) sponsored project was undertaken to standardize applications and practices with regard to the use of amber advance warning flashers in Canada. A survey undertaken at the time revealed several different signs and applications in use by different provinces. At the same time, the project committee identified a variety of research studies with contradictory conclusions regarding the potential benefits of the devices. Two studies found that advanced warning flashers are beneficial safety devices while two other studies found that the devices could contribute to crashes and suggested they not be used. Based on the results of the survey and the studies, a sponsored project was undertaken to analyse the potential benefits of the devices and provide criteria for their application in Canada.

The project produced the Advanced Warning Flashers Application and Installation Guide, which synthesized the evaluation of the benefits of advanced warning flashers, justification and recommendations for application criteria and options where application is not recommended.

The second phase of this project is currently in progress. Further research is being conducted to improve understanding of AWF performance and benefits. The objective is to develop a research and data collection program to support a before-and-after analysis in order to assess the implication of the variance between posted and operational speeds and effectiveness by application criteria.

5.0 COST-EFFECTIVENESS OF TC’s FUNDING TO TAC

4) Is the funding provided by TC the most cost-effective means to meet departmental objectives?

Finding: The evaluation found that funding to TAC is a low cost investment that allows the department to share costs of projects with other jurisdictions.

The evaluation found that by contributing financially to TAC, TC receives many services and benefits, as highlighted earlier in the report, at a cost that is shared amongst a wide variety of stakeholders.

As stated in Table 6, between April 1, 2001 and March 31, 2004, TC contributed $98,585 to TAC for sponsored projects and other jurisdictions provided $745,498. By using TAC, all jurisdictions including TC received the full benefits of projects without having to pay all the costs.

Table 6: TC Sponsored Projects Funding between 2001 and 2004

TC Sponsored Funded Projects between 2001/2004	2001/2002		2002/2003		2003/2004
Contribution by Jurisdiction	TC	Others	TC	Others	TC	Others
In service Road Safety Guide	$12,000	$66,652
Roadway Illumination Guide Update			$50,000	$245,000
Advance warning flashers			$10,000	$43,500
Salt Management Syntheses of Practice			$10,000	$96,000
Performance Measures			$16,585	$0
AASHTO DG Calibration (pavement structures)					$0	$294,346.
Total Funding	$12,000		$86,585		$0
TC Total Funding	$98,585
Total Project Budget	78,652		$454,500		$294,346

While the evaluation concluded that a full cost-effectiveness analysis could not be done due to weak outcome reporting, it can make a case that, based on logic and qualitative information the department is achieving some positive outcomes with minimal costs. As shown in Table 7, the evaluation does this by comparing TC’s costs to outcomes under two scenarios: (1) status quo funding and (2) no funding to TAC.

Table 7: Costs and Projected Outcomes for TAC

	Scenario 1 – Status Quo	Scenario 2 – Provide no funding to TAC
Costs
TC Funding to TAC:	$250,000 Contribution $175,000 Membership $86,585 Sponsored projects $20,000 C-SHRP $556,585	$0
Salaries	Minimal HQ FTEs are required to administer the contribution program and sponsored programs and monitor developments	None
Outcomes
	Association is already established for continued dialogue and consensus building with the provincial and territorial governments	TC would have to set up its own association or join other associations e.g. US TC would have to buy off the shelf products, with no Canadian specifics TC would have to form its own ad hoc committees, potentially at a higher cost
	Cooperative funding arrangements with provincial and territorial governments	Provincial and territorial governments would have to cover the shortfall, resulting in loss of goodwill TC would have to contribute more for projects to be conduced, not being a member of TAC
	Keeps TC informed and in the “game” for a minimum investment	TC would not have access to useful provincial and territorial information.
	Access to a set of national standards /guidelines for highway and safety matters	TC would likely need to find another source for these standards and guidelines or develop them on its own.

[top]

6.0 ANNEX

Annex 1: Evaluation Matrix

Question	Indicators	Data Sources / Methodology
Relevance
Is the TC funding consistent with federal government and TC strategic objectives and priorities?	Comparison of contribution objectives to departmental and governmental objectives	Document Review TC strategic documents e.g. Straight Ahead, Speech from the Throne Federal government strategic documents Review of sponsored projects
Is there an on-going need for TC’s financial contribution to TAC?	Anticipated impact on TAC Anticipated impact on TC Opinions of TAC staff Opinions of TC staff	Interviews with TAC staff Interviews with TC Staff Program file review
Success
What is the impact of TC’s funding to TAC?	Opinions of TC Staff Opinions of TAC staff Management files of TAC contributions Review of funded projects and councils	Interviews with TAC staff Interviews with TC staff TAC Business Plan
Cost-effectiveness
Is the funding provided by TC the most cost-effective means to meet departmental objectives?	Opinions of TC staff involved with TAC	Interviews with TC staff involved with TAC Committees

Annex 2: Review of Projects Funded

Contribution Project 1
Project Name	The Task Force on Vehicle Weights and Dimensions (VWD)
Objective of Project	The program was originally set up to develop and recommend a national strategy for the evolution of VWD regulation in Canada. Later the objective of the program was changed to assess the desirability of more uniform provincial policies, regulations and practices and to seek consensus on Canada’s positions with regard to harmonization of regulations under the North American Free Trade Agreement. The Task Force on VWD policy was established in the 1988 MOU as the coordination mechanism for the identification of issues, cooperative analysis, and development of recommendations for harmonization of vehicle weights and dimension policy within Canada.
Total Costs	The funding formula for the VWD Task Force is as follows: TC pays 33.33% The provincial/territorial pay 66.67%, with the split by jurisdiction done by population 2001/2002 – $35,743 2002/2003 – $33,825 2003/2004 – $29,370 Total Funding by TC for 2001/2004 =$98,938
Results Achieved	Results to-date at the national level include amendments to the F/P/T MOU in order to improve regulatory harmonization, and the creation of a national forum and resource centre for VWD issues. The Task Force provides an ongoing focus and continuity for deliberations on VWD issues, both between government agencies and with motor carriers, shippers and other stakeholders.

Contribution Project 2
Project Name	Conference Board of Canada Study on Governance and Infrastructure Investment
Objective of Project	The Council of Deputy Ministers agreed to assist in funding a study proposed by the Conference Board of Canada that was to examine issues related to governance and funding of transportation infrastructure investment. Other sponsors of the study included Air Canada, the Railway Association of Canada, VIA Rail, and the Greater Vancouver Regional District.
Total Costs	TC contributed a total of $20,000 in 2004 to the project.
Results Achieved	The study resulted in a report titled Canada’s Transportation Infrastructure Challenge – Strengthening the Foundations, which was published by the Conference Board of Canada in December 2004.

Sponsored Project 1
Project Name	Roadway Illumination Guide Update
Objective of project	The objective of this project, advanced through the Traffic Operations and Management Standing Committee (TOMSC) of the Transportation Association of Canada, was to rewrite TAC's Guide for the Design of Roadway Lighting.
Total Costs	Transport Canada contributed $50,000 out of a total budget of $295,000.
Results Achieved	A draft report has been produced and should be available for publication shortly, once the Chief Engineers’ Council has approved the documents.

Sponsored Project 2
Project Name	Advance Warning Flashers (AWF)
Objective of project	This project was undertaken to analyze the potential benefits of advance warning flashers in Canada.
Total Costs	Transport Canada contributed $10,000 out of a total budget of $53,500.
Results Achieved	The project produced the Advanced Warning Flashers Application and Installation Guide, which synthesizes the evaluation of the benefits of advanced warning flashers, justification and recommendations for application criteria and options where application is not recommended.

Conclusion

A final report is published after each sponsored project is completed. These technical publications provide tangible advice on engineering practices and the development of national guidelines.

Annex 3: Interview Guides

Questions TAC Officials

1) In your opinion, what benefits does TAC provide to Transport Canada and how valuable is TAC to TC?

2) The table below represents the contribution amounts provided by TC to various projects under the guidance of TAC. (Projects funded by TC between 2001/2004)

Table 1: Contributions paid to the TAC, between April 1, 2001 and March 31, 2004

3) Table below represents the sponsored projects funded by Transport Canada 2001/2004.

Source: Briefing Note – Chief Engineers’ Council Report August 2003 and March 2004

4) What would be the impact if the contribution or sponsorship funding was not available from TC? Where would TAC possibly get funding?

1) What are the advantages to Transport Canada being part of TAC? How valuable is TAC to TC?

5) Which of these objectives were accomplished? What methodology did you use to assess your results? What are / were the results? Where did you report your results?

9) What would be the impact if the contribution or sponsorship funding was not available?

Annex 4: Key References Reviewed

Briefing Note – Chief Engineers’ Council Report, August 2003 and March 2004.

[1] TAC Annual Business Plan 2003-2004.

[2] Straight Ahead – A vision for Transportation in Canada 2003 pg 15.

LIST OF ACRONYMS

Cost-effectiveness

1.0 INTRODUCTION

1.2 Logic Model – TC’s Funding to TAC

1.3 Profile of TC’s Funding to TAC

1.3.1 Contribution Program

Total Contribution

1.3.2 Profile of the Sponsored Projects

1.3.3 Membership Fee

Year

1.3.4 Strategic Highway Research Program (C-SHRP)

Program

1.6 Evaluation Issues

A. Program Relevance

1.7 Data Sources and Methodology

Document and Website Review

Interviews

2.0 RELEVANCE OF TC’s FUNDING TO TAC

3.0 SUCESS AND IMPACTS OF TC’s Funding to TAC

4.0 OTHER FINDINGS

5.0 COST-EFFECTIVENESS OF TC’s FUNDING TO TAC

Costs

Annex 1: Evaluation Matrix

Annex 2: Review of Projects Funded

Annex 3: Interview Guides

Questions TAC Officials

Total Contribution

Other Operating Costs (OCC)

Total contribution + Other Operating Costs

Annex 4: Key References Reviewed