Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

This chapter presents a description of the findings related to program implementation and lessons learned from the delivery of TJF to date. At the time of the study, the program had been in place for approximately 20 months. The results are based primarily on qualitative evidence obtained from 30 key informant interviews. The perceived satisfaction of project sponsors with TJF and their suggestions for improvement based on the survey data are also incorporated into this chapter.

3.1 Program Promotion

In general, key informants indicated that they engaged in limited active promotion of TJF to communicate the program to prospective project sponsors. For the most part, promotion of the program was limited to contacts between HRCCs and TJF partners, such as federal economic development agencies, provincial government departments, and community groups.

According to key informants, the absence of active promotion of TJF did not appear to have a negative impact on the level of participation of sponsors, although, with the exception of Quebec, there was some concern with a lower than anticipated number of TJF applications received in the first year of the program. As awareness of the program increased in the second year, however, the number of applications for TJF funds generally surpassed original targets. In fact, most interview respondents reported that the non-repayable aspect of the program was a very attractive feature of the program, and as a result, few marketing efforts were required to stimulate interest and participation among sponsors.

Although active targeting of specific groups was typically not undertaken by TJF partners, the majority of recommended proposals were generally those which reflected local and/or regional priorities, and which were consistent with the target groups set out in local strategic economic plans. In the few instances where active targeting was undertaken, these included projects designed to serve the needs of social assistance recipients, persons with disabilities, visible minorities, and aboriginal groups.

3.2 Approval Process

The process for approving TJF funding is a fairly rigorous one involving several levels of approval and input from partners. Typically, proposals are received by HRCCs and reviewed by local assessment committees made up of representatives from various local, provincial and federal stakeholders. Applications which are deemed viable and which meet approval criteria are forwarded to TJF consultants at the district and/or regional level. Proposals which do not meet the basic criteria are generally returned to the proponent for additional information or clarification, or are referred to other federal or provincial/territorial programs better suited for the proposed project.

At the regional level, TJF consultants confer with their provincial counterparts to ensure that the projects conform to provincial policy objectives and do not conflict with strategic economic plans. Written provincial concurrence is required for all TJF proposals. This stage of the approval process also ensures that approval from the local MP has been obtained, that all necessary licences and permits have been acquired, and that all affected provincial and federal government departments have been consulted on the project. Once the region is confident that all regional criteria have been met, recommended proposals are forwarded to National Headquarters (NHQ) for final Ministerial approval.

Although the funding criteria vary from one community/region to the next, the general TJF guidelines are applied to assess proposals. Generally, applications for TJF funding must demonstrate that the projects will create sustainable jobs. In addition, HRDC's total contribution to the project must not exceed 50 percent of total project costs. The business plan must be feasible, and must demonstrate the long-term viability of the project and value-added benefits to the community. A fourth core criterion is that the project must be consistent with local and regional economic priorities.

In addition to these criteria, proposal review teams also consider a number of other factors when assessing applications. These factors, however, vary across regions. For instance, Atlantic Canada appears to place considerable emphasis on the types of jobs created by the project. The creation of full-time rather than part-time opportunities, year-round rather than seasonal employment, and the wage levels and skill requirements associated with the jobs all play important roles in the assessment procedure in Atlantic Canada.

HRDC's cost per-job created via the project is another commonly used criterion in the assessment of TJF proposals. For example, the criterion used by some areas is that HRDC's total contribution to the project from all program sources must not exceed $20,000 per job.

Particularly given the non-repayable aspect of TJF, in many communities, the potential for unfair competition with existing organizations is an important issue in reviewing funding proposals. As a result, a thorough analysis of the competitive impact of a proposed project is often conducted prior to the approval of funds.

A fourth commonly utilized criterion for funding is the level of personal investment and risk on the part of the sponsor, and likelihood that the sponsor will obtain financing from other sources. Projects are rarely approved until the review teams are satisfied that other sources of financing are in place, and that the sponsor will take on sufficient personal risk to help ensure long-term project success.

As a result of the consultative approach utilized at the HRCC level, in which applicants are provided the opportunity to provide more information, clarify certain information, or modify portions of the proposal to better meet approval criteria, few TJF applications are rejected. Nevertheless, some projects are ultimately not approved. These typically consist of those which are too costly, cannot meet basic TJF guidelines, do not demonstrate long-term viability, do not conform to local, provincial and/or federal policy objectives, display limited personal investment and risk from the sponsor, or are seen as capable of proceeding without TJF funding.

Overall, the approval process is generally perceived as highly effective among the key informants interviewed. The inherent flexibility of the program, in combination with the consultative approach to assessing proposals, helps to ensure that the projects most in need, as well as those most likely to benefit the community, will be approved. Moreover, there appears to be genuine interest from all partners to fund only those projects which will create "real" employment opportunities rather than "make-work" jobs.

3.3 Partnerships

According to the vast majority of key informants, partnerships form an important component of the effective delivery of TJF. The TJF program makes extensive use of formal and informal partnerships at the local and regional levels, particularly in the proposal review process. Based on interview responses, the sharing of expertise and information resulting from the partnership approach brings increased scrutiny to the proposal review process, and tends to lead to the approval of projects which are most likely to succeed. In addition, the various priorities and perspectives of the partners contribute to ensuring that projects reflect local, provincial/territorial and federal objectives and needs. Moreover, a good mix of partners helps sponsors identify and leverage additional funds from other sources.

For the most part, TJF partners include HRDC, other provincial/territorial and federal government departments and/or agencies, local economic development boards, municipalities, local industry, and community organizations. In addition, certain areas also involve tribal/band councils, education, and/or labour representatives.

Overall, partnerships which appear to be most successful include those which are built on established relationships, and which encourage extensive and open communication. Other less frequently cited characteristics of successful TJF partnerships include an equal voice among all partners, sufficient local representation, a vested interest in the creation of "real" jobs among all partners, and balanced decision-making authority in the proposal approval process.

3.4 Monitoring and Follow-up

Interview respondents indicated that the procedures currently in place for monitoring on-going job creation activities and for tracking the progress of business operations are adequate. For many projects, sponsors are required to submit periodic payroll reports which identify the wages paid to employees in the jobs created via the TJF program. The payment of TJF funds is frequently contingent on the information presented in these reports. In addition, TJF representatives occasionally conduct on-site visits to projects to assess the extent to which operations are proceeding as planned.

According to key informants at HRDC, however, a need may exist to develop and/or improve follow-up mechanisms for assessing the on-going sustainability of the jobs created, as well as for monitoring the use of capital expenses covered by TJF funds. The majority of HRDC respondents expressed concern with their ability to evaluate the extent to which specific jobs created within the projects would persist in the long run. The extent to which the program will create sustainable jobs is not well monitored.

Concern was also expressed with respect to monitoring the use of TJF funds allocated to capital expenses. The data from this study have indicated that use of TJF for capital expenditures is lower compared to wage-related expenses. Current reporting requirements are typically limited to the submission of invoices for capital purchases, and no formal mechanisms for tracking the purchase/sale of these goods have been implemented. Moreover, although NHQ provided TJF staff at the HRCC level with training in assessing business plans, because of staff turnover (e.g., due to implementation of federal-provincial labour market development agreements), some HRCCs indicated that they now lack some expertise in reviewing and assessing the progress of business operations.

3.5 Duplication and Overlap

In general, key informants reported that they consider the TJF to be a complement to other federal and provincial job creation and economic development programs. According to the majority of respondents, the TJF represents an additional source of financing for viable projects rather than a competing alternative. Particularly given the funding limits of the program, TJF funds are frequently used in conjunction with other federal and provincial sources of financing, and often serve to provide funding to projects which could not be obtained within the existing network of funding sources. Moreover, TJF funds, in combination with the financing obtained from other federal and provincial programs, are often sufficient for increasing the sponsors' equity, thus enabling them to leverage additional funds from traditional commercial lenders.

There is also some indication of complementary activities in the way in which the funds obtained from the various sources are used. Particularly given the job creation aspect of the program, TJF funds are typically used for labour expenses. Other similar federal and provincial economic development programs, however, appear to focus on providing funding to help cover capital expenses.

Although a minority of key informants expressed some concern with duplication between TJF and other programs, most respondents indicated that duplication was not an issue. TJF is highly flexible in terms of which projects qualify for funding, and the jobs created under the TJF projects are generally open to any worker. Many other HRDC job creation programs require that individuals hired into new employment opportunities be EI eligible, on social assistance or employment disadvantaged. In addition, sponsors are frequently referred to other federal and provincial programs should these be deemed more appropriate for the project.

The primary concern with duplication expressed by some HRDC and partner organizations related to the potential for TJF activities to overlap with those of regional economic development agencies. These agencies provide financing opportunities to similar ventures as TJF projects, often in combination with TJF funds, but on a repayable basis. As a result, in some cases project sponsors will attempt to access TJF funds first before those of their repayable alternatives. Consequently, if TJF funds are used as a first resort and principal source of funding, the proportionate economic development role played by these agencies may be diminished for these projects.

3.6 Satisfaction

Project sponsors' overall ratings of satisfaction with the Transitional Jobs Fund were extremely high: 93 percent indicated they were satisfied (responded 5, 6 or 7 on a 7-point scale) overall and 81 percent stated they were very satisfied (responded 6 or 7 on a 7-point scale). Only 3 percent expressed dissatisfaction with the program. Overall satisfaction ratings were similar for private and public sector organizations. Responses were also not influenced by organizational size or the percentage of total costs that the TJF funding represented.

Respondents were asked to rate their satisfaction with various aspects of the program. Reflecting the overall satisfaction ratings, satisfaction with the implementation of TJF was also very high (Exhibit 3.1). The highest average ratings were for support provided by HRDC following approval of the project, with 90 percent of respondents indicating they were satisfied, and for the overall administration of the program (88 percent). The next highest approval ratings were for the amount of funding received (86 percent) and the availability of information about the Transitional Jobs Fund (78 percent).

The lowest average rating of satisfaction was for the application approval process (72 percent). Project sponsors in the public sector were less satisfied with the approval process: 22 percent indicated they were dissatisfied compared to 8 percent of those surveyed in the private sector. Smaller organizations and those in the Atlantic region were also more likely to be dissatisfied with the TJF approval process.

3.7 Lessons Learned

(a) Perceived Program Strengths

The most frequently cited strength of the TJF program among key informants was that it focuses on the creation of "real" sustainable jobs. One of the primary objectives of TJF is the creation of sustainable jobs, and the rigorous approval process was developed to help ensure that funded projects are sufficiently viable to achieve this objective. According to respondents, the procedure for reviewing proposals and for obtaining multi-level approval is an important component of the program, as it improves the program's ability to identify viable projects likely to create sustainable employment. There is also an indication that all TJF partners demonstrate continued commitment to recommending only those projects which create long-term permanent employment. Moreover, the vast majority of interview respondents indicated that they thought that the TJF represented a fundamental attitudinal shift among all stakeholders from that of accepting "make-work" jobs as a feasible option, to that of creating "real" opportunities for sustainable employment.

Another commonly reported perceived strength of the TJF program was that it expanded financing options for project sponsors. According to respondents, it is frequently difficult for sponsors to obtain financing for projects from existing sources. Consequently, the TJF was seen as an essential alternative to these other sources of funding, particularly in areas where commercial lenders are reluctant to take risks. Moreover, the increased equity resulting from TJF funding often improves the sponsor's ability to leverage funds from these other sources.

Flexibility in the approval criteria was also a frequently reported strength of the TJF program. It appears that the open criteria for funding projects has allowed for the start or expansion of a variety of projects, and has led to community economic diversity. This was seen as particularly important in more depressed areas. In addition, as a result of the flexible hiring requirements for jobs created under TJF, key informants believed that sponsors have created employment opportunities which are more stable, higher paying, and more broadly skill-based than those of existing job creation programs.

A fourth strength of TJF identified by respondents was that the program tends to attract sponsors who are experienced and professional rather than "fly-by-night" business operators. According to key informants, this is primarily a result of the rigorous approval process in which business plans undergo considerable scrutiny, and the fact that approval of projects is often contingent on the level of personal investment and risk brought to the project by the sponsor. As a result, TJF partners can be fairly confident that approved projects are viable, and are likely to meet their job creation targets.

The final commonly reported strength of the program was the partnership approach to reviewing business plans and funding projects. The TJF approval process relies heavily on the input and perspectives of the various partners involved in reviewing business plans. Based on responses from key informants, joint consultation and sharing of expertise in this process not only increase the likelihood that approved projects are viable and create sustainable jobs, but also help to ensure that the needs of all federal, provincial, and local partners are met. Moreover, required partner consensus in the approval process, and the fact that projects are typically financed through a variety of sources (including the sponsor's portion) lead to a shared accountability for the projects, and necessarily result in a vested interest among all partners for projects to succeed.

(b) Perceived Program Weaknesses

Despite the reported strength of the proposal review process, this same process was, by far, the most frequently cited weakness of the program. While seen as effective for helping to ensure that TJF-approved projects are viable and meet federal, provincial and local needs and guidelines, the vast majority of key informants reported that the approval process was too lengthy and involved. According to respondents, many sponsors are not in a position to be able to wait six to eight weeks for a funding decision, as they require immediate financing in order for projects to proceed. In addition, knowledge of the lengthy process may have discouraged sponsors with viable projects from applying for TJF funding.

Some key informants also expressed concern with respect to some political presence in the program at times. While a certain amount of political involvement is expected in a program involving partners from the various levels of government, there was nevertheless some uneasiness among some respondents regarding projects which may have been approved for political reasons rather than based on the strength of the business plan.

The final commonly reported weakness of the TJF was the non-repayable feature of the program. As indicated by key informants, non-repayable funds may encourage sponsors to use TJF monies as a major source of funding rather than as a supplement to existing financing options. Moreover, particularly in Atlantic Canada, respondents reported that the provision of non-repayable funds to TJF projects may diminish the role played by federal economic development agencies and provincial/territorial programs which offer similar financing but on a repayable basis.

(c) Suggestions for Improvement

Key informants were asked for their opinions on any changes they would recommend in order to improve the TJF program. By far, the most common response received was to extend the life of the program. According to respondents, the TJF program is a cost-effective means of creating "real" sustainable jobs, and with a few minor adjustments, has the potential to bring considerable economic relief to communities experiencing high unemployment. Moreover, the majority of key informants ranked the TJF as being as good as or better than other previous or existing job creation programs.

The second most frequently cited suggestion for improvement was to delegate more decision- making authority to the local level. The current approval process only grants HRCCs the authority to recommend projects. Although the vast majority of projects recommended at the local level are ultimately approved, the need to obtain approval at all stages often results in unnecessary delays. Particularly for smaller projects (less than $100,000 of TJF funding) which frequently require the prompt payment of funds, local level authority to approve projects would help to speed up the process. It is important to note, however, that every respondent who recommended this change also indicated that the current multi-level approval process should be retained for larger projects.

Key informants also recommended making concerted efforts to continue to target small and medium sized projects in order to ensure that TJF helps projects which are most in need. Large projects typically possess more equity than do smaller ones, and are thus in a better position to leverage financing from other sources than are smaller ventures. It was also reported that it is difficult to assess the extent to which TJF funds are truly helping the larger projects when the proportion of total project financing covered by TJF funds is relatively small.

The implementation of a formal procedure for linking the payment of TJF funds to project payroll reports was another frequently mentioned suggestion for improving the program. Tying the payment of funds to confirmation of wages paid has been successfully utilized as a tool for monitoring the progress and job creation activities of several existing projects, and a formal system may help reduce the potential for abuse in the use of TJF funds.

A fifth suggestion for improving TJF was to solicit additional involvement of local agencies and boards in the proposal approval process. Additional input from community-based social and economic organizations would increase the likelihood that projects reflect strategic local priorities and better meet community level needs. Moreover, increased involvement of local representatives would help to ensure that the TJF program retains its bottom-up orientation. It is interesting to note that the majority of these recommendations were provided by respondents in Atlantic Canada, where local level input is currently solicited on a fairly extensive basis.

Several key informants noted that it is currently difficult to identify whether or not project sponsors are accessing TJF funds as a last resort. These respondents thus suggested developing formal mechanisms for assessing the extent to which applicants have exhausted all other financing possibilities prior to approving TJF funding and increase the incrementality of TJF funds. According to these individuals, a system such as this would provide assurances that approved projects are indeed those most in need of non-repayable TJF funds and increase the incrementality of TJF funds. In addition, this mechanism would help maintain the economic development role played by other funding agencies. Unfortunately, no respondent who made this recommendation could formulate a precise strategy for achieving this goal.

A final common recommendation provided by key informants, particularly in the central and western regions of the country, was to offer additional or on-going training to HRCC personnel. Additional business training would allow HRCCs to acquire sufficient expertise in reviewing business plans, thus reducing their reliance on partners in the assessment of proposals. In addition, this training would help improve the HRCCs ability to monitor the progress of approved projects and to assess sustainability of jobs. It was also noted that, because of staff turnover, the training should be available on an on-going basis.

Project sponsors who were surveyed as part of this evaluation were also asked to provide any suggestions for improving TJF. A substantial proportion (44 percent) offered no suggestions to improve the program (Exhibit 3.2). Reflecting the satisfaction ratings described in the previous section, the most frequently suggested improvement to TJF, mentioned by one in five respondents, was streamlining the approval process. Smaller projects and sponsors from the Atlantic region were most likely to suggest streamlining the approval process.