Formative Evaluation of the
Aboriginal Youth Business Initiative
Final Report
Audit and Evaluation Branch
December 31, 2002
Executive Summary
Background
Aboriginal Business Canada (ABC) established the Aboriginal Youth Business Initiative (AYBI) in 1996 to enable Aboriginal people between 18 and 29 years old, inclusive, to access financing and support services for the purpose of setting up businesses. The initiative is implemented through two types of Aboriginal controlled organizations, Aboriginal Capital Corporations (ACCs) and Community Futures Development Corporations (CFDCs), collectively referred to as Delivery Agents (DAs). These organizations have been allocated funds through contribution agreements, called projects. To date, 37 AYBI projects have been approved, involving 20 DAs.
ABC's financial commitment to the projects amounts to $11.4 million. Of this, $4.5 million is allocated to loan capitalization, $1.8 million is available to cover the forgivable component of the loans, and $5.2 million is directed toward AYBI operating costs incurred by DAs.1 The maximum loan size is $15,000 with the average being approximately $10,000. The DAs are distributed across Canada, including the Northwest Territories and the Yukon, but with no presence as yet in Saskatchewan, Nunavut, parts of Alberta and Ontario, and a limited presence in the Maritimes.
Eligible clients of AYBI are Status and Non-Status Indians, Inuit and Métis in Canada. In a business partnership, the youth client must be the majority owner. Loan contributions are available at an interest rate normally not exceeding 8.5%. DAs are required to match the loan capital component of the AYBI contribution. DAs have considerable flexibility in offering the program to suit their operational requirements and adapting it to the opportunities for their targeted clientele. They provide assistance for entrepreneurial training, business planning and mentoring.
Study Approach
In accordance with Treasury Board Guidelines for formative evaluations, the study focused on program delivery, client reach and short-term impacts of the initiative; a more thorough examination of these together with program rationale will be conducted in a later summative evaluation. The approach involved four lines of evidence: document and file review; 28 interviews with ABC/AYBI management (7), DAs (12), AYBI clients (6), and other AYBI stakeholders (3); a survey of all DAs (20) for loan information; and site visits to two DAs and two clients.
Findings and Conclusions
Initiative Rationale
Overall, the goals and objectives of the initiative are consistent with the expectations of stakeholders, although there is wide recognition that they are difficult to achieve. At the same time, based on our consultations with the DAs, it was apparent that DAs place a different priority on serving the youth population. It was also apparent that the capability of the DAs to serve youth varied significantly. DAs cited many challenges faced by Aboriginal youth entrepreneurs, including a lack of formal education, work experience, and entrepreneur role models. The majority of DAs have difficulty drawing viable business plans from their targeted populations at the rate they had forecasted. On the other hand, while the number of clients interviewed was limited, they often indicated that AYBI was the only initiative that fit their needs, and that without AYBI the DAs would not have been in a position to be of assistance.
The evaluation team concluded that AYBI is consistent both with federal government priorities and those of the DAs. The initiative appears to be filling a gap between sources of financing available to Aboriginal youth entrepreneurs. The evaluation team concluded that the innovative approach of AYBI, that of delegating responsibility to the DAs within certain pre-determined constraints, continues to be appropriate given the initiative objectives, scope and resources.
Initiative Design
DAs and clients broadly agreed with the design of AYBI in addressing the challenges of encouraging entrepreneurship in Aboriginal youth. AYBI provides DAs with considerable flexibility in implementing the program to suit the requirements of their target population. A majority of DAs consulted still feel, however, that components of the program design not under their control require revision. These are discussed in the report.
The evaluation team concluded that the major design components of the initiative are felt to be well founded and appropriate. Specifically:
- The interest rate of 8.5% is sufficiently high to be attractive for DAs, while permitting clients to develop a credit rating.
- The current age restriction (clients must be less than 30 years old) is appropriate given the objectives of the initiative and the anticipated negative effect an age limit increase would have on the availability of services and products to those under 30.
- The total AYBI loan contribution amount of $15,000 per client is sufficiently high to complement other micro-credit programs, such as that of NACCA, and sufficiently low to keep funds available for providing start-up funds to the targeted market of qualified early stage entrepreneurs.
- The forgiveness rate of up to 25% is sufficiently high to provide an additional incentive to both repay the loan, resulting in a favourable credit rating, and to reduce the exposure of the DAs' AYBI loan portfolio.
- The requirement for matching funds should be continued to secure the commitment of the DAs to the success of the loan, while diluting the risk to the Crown.
- Those matters relating to the loan administration process (e.g. borrower equity requirements) currently under the purview of the DA should remain so.
Service and Delivery
There is a continuing recognition of the extensive efforts required by financing organizations to identify potential youth clients, and the relatively low proportion of these who become loan recipients. DA efforts vary widely, in large part depending on the size of the catchment areas of DAs, and whether they are rural or urban. Most DAs are providing pre-care and after-care services to clients, often including business planning and sometimes bookkeeping assistance and tax advice. DAs generally reported that the lack of business sophistication of most clients puts high demands on staff.
The view of the evaluation team from the evidence gathered is that DAs vary considerably in their approaches, targeted population, level of commitment, and capabilities. At the same time, it is clear that the challenges AYBI seeks to address are formidable, and that the needs of the different targeted populations vary widely. The DAs consulted reported a number of practices that they have found to be effective in AYBI implementation and these are listed in the report.
Given the considerable challenge faced by the evaluation team in collecting and interpreting service and delivery information from DAs, it was concluded that the initiative would benefit significantly from obtaining agreement between ABC and the DAs on an AYBI performance measurement and reporting strategy. We believe more diligence on the part of ABC in the earlier years of the program to address this shortcoming would have been beneficial to both ABC and the DAs. We also concluded that the program design, while innovative and appropriate, is poorly communicated and/or understood both by DAs and by their clients.
Client Reach
Overall, 7% of the population targeted by the DAs has participated in the initiative through services received and/or as loan recipients. DAs reported that most of their potential clients were aware of AYBI. DAs used a wide variety of promotional methods, including newspapers, radio and television advertising (when budgets allowed), and informing Bands' Economic Development Officers (EDOs) and officials at Chambers of Commerce so that they could promote the initiative. It became apparent in our consultations that the most effective method for promoting awareness of AYBI was by DA staff visiting schools, job fairs, and other youth venues, including hockey tournaments, and holding office hours in local communities.
Despite extensive efforts of the evaluation team, the information received on the reach and impact of the program over the course of the study was frequently of poor quality or incomplete. We were able to determine, however, that a broad range of services is being offered through AYBI to a relatively small number of clients. In order to improve client reach, we examined the possibility of merging AYBI with FNIYB and concluded that, with the recent transfer of responsibility of AYBI to ABC regional offices, it would be premature to consider a merger at this time.
Short Term Impacts
Most client-related evidence gathered was anecdotal given our limited success in arranging interviews with them. The businesses of the clients interviewed employ an average of less than three full time equivalent positions, including that held by the client, but are viable, though most are still in the early stages. Overall, the businesses supported by AYBI appear to be sole proprietorships requiring mostly low level skills, but sometimes technical skills in a broad range of fields. Generally, clients interviewed report optimistic expansion plans.
From the survey analysis it was determined that DAs received, on average, 50 loan applications to date (median was 32), and approved 27 (median was 21), for an approval rate of 54%. With 19 of 20 DAs reporting, and based on what are felt to be reasonable interpretations of the responses provided, it is estimated that 568 loans have been supported by AYBI with an average loan size of approximately $10,000, and the average repayment period was 3.3 years. Approximately 52% of the loans remain open, which appears high given that the program is five years old and a higher proportion of loans were written in the earlier years of the initiative. To date, 11% of all loans have been written off; if this relatively low rate is maintained the program will have made an important contribution to clients' credit ratings.
Summary of Recommendations
The study makes thirteen recommendations divided into four areas, as follows:
1) Improving the awareness of DAs about information on the AYBI terms and conditions and the best practices in delivering it;
Recommendation #3 (see page 3-10): That the eligible uses of forgiveness funds be communicated to AYBI stakeholders.
Recommendation #7 (see page 3-16): That a program guide be prepared for DAs, and be published on the ABC website. This guide would include information such as:
- Initiative objectives and design
- ABC stacking rules that apply to AYBI loans and their forgiveness component
- Summary of terms and conditions in a typical contribution agreement
- Summary of terms and conditions under the discretion of DAs
- Respective responsibilities of DAs and ABC regional and central offices
- Standardized reporting forms and reporting deadlines from DAs to ABC
- Claims and disbursement processes, and
- The AYBI contribution agreement renewal process.
Recommendation #8 (see page 3-17): That regional offices facilitate the sharing of best practices among DAs.
Recommendation #11 (see page 3-27): That ABC request that all advanced forgiveness funds be returned or deducted from future payments, and that the practice of advancing forgiveness funds be replaced by disbursing such funds on an as required basis.
2) Improving the quality of the AYBI youth market information used to forecast demand in AYBI proposals from DAs;
Recommendation #1 (see page 3-5): That a standard market assessment tool be submitted by DAs as part of the application and renewal processes for AYBI projects. This tool should segment and quantify the targeted populations, their growth rates, the opportunities — or problem areas – to be addressed in each segment, and the number of clients that have been, and that are forecast to be, served by each AYBI product or service.
Recommendation #2 (see page 3-5): 2 That each DA conduct a broader assessment of the product and service mix of all the programs targeting youth entrepreneurs within their catchment area. This analysis would include:
- An inventory of entrepreneur support products and services, by funding body and program, and including important terms and conditions;
- A positioning of AYBI in the DAs' overall product and service mix; and
- A specification of the DAs' governance and accountability responsibilities for their product and service mix.
Recommendation #13 (see page 4-2): That should AYBI be considered for an increased level of funding, that this funding be made contingent on the implementation of the preceding recommendation.
3) Introducing performance measurement and reporting strategies to the initiative;
Recommendation #4 (see page 3-16): That in addition to financial audits, reports of compliance3 with AYBI policies be prepared.
Recommendation #5 (see page 3-16): That ABC develop a model performance measurement and reporting strategy for AYBI. This strategy, and relevant reporting tools, should be developed in consultation with ABC regional offices and with DAs.
Recommendation # 6 (see page 3-16): That DAs include an adapted version of this model in their funding applications reflecting their corporate strategy in meeting the objectives of AYBI.
Recommendation #9: That, for DAs deemed to have inadequate approaches to delivering the AYBI program and/or not to be responding to requests for performance information, ABC promptly exercise the right to sit on the Board of that DA to determine the appropriate action to be taken.
4) Should funding be increased to the initiative, positioning AYBI by each DA in the overall mix of programs and services targeting youth entrepreneurs in the catchment areas of these DAs.
Recommendation # 10 (see page 3-22): That should an inventory of entrepreneur client support products and services be prepared by DAs (see recommendation above), that a case study should be undertaken of a selection of three DAs (one urban ACC, one rural ACC and one CFDC, each in different regions) of the anticipated impact of the merger of AYBI with FNIYB and other youth support programs.
The study further recommends that AYBI continue to be supported at its current funding level, contingent on the implementation of the recommendations made in the report, with increased funding being dependent on a broader assessment of the needs of the target population.
Recommendation # 12 (see page 4-1): That AYBI continue to be supported at its current funding level, contingent on the implementation of the recommendations made in this report — with the exception of recommendation #2 — in recognition of the sound rationale and design of the program and the general capabilities of the DAs in serving the needs of the targeted youth population. The period of the renewal would be determined based on the amount of time required to receive better evidence on its delivery, reach and impact. This period would be defined in the performance measurement and reporting strategy.
Formative Evaluation of the Aboriginal Youth Business Initiative — Final Report (PDF, 2,366KB, 46 pages)
Formative Evaluation of the Aboriginal Youth Business Initiative — ABC Management Response (PDF, 104KB, 8 pages)
Note: to read the PDF version, you need Adobe Acrobat Reader on your system. If the Adobe download site is not accessible to you, you can download Acrobat Reader from an accessible page. If the accessibility of PDF is a concern, you can have the file converted to HTML or ASCII text by using one of the access services provide by Adobe.
Date Created: 2004-01-13
|