Minister

Services for Business

WD Programs

Reports and Publications

• Audit & Evaluation
• Departmental Plans & Reports
• Departmental Publications
• Departmental Strategies
• Economic Research & Market Studies

Media Centre

• News Releases
• Speeches
• Fact Sheets
• Media Contacts
• Subscribe For News Updates

Investments in the West

• Innovation
• Entrepreneurship
• Community Economic Development

Proactive Disclosure of WD Expenditures

• Disclosure of Travel & Hospitality Expenses
• Disclosure of Contracts
• Disclosure of Position Reclassifications
• Disclosure of Grants & Contributions

Frequently Asked Questions

Feedback

Home : Reports and Publications : Audit & Evaluation : Program Evaluation Study: EDP and UEDI

Program Evaluation Study
Entrepreneurs With Disabilities Program (EDP)
Urban Entrepreneurs With Disabilities Initiative Fund (UEDI)

Executive Summary

Purpose and Method of Study

Western Economic Diversification (WD) has developed two programs tailored to the needs of Western Canadian entrepreneurs who have a physical disability or mental impairment that constraints their ability to perform at least one of the basic activities of self-employment or entrepreneurship.  The two programs are: 

The Entrepreneurs with Disabilities Program (EDP), which was established in 1997, is delivered through the network of Community Futures Development Corporations (CFDCs) in Western Canada with support provided by provincial associations of CFDCs that have been established in each of the four western provinces.

The Urban Entrepreneurs with Disabilities Initiative (UEDI), which was established in 1998 to extend the services of EDP into the major urban centres, is delivered through local non-government organizations in Victoria, Vancouver, Edmonton, Calgary, Regina, Saskatoon and Winnipeg. 

The two programs provide both loans and other business support services.  Since the programs were established, 765 loans totaling $16.2 million have been issued to clients across Western Canada.  Funding for EDP and UEDI was scheduled to end March 31, 2005 but has been extended for one more year.  

The purpose of this evaluation is to provide an objective assessment of EDP and UEDI for the purpose of supporting senior management decision-making for continued funding of the programs.  In addition, the evaluation report recommends areas where changes can be made to better meet the needs of entrepreneurs with disabilities as well as the strategic objectives of the Department.  

We conducted this study in two phases.  The primary objective of the first phase was to prepare a detailed work plan that was implemented in the second phase of the study.  The major components of our field research included in-depth interviews with:

• 8 representatives of WD from each of the four provinces and Headquarters who have been involved in the management, development and delivery of EDP and UEDI.  
• Representatives of 37 of the 90 CFDCs and all 7 urban partners involved in delivering the EDP and UEDI as well as 6 representatives of CFDC provincial associations across the four provinces.
• A sample of 75 community stakeholders who share a similar target market to that of EDP and UEDI. 
• 148 clients who received assistance through EDP or UEDI. 

In addition, we conducted field trips to each of the four provinces in Western Canada to undertake personal interviews with organizations involved in the delivery of EDP and UEDI.

Major Findings of the Evaluation

The major findings of our evaluation are as follows:

1.         It is widely recognized amongst those surveyed that there is a significant need for this type of program.

The representatives whom we surveyed during the course of the study were asked to rate the need for the program on a scale of 1 to 5, where 1 is no need at all, 3 is somewhat of a need and 5 is a major need.  The average rating varied from 3.6 amongst representatives from delivery agencies to 4.2 amongst community stakeholders and 4.5 amongst the clients surveyed.  The perceived need for the programs reflects that:

• Self-employment can represent an important alternative for people with disabilities.  The programs were created in response to the Federal Task Force on Disabilities, which identified employment as a major issue for persons with disabilities and a leading cause of the relatively high incidence of poverty.  Only 42% of adults with disabilities were employed in 2001 as compared to 74% of adults without disabilities. The discrepancy in self-employment rates is even greater.  Across Canada, only about 3% of residents with disabilities are self-employed as compared to 14% of those without disabilities.
• The target group is potentially very large. According to Statistics Canada, over 12% of the Canadian population (3.6 million people) have one or more disabilities.  Among aboriginal people, the disability rate is even higher and is reported to be 30%.  The incidence of disability is projected to increase as the population ages.
• The disabled entrepreneur can face significant barriers related to access to capital, a lack of business sector and marketplace understanding of disability, limited business experience, low self-confidence, more restricted access to information, the demands of the disability, and systemic issues (such as the potential loss of income supports or medical coverage).   In addition, apart from EDP and UEDI, only very limited services are available to assist entrepreneurs with disabilities to start-up or further develop their business. 
• Assisting entrepreneurs with disabilities to establish and further develop businesses can help strengthen the economy in Western Canada.

2.         The programs provide a range of services to assist entrepreneurs with disabilities.

Most clients report receiving more than one type of service.  Of the clients whom we surveyed, 68% indicated that they received a loan under the program, 59% received business counselling and 50% received some form of business mentoring. The EDP and UEDI have complemented other programs supported by WD in a number of ways.  More specifically, the programs have enabled:

• Loans to be provided in urban areas where CFDCs are not located;
• Business counselling and support to be provided that is often over and above what would normally be provided to CFDC loan clients; and
• Loans to be provided on less stringent terms that would otherwise have been available (e.g. less security required, less emphasis on credit history, higher risk businesses, and more flexible repayment terms).

3.         The majority of clients rated themselves as successful or very successful in achieving the objectives they held when they first approached the program. 

For most clients (72%), their primary reason for approaching the program was to obtain assistance in starting a business.  Other objectives were to access business services, expand an existing businesses, upgrade their skills, start a website, and obtain a loan.   On average, the clients rated their success in achieving these objectives as a 4.0 on a scale of 1 to 5 where 1 is not at all successful and 5 is very successful.

4.         Most clients are satisfied with the services that they have received. 

The clients were asked to rate how satisfied they are with the assistance they received on a scale of 1 to 5, where 1 is not at all satisfied, 3 is somewhat satisfied and 5 is very satisfied, The average rating was 4.1 and 48% of the clients indicated they are very satisfied with the services received.  The clients noted that service was good, the loans and other services met their needs, and the staff were very supportive. Those who were less satisfied indicated that they were not able to get the assistance that they needed (particularly loans) or that the process took too long.

5.         The EDP and UEDI clients indicated that the services had a wide range of impacts on themselves and their businesses. 

The clients were asked to rate the impact that participation in the program has had on them on a scale of 1 to 5, where 1 is not impact at all, 3 is somewhat of an impact, and 5 is major impact.  Of the 148 clients surveyed:

• 57% indicated that program had a notable impact (a rating of 4 or 5) in increasing their access to capital;
• 53% indicated that program had a notable impact in improving their quality of life, through improving their financial position, level of independence and/or self-confidence;
• 40% indicated that program had a notable impact in increasing access to business information through means such as one-to-one communication with delivery agency staff or contractors, seminars & speakers, training and written materials; and
• 38% indicated that program had a notable impact in helping them to further develop their business skills through training in areas such as accounting, marketing, legal issues, computers, and website development.

6.         Sixty-five percent of the clients surveyed are currently operating businesses.

At the time when the clients first approached the delivery agency, 11% were operating businesses and 89% were not.  Of that 11% of clients, 10% are still operating businesses while 1% has ceased operations.  Of the 89% of clients who were not yet in business, 69% subsequently started-up operations of which 55% are still operating and 14% have ceased operations or were sold.

7.         We estimate that the average EDP and UEDI loan client generates $260,000 in revenues and 6 person years of employment over the first five-year period after they receive assistance, that is attributable to the assistance that they received.

In developing this estimate, we used the survey results and the results of past surveys to estimate that:

• The average UEDI and EDP client business that receives a loan and continues to operate will generate about $554,000 in revenues over a five year period;
• The percent of businesses that continue to operate after receiving a loan declines from about 85% in Year 1 to about 50% in Year 5.  After adjusting for survival rates, we estimate that the average EDP and UEDI business (including those which are still operating and those which have ceased to operate) would generate revenues of approximately $346,000 over the five year period after they receive the loan. 
• Approximately one year of employment is generated for every $42,500 in revenues.  As such, $346,000 in revenues would represent about 8 person years of employment over a 5 year period.
• About 76% of the economic impacts are incremental in that they would not have occurred in the absence of the program.  As such, we estimate that the services provided to each EDP and UEDI loan client have, on average, over a five-year period generated $260,000 in revenues and about 6 person years of employment.  The importance of the program is reflected in the findings that the clients:
• Were, on average, only 12% confident that they could have obtained financing elsewhere if the program had not been available; and
• Indicated that, on average, that there was only a 22% likelihood that they could have developed their business to this point without the assistance of the CFDC or UEDI delivery agency. 

Given that the average loan client we surveyed received an average of $28,700 in loans, we therefore estimate that the loans result in about $9.16 in incremental revenues for every dollar in loans provided and generate about 1 person year of incremental employment for every $4,500 in loan funding provided over a five-year period.  By extrapolating the survey results to the entire population of EDP and UEDI loan clients, we estimate that (going out over a five year period from when assistance is initially provided) the loan programs have generated about $145 million in incremental revenues and 3,400 incremental years of employment.

Opportunities for Improvement

There is strong support for continuing the programs.  Of the representatives surveyed, 98% of the stakeholders, 91% of the WD and CFDA representatives, and 75% of the delivery agency representatives recommended that the program or programs be continued.  However, most representatives also believe that there is considerable room for improvement with respect to the effectiveness of the program.  When delivery agencies, WD representatives, associations and stakeholders were asked to rate the effectiveness of the programs, on a scale of 1 to 5 where 1 is not at all effective, 3 is somewhat effective and 5 is very effective, the average ratings were relatively consistent across the groups, ranging from 3.1 amongst the stakeholders to 3.3 amongst WD and CFDC representatives.

As such, in the minds of most stakeholders and representatives involved in the program, the issue is not whether the program should be continued but rather how the programs can in fact be enhanced and improved so that they will become even more successful in meeting the needs of entrepreneurs with disabilities as well as the objectives of Western Diversification.  Areas that we have specifically highlighted where there are opportunities for improvement include:

• No formal goals, shared vision or agreed upon strategy have been established to guide activities related to the programs.   As a result, perceptions regarding the goals and objectives of the program vary significantly across the delivery agencies.
• The programs have only been moderately successful in capturing the potential demand for services.  Awareness of the program appears low amongst potential clients and community stakeholders who work with the target group. 
• In comparison to the other programs such as the Community Futures general loan fund, loan default rates for the EDP and UEDI are high.  On average, the CFDCs reported an average loan default rate of 25% while the UEDI agencies reported an average default rate of 49%.  While no guidance has been provided to the delivery agencies regarding what would be a reasonable default rate, the delivery agencies proposed a target loan default rate of about 20%.
• The high loan default rates suggest that the programs need to become more effective in screening potential loan clients and in providing the pre-care and aftercare services that clients need to be successful.  Entrepreneurs with disabilities tend to require more intensive pre-care and aftercare services because they may have less business and work experience, under-developed skills and self-confidence, and more limited access to information and other resources.  In addition, the disability itself can serve as a constraint.  Many of the delivery agencies noted that the resources needed to provide extensive pre-care and especially aftercare services have not been available.  
• The existing management information systems do not provide useful information that assists in making on-going strategic decisions regarding the design and delivery of the program.  No common benchmarks or targets have been defined against which performance can be assessed.  The data which is reported is not consistent between the two programs or across UEDI delivery agencies and cannot be aggregated.
• Support for the EDP amongst CFDCs has been eroding.  Over 30% of the CFDCs have not made any EDP loans over the past three fiscal years and only 70% of the CFDCs we surveyed were in favour of the program being continued.  Some of the concerns expressed about the program include the level of demand in some regions has been low (although this can be attributed, at least in part, to not being actively marketed), resources have not been available to cover the additional costs associated with precare and aftercare, and many of CFDC managers were not supportive of the transition which occurred in 2001 when WD changed from providing EDP operating funding for individual CFDCs to providing funding through the provincial associations.  However, should the program model be improved, all but two of the CFDCs indicated that they would be interested in continuing to deliver the program.

Recommendation for Action

The core recommendation of this report is that a formal strategy should be prepared for the EDP and UEDI that defines the key elements and approach for the programs going forward.  Reflecting the results of the evaluation, further market research, and input from those involved in delivering the programs, the strategy should define:

• What is the mission and goals of the program? A fundamental issue is to determine where the program should be positioned along the risk spectrum, which will have implications for target loan default rates as well as the level of pre-care and aftercare services that will need to be provided.
• What key services will be provided and how will those services contribute towards achievement of the goals?  The most common recommendation provided by the delivery organizations to improve the effectiveness of the program is to increase the level of pre-care and aftercare support.  Such support is considered critical to the survival and development of the businesses and has a major impact on the ability of clients to repay their loans. 
• What is the relationship of the program to other resources in the community?  A few delivery agencies have been very successful in developing relationships with other programs that provide specific types support to entrepreneurs with disabilities such as income supports, assistive devices, training, technology and small loan funds.  A key strategy in any renewed program structure, therefore, may be to become more proactive in identifying and developing resources (e.g. HRSDC, provincial government programs and private sector partnerships) that can complement the support provided by Western Diversification.
• What is the marketing strategy for the program?  Some of the strategies that should be considered include developing a common brand identity for the programs that would be consistent across all delivery agencies, working to develop closer relationships with other organizations that interact with potential entrepreneurs with disabilities, undertaking joint advertising programs and promotion initiatives, establishing a common 1-800 number for the programs in each province, and establishing a common website.
• What is the operating model and funding structure?  The existing delivery structure, which utilizes local delivery agencies that also provide other services related to the target market, appears to be appropriate.   A possible change to the current model could be to maintain the position of EDP provincial coordinator but to find ways to directly apply more of the EDP funding towards the direct costs of delivering services at the local level.  A provincial program coordinator could play a very useful role in assisting in the development of the program strategy, developing and implementing marketing programs at the provincial level, networking with other organizations to access additional resources for the program and strengthen the referral networks, facilitating the sharing of information and best practices, and identifying special initiatives or projects which could be funded through other channels.  The coordinator could work with both programs, supporting the activities of the CFDCs and the UEDI delivery agencies.  
• What key indicators and methodologies will be used to collect the data needed to effectively manage the program and report on its progress?  It will be important to develop a set of performance indicators that are consistent with the specific objectives of the program and consistent across the delivery agencies.

<< previous | next >>