Home : Reports and Publications : Audit & Evaluation : Evaluation of the WCBSN – Sept 2004
Executive Summary
This document presents the findings and conclusions of the evaluation of the Western Canada Business Service Network (WCBSN).
The mandate of Western Economic Diversification Canada's (WD) is to promote the development and diversification of the economy of Western Canada and to advance the interests of the West in national economic policy. These are in part achieved through WD's Western Canada Business Service Network, which includes: Community Futures Development Corporations (CFDCs), offices of the Women's Enterprise Initiative (WEIs), Canada Business Service Centres (CBSCs), Francophone Economic Development Organizations (FEDOs) and WD regional offices.
The objective of the evaluation is to assess the relevance, governance and delivery, results and impacts, monitoring and cost-effectiveness of the WCBSN. Alternatives to the current approaches were also examined.
Methodology
The evaluation findings were gathered through key informant interviews, a telephone survey and focus groups of WCBSN members.
Nineteen key informant interviews with WD (14) and other stakeholders (5) were conducted for this evaluation. In addition, relevant field data from FEDO and WEI evaluations were used to gain additional information about the WCBSN.
A telephone survey was also conducted among network representatives (CFDCs, WEIs, CBSCs, FEDOs). In total, 77 network members were surveyed. The overall response rate was 75 per cent.
Eight focus groups with network representatives were held, including two in Winnipeg, two in Regina, two in Edmonton, and two in Vancouver. A total of 55 network representatives participated in the focus groups.
Evaluation Findings
Relevance
Findings indicate that that the WCBSN has potential value with respect to reducing duplication, sharing information and providing opportunities for partnership. Most network members were not sufficiently aware of the concept of WCBSN to comment on its relevance. Some community representatives indicated that the network had potential value for sharing information particularly with respect to training and best practices.
The majority of WD key informants and some stakeholders agreed that there was some overlap between members of the network with respect to business planning, training and loan services. However, they also said that this overlap is not problematic and ensures a better reach to clients. About half of the network members work cooperatively to minimize any duplication between them.
Relationships With Other Network Members and WD
In spite of recent improvements, WD and stakeholder key informants indicate that WCBSN cannot be characterized as a network as there are few formal linkages across the organizations.
While many stakeholder key informants and focus group participants had little awareness of the WCBSN concept, the majority of survey respondents reported regular exchanges with other network partners.
Some community representatives expressed the need for increased awareness as to role of network members particularly FEDOs and WEI organizations (i.e., mandate, products, services, locality).
WEI organizations have a good relationship with CBSCs and FEDOs and some CFDCs. There are some examples of partnerships with WEI and CFDCs in rural areas.
The strength of the relationships between network members and FEDOs varied somewhat, depending on various factors such as location, region and organizational management style.
Challenges to strengthening the WCBSN include the need to maintain local responsiveness, the need for network members to operate within a number of networks, geographic distances and lack of awareness with respect to WCBSN.
The majority of key informants and survey respondents agree that there is an ongoing need for the CFDC Pan Western Association.
Many WD and community representatives note that the partnering arrangement between WD and the network members is appropriate. Some concerns were expressed with respect to the lack of clarity with respect to WD's role, the need for consistent WD support for the WCBSN and the need for enhanced communication from WD.
A major source of dissatisfaction for network members relates to financial support with only 21 per cent agreeing that WD funding met the basic operational needs of their organizations.
Success
There were mixed opinions as to the network's success. A number of key informants indicated that the network, particularly the meetings, had some impacts, mostly with respect to improved information exchange. Some key informants state that the Pan West All Partners Meetings have helped to lessen tension among the groups, thereby setting the stage for strengthened relationships. WD and stakeholder key informants also cited a number of collaboration examples.
Performance Monitoring
- Many WD and community representatives felt that the reporting process needs to be streamlined. The need to collect more follow-up and results-based information was frequently cited. Many WD and community representatives, however, recognized the challenges in collecting follow-up information. Most network members agree that the quarterly reports reflect the achievements of their organizations.
- Concerns were also raised with respect to consistency of reports. Community representatives indicate that there needs to be clearer guidelines as to what types of information to include in the reports.
Cost Effectiveness
WD key informants agree that community-based organizations, such as CFDCs, are cost effective given their use of volunteers and because they are uniquely placed to respond to local needs. Many key informants also noted that the FEDOs and WEI organizations provide better reach.
Recommendations
- Improve WD's messaging about the WCBSN to its partners. There is a need to clarify and communicate WD's role as a regional development organization and its strategic directions to its network partners, in particular regarding the partners and the WCBSN. WD should also communicate their expectations of the network members particularly as they relate to WD's three strategic priorities of entrepreneurship, innovation, and sustainable communities. WD could also consider providing training/information packages to volunteers to help them better understand the WD system, including WCBSN. WD could play an enabling and support role to increase members' awareness as to the value of the network. WD should continue to raise awareness of other network members (i.e. delivery structure, purpose and mandate, services and products offered).
- Provide consistent WD encouragement and facilitation of structured and unstructured networking opportunities among network members. WD needs to ensure that networking among the partners is encouraged in all regions to ensure maximum benefits are derived from the network's potential, ensuring that all eligible clients are served in the best possible way.
- Review the performance monitoring system with respect to clarifying and updating the reporting guidelines and the system for gathering information. WD should continue to examine the feasibility and potential of developing and implementing an on-line reporting system, with automated rollups and trend analysis. Where appropriate, partners should report against identical items.
- Pursue ongoing improvements to WCBSN meetings . WD should continue to facilitate WCBSN meetings and refine them to ensure that they are effective and meet both WD and partner needs. WD should continue to seek partner input to the agenda and shared responsibility with the partners for organizing the meetings. Discussions/workshops around network issues could be incorporated into these meetings. An action plan around these issues could then be developed for each meeting. There should be adequate monitoring and follow up of the actions plans.
- WCBSN members should seek opportunities to understand each other's mandates and activities, to refer clients as appropriate, and to share expertise among themselves and at public events. WCBSN members had varying degrees of understanding of each other's roles and responsibilities. They should actively pursue opportunities to work more closely, and to seek opportunities of finding efficiencies by working together, and of more effectively meeting the needs of any of their clients.