Home : Reports and Publications : Audit & Evaluation : Canada Business Service Centres in Western Canada: Evaluation Report 2002
Canadian Business Service Centres
in Western Canada:
Evaluation Report 2002
4.0 Evaluation Findings
4.1 Client Needs
EVALUATION ISSUE |
RELEVANCE |
EVALUATION QUESTION |
Are client needs being met? |
Responsiveness to client needs was gauged by assessing levels of satisfaction, importance and usefulness attributed to CBSC services and products.
Clients responding to the survey gave the CBSCs a resounding vote of confidence when asked if they were generally satisfied with the information they received in the last twelve months. In total, 81.9% were very satisfied or satisfied (Figure 17). Women tended to be more satisfied overall than men. The Staff Survey corroborated these findings, although staff tended to be somewhat more optimistic - 93% said they felt clients were satisfied with the products they delivered.
Figure 17: Clients' General Satisfaction
By way of contrast, when potential clients were asked about satisfaction, they tended to express a lower level of contentment with their information sources than clients had reported for CBSC services and products. At best, only 12.5% of potential clients registered a 'very satisfied' rating for information services (Figure 18). CBSC clients, on the other hand, were much more likely to express themselves as being very satisfied (48%, Figure 17).
Figure 18: Potential Client Satisfaction with Information Sources
Clients also rated services and products high on the importance scale. About 83% said they received business information that was very important or important in the creation and operation of their business. Only 6.2% felt it was not very or not at all important.
Reported levels of satisfaction and importance do not, in themselves, tell the whole story. The Institute therefore examined client responses more closely to determine how satisfied clients were in connection with 'important' information. Figure 19 (next page) correlates satisfaction ratings with the degree of importance assigned by clients. Satisfaction levels are highest amongst clients who felt they received important information. Dissatisfaction levels are highest in connection with information considered to be of low or no importance.
Figure 19: Client Satisfaction Ratings Correlated with Importance of Information
To assess the usefulness of information from a client's point of view, the Institute asked both what the information had been used for, and how useful the client judged the information to be. Putting responses to the two questions into one chart provides a rudimentary gap analysis of whether clients' needs are being met (Figure 20).
Figure 20: Client Information Used and Useful
Figure 20 presents graphic proof that client needs are being met - at least for start-up clients. The emphasis that CBSCs have placed on providing services and products to clients who want to start their own business is clearly paying dividends. The Business Start-up Assistant is a good example. It provides accurate information, responds directly to client needs and is available on-line, which is fast becoming the preferred mode of access for SMEs.
The situation is not so clear with respect to clients who are already in business or are professional advisors, partly because the National Evaluation Framework Common Questions asked survey respondents to "select all that apply" from a list which gave several operational categories, but only one start-up category. When Figure 20 is simplified by aggregating all the categories pertaining to business operations, the picture changes (Figure 21).
Figure 21: Client Information Used and Useful (Figure 20 Simplified)
Figure 21 emphasizes the fact that more clients used information for business operations than for any other single purpose. They also reported that this category of information was most useful. Relatively speaking, however, when "used for" and "useful" attributions are compared, survey respondents were more impressed with start-up information than they were with operational information.
The Institute tested this finding by looking at two other factors: first, the proportion of clients in a business start-up situation, compared to those already in business; and second, the kinds of information both clients and potential clients said they were seeking in the past twelve months.
With respect to the first factor, only 31% of CBSC clients in the three prairie provinces were thinking of going into business (Figure 14, which excluded BC data for the reason given earlier). This proportion represents a change from the last CBSC evaluation, in which 44% of respondents were identified as "thinking of starting a business" (FW 1998, Table 3.1). However, it also reflects a change being experienced by the three CBSCs. Statistics gathered by the Manitoba CBSC, for instance, show a 100% increase in the number of existing businesses using its services over the past year, confirming a four year trend in the growth of such clients. Saskatchewan has also seen an increase, from 13% existing to 25% new and established businesses between 1998/99 and 2001/02. And Alberta commented last year that the "trend of people wanting to start their own businesses may be declining somewhat, however there is a steady demand from small businesses wishing to expand their operations and take their businesses to the next level" (Organization/Human Resource Review, Final Report, June 2001). At least three CBSCs are therefore trending in the same direction - a significant and growing number of clients are seeking assistance with ongoing management and marketing issues.
As to the second factor, clients were asked what type of business information they were seeking on their last contact with CBSCs. Similarly, potential clients were asked a series of questions about their information needs within the past twelve months. A rough comparison between the two sets of responses is shown in Figure 22. In both cases, it appears that small businesses focus on operational issues to a fairly significant degree.
The Institute has concluded, therefore, that CBSCs are meeting the needs of clients in existing businesses to some extent, but have yet to capture this market as fully as they might. Tools such as Talk-to-Us! could very well prove to be an important innovation in this regard, since it will allow CBSCs to respond directly to immediate client (and potential client) needs, and therefore increase the amount of attention given to operational issues.
Figure 22: Business Information Needs - Clients and Potential Clients
4.2 Partner Needs
|
EVALUATION ISSUE |
RELEVANCE |
|
EVALUATION QUESTION |
Are Partner needs being met |
Partner needs vary depending on the type of relationship that has been established. All Partners, however, need to demonstrate they are not duplicating one another's services. As one senior official said, "The public would not put up with differing government access points; we could not go back." Judging from the greater levels of satisfaction expressed by clients as compared to potential clients (Figures 17 and 18), it is fair to say that the CBSCs are helping to reduce user frustration levels and thus meet Partner needs in this respect. Figures 20 and 21 also testify to the fact that clients who use the CBSCs as a portal to other information found this facility to be a useful one.
Both staff and senior officials were asked a series of questions addressing various Partner needs (Figure 23).
Figure 23: Meeting Partner Needs - Staff and Senior Official Responses
Their replies indicate that senior officials tend to be more satisfied with current arrangements than their staff. The Partners themselves responded favourably when asked about the CBSCs. One interviewee acknowledged that "partnership is cheaper than the alternatives". Others explicitly recognized that the "process, or network, adds benefits" and that "the CBSC structure allows the total to be greater than the parts". However, it was pointed out that the "information [provided to Managing Partners] is mostly activity focused" and therefore satisfies only a basic need.
Certainly the referral system is working well. When clients were asked about the accuracy of referrals, they responded positively (Figure 24, on the next page). Approximately 53% received referrals; 91% said they were accurate; and a total of 94.5% said they were somewhat useful, useful or very useful.
Figure 24: Referrals
The Institute has concluded that CBSCs are meeting basic Partner needs, with one exception - general public awareness.
Government Partners have a need for the public to be aware of their efforts on behalf of small business, which makes up a large part of their constituent audience. For instance, the Canada - British Columbia Agreement on a Business Service Centre expressly stipulates that the Partners' collaboration must serve to "ensure recognition of governments' support and funding to this initiative amongst small business clients, service providers and the general public" (clause 2.2f).
To assess whether Partners' awareness needs have been met, the Institute asked potential clients whether they had ever heard of the CBSCs. Approximately two-thirds replied that they had not. In addition, anecdotal evidence (gathered when the Institute was conducting the Client Survey) suggests that the trade name "Canada Business Service Centre" generally has a low recognition factor. Many clients failed to respond until the CBSCs were referred to by their regional name - the Business Link, for example, or Victoria Connects. All four Centres have identified branding as an unresolved issue (this subject is discussed again in connection with the issue of outcomes).