Home : Reports and Publications : Audit & Evaluation : Canada Business Service Centres in Western Canada: Evaluation Report 2002
Canadian Business Service Centres
in Western Canada:
Evaluation Report 2002
5.0 Summary and Conclusions
The Institute's evaluation addressed three issues: relevance, success and alternatives.
5.1 Relevance
Three evaluation questions framed the assessment of relevance -
are client needs being met? are Partner needs being met? and is
the mandate relevant?
Meeting Client Needs
CBSCs are meeting most client needs, although they have yet to capture
the market for operational business needs as fully as they might.
The emphasis placed on serving start-up business clients has paid dividends in terms of client ratings. New business start-up information was deemed highly useful by respondents to the Client Survey (Figure 20).
However, CBSCs (at least in the prairie provinces) are experiencing a shift in their market towards clients who own or operate an existing business. The shift is quite dramatic in some provinces. Manitoba, for instance, has gathered statistics that show a 100% increase in the number of existing businesses over the past year, confirming a four year trend. Saskatchewan and Alberta have documented a similar trend. This market segment (both clients and potential clients) demonstrated a significant interest in information and business tools related to operational issues (Figure 22), yet the extent to which they found CBSC products and services useful was relatively low compared to start-up clients (Figure 21).
Meeting Partner Needs
Basic partner needs are being met. Clients have rated the referral
service highly; 90.4% said they were accurate and a total of 94%
said they were somewhat useful, useful or very useful. Partners
themselves responded favourably when asked about the CBSCs, commenting
that the network adds benefits and that the total is greater than
the parts. However, one senior official made the point that tracking
information provided to Managing Partners is mostly activity focused
and therefore satisfies only a basic need.
Mandate
The Institute concluded that CBSCs' mandate - as practised by
the Centres - is relevant. The way it is written in the CBSC
Annual Report 2000-01, however, leaves much to be desired.
One issue that arose with respect to a review of the mandate was the meaning of the phrase 'primary source'. Some people use the term to signify pre-eminence; others interpret it in the sense that basic products and services are being delivered. The Institute concluded that the second of the two meanings accurately describes what CBSCs actually do.
Clients and potential clients reported a need for management skills as well as business-related information. The Business Link has described the situation succinctly by saying "what you know and what you do with it makes all the difference in the world." CBSCs are responding to both needs. Although the written mandate specifies information (and referrals), it fails to mention knowledge. Nor does it refer to the SME sector, which constitutes a CBSC's target market. To this extent, the written mandate does not fully describe a Centre's function.
The written mandate incorrectly describes the information (and knowledge) being disseminated by CBSCs. They deal with far more than just "federal programs, services and regulations". Apart from materials provided by provincial, municipal and private sector Partners, CBSCs have developed and promoted general business data and tools such as the Interactive Business Planner. Furthermore, the written mandate states that CBSC services and products are delivered without charge, which is manifestly wrong. Business registrations typically require payment of a processing fee, and some CBSCs are in fact charging fees for services (customized reports are an example).
5.2 Success
Three factors were considered in evaluating success - intended outputs, intended outcomes and unintended impacts.
Intended Outputs
The Institute concluded that CBSCs are producing their intended
outputs, although shifting market demands and increasingly prevalent
use of information technologies are presenting the Centres with
some challenges.
An array of information products and services are made available through the Centres, and new ones are added each year (Table 7). The change in emphasis from start-up to operational issues should encourage CBSCs to put a greater amount of effort into products and services directed at ongoing business needs of the SME sector.
Figure 25 highlights the fact that most clients and potential clients hear about CBSCs through word of mouth or networking. Only 2% were introduced to the Centres at a trade show, which suggests that this activity and associated outputs could be curtailed.
CBSCs have succeeded in producing partnership outputs, most notably by co-ordinating a total of 136 regional access centres and three satellite offices across the four western provinces (Figure 2). All Centres have renewed their operating agreements for a second term.
Intended Outcomes
CBSCs have achieved their intended outcomes, of which there are
three (Figure 4):
- SME access to timely, accurate and relevant business information and referrals;
- SME participation in business planning and analytical practices; and
- a single portal to multiple sources of information.
The Institute believes it is reasonable to assume that these three outcomes have exerted a positive influence on business literacy in the SME sector.
CBSCs identified SME access to information and knowledge as their core service. Clients generally agree that the Centres have achieved this outcome. Although not everyone was satisfied with the time it took to deliver services or products (Figure 26), over 85% of the clients responded that CBSCs helped improve their knowledge of government programs, and the quantity and quality of information available to them (Figure 27). In addition, CBSCs have recognized the strategic importance of management skills in the SME sector, and have succeeded in engaging SMEs in business planning and analytical practices.
Senior officials have recognized some of the implications of getting SMEs to adopt proven management skills. "Building more diagnostic and interactive tools helps bring clients along," but, at the same time, "a web-focus brings in a more sophisticated client, increasing the level of service demand, and the need to train our staff."
Executives have also identified a temptation to move further up the Information Services Curve (Figure 5) than is warranted by the CBSC mandate (a phenomenon described as "mandate creep"). In addition, the CBSC Mission tends to reinforce this temptation. It contains a great many phrases that reach beyond the CBSCs' mandate, and encourages CBSC staff and management to take on responsibility for outcomes that are beyond their control.
The single portal outcome, when implemented successfully, enhances SME access to information and facilitates business owner/operator participation in management practices such as planning and analysis. Figure 4 illustrates this connection by way of an arrow pointing from the single portal to the other two outcomes.
CBSCs were pioneers in developing a one-window approach to service delivery. They have, by and large, succeeded in implementation, although a number of issues arose in evaluating this outcome.
Increased awareness, a somewhat imprecise criterion, has been used over the past few years to judge whether the single portal has succeeded. Equating awareness with success has led to pessimism, because only a third of the potential client market has ever heard of CBSCs. The pessimism is unwarranted, however. The same number of SME owner/operators said they were also aware of federal government websites, but half of them failed in their attempt to identify specific sites. Clearly the CBSCs are succeeding as well as most (if not all) government departments in the visibility stakes.
Industry Canada's poll revealed another, more fundamental, characteristic of the SME market - half of the respondents agreed that they "don't really think of government as a provider of useful business information." This finding raises important implications for CBSCs. It strongly suggests, for example, that branding will not succeed unless potential clients are first prepared to hear the message.
Planning for success with respect to a single portal in today's marketplace requires dual tracking - both websites and office sites. In addition to changing client profiles, CBSCs are experiencing a dramatic shift in the way clients access their products and services. Website visits and email have increased substantially; fax-on-demand and automated telephone use have taken a corresponding dive (Figure 16). Even walk-in traffic has declined over the past three years. In absolute numbers, however, CBSCs continue to deal with large volumes of client requests, whether digital, automated or officer assisted. Advertising is required, regardless of mode of access, if CBSCs want to increase their market share.
The Institute used four criteria to assess effectiveness of the single portal. First, it determined that clients could complete most transactions with a CBSC without having to resort to other sources. This achievement is made easier by the fact that CBSC transactions tend to involve relatively simple exchanges. It is possible that CBSCs will move to the next level of transaction - bundling services (bringing several service providers together in a single, simultaneous transaction). In this scenario, CBSCs would not actually deliver the services, but would co-ordinate the activities of others who dispense expertise requiring increased levels of interaction and knowledge intensity. The Information Services Curve would then look something like Figure 33.
Figure 33: The Information Services Curve Revisited
If CBSCs take on the role of co-ordinators to bundle services - in effect acting as brokers to implement a seamless government - then their Partners would need to accelerate existing efforts in order to achieve the necessary degree of service and delivery integration.
The other three criteria all fall under the heading of customer relationship management (CRM).
The extent to which clients were tracked (for the purpose of offering more tailored services to the SME marketplace) was examined. CBSCs do not systematically track customers, and so did not satisfy this criterion. Privacy rights complicate this issue but, as the Centre for Collaborative Government has said, "privacy can, and sometimes is, used as an excuse" for inaction (CCG 2002, page 18).
The extent to which CBSCs have organized their services around client needs was examined. CBSCs more than ably satisfied this criterion.
The extent to which CBSCs have routinely taken a client's individual situation into account was examined. Again, CBSCs met requirements for this criterion.
The number of partnerships is a simplistic measure of success for the second part of the outcome - multiple sources of information. All CBSCs have succeeded in establishing relationships with a large number of government departments, as well as with other organizations; they have all maintained a wide array of useful contacts, and they have all built extensive information collections.
CBSCs have not, however, fully exploited their strategic advantage as custodians of and gatekeepers to their information (and knowledge) collections.
Three unintended impacts emerged: competition, more sophisticated clients, and loss of visibility ("the more partners we have, the harder it is for us to have an identity"). Many senior officials also raised the issue of governance in connection with the incorporation of two Centres (as a society in BC, and a not-for-profit corporation in Alberta). The Institute has not addressed this issue since it is beyond the scope of the current evaluation.