Summative Evaluation of the Community Skills Centres - British Columbia - May 1999

Summative Evaluation of the Community Skills Centres - British Columbia

View Whole Report

• The Community Skills Centres of British Columbia
• Evaluation Purpose and Approach
• Summary of Evaluative Conclusions on Objectives Achievement
• Overall Evaluative Conclusions

Community Skills Centres (CSCs) are a joint endeavour of the federal government (through Human Resources Development Canada, as part of its Strategic Initiative program) and the province of British Columbia (through the Ministry of Advanced Education, Training and Technology, as part of its Skills Now programming). The CSCs were designed to assist individuals, employers, and communities in meeting their needs for innovative, flexible training to help meet with labour market adjustment. The CSCs were to be community-driven and community managed. They were to add value to existing training resources, especially through the use of electronic technology. The funding period for the Strategic Initiative (SI) was five years (ending with fiscal year 1998/99), and by the end of this time the CSCs were to be independent of government funding for operational costs.

Twenty CSCs were fully operational at the time of the summative evaluation. Some had been open for nearly five years and some as little as less than two years. They are structured as community-based non-profit organizations, and are guided by community Boards. The CSCs typically facilitate or deliver under a contracting arrangement a very wide range of training programming. The six categories under which these fall are: Advanced Technology; Skills Upgrade, Vocational, Technical, Trade; Business Management, Professional Development; Adult Basic Education; Job Readiness, Career Preparation; and "other" (i.e., management services, targeted wage subsidy programming).

Clients include a wide range of trainees, including employees of companies who have contracted for the training services, individuals on benefits programs such as Employment Insurance and Income Assistance, self-paying members of the community. There are often drop-in services provided, such as sessions for orientation to the Internet. Most services are provided on-site but there is a trend toward mobile or satellite services in outlying communities in the more rural areas.

The evaluation highlighted three features of CSC programming. One is the individual variation from community to community in CSC size, and operational structure. There are as few as four staff in one CSC (which is a recently operational one) and as many as 32 in another. (These figures cannot be precise, given the changing human resources situation in CSCs and the ambiguity of categories for identifying "staff." But the sheer range is notable.)

The second feature is the breadth of programming as well as the commonality of many of the programs offered. There is a central core of educational and skills upgrading – ABE, basic computer literacy, and advanced technology training and upgrading. Then there is a staggering array of other courses, usually developed in a process of needs identification with individuals, community groups, or employers. Courses include bark beetle control, many kinds of safety skills upgrading, management skills, and agricultural skills enhancement (calving, commodities pricing, manure management).

Finally, because partnership is a major objective of the CSCs and their relationship with the public training sector is an important issue, it was noted that fully 19 of the 20 CSCs have some form of partnership with either the local School District and/or a post-secondary institution. (The latter can be at the college or university level, and may or may not be with a local branch. Some CSCs have the partnership with a college or university whose main campus is elsewhere, whether or not they have a local campus.) Thirteen of the CSCs have partnerships with School Districts and 16 with a post-secondary institution.

There were three stages to the evaluation of the CSCs, ending in a summative evaluation, upon which this brief is based. The summative evaluation was to:

• Determine the extent to which the CSC initiative is meeting its original objectives;
• Determine the impacts and effects that CSCs have had on their main client groups...;
• Assess whether funding for CSCs is being used effectively;
• Assess the degree to which CSCs have developed new and innovative approaches to meet community needs;
• Assess the degree to which CSCs have promoted partnerships...and the degree to which federal/provincial partnerships have assisted the CSC initiative in meeting its objectives;
• Assess other performance data on CSC activities and operations.

A combination of research methods, drawing on quantitative and qualitative data was used for the summative evaluation. The research components consisted of:

• A case study approach, involving a full spectrum of interviews and program document reviews for each of the 20 CSCs in the province. Twelve of the CSCs were visited in person by evaluation team members. The rest were covered to the same degree, but by telephone.
• A telephone survey of 58 employers and 122 employees who used the training services of the CSCs.
• A survey of key respondents involved in the development and implementation of the CSC programming at an Initiative-wide level. They included representatives from the government partners, and from the public and private post-secondary system, as well as labour, at senior levels.

In all, 576 individuals were interviewed and numerous documents were reviewed and analyzed. Most data analysis was content analysis, but some secondary statistical analysis was done of program usage and financial records.

The CSCs were designed to achieve four objectives at the operational level. The evaluative findings on their success are as follows:

Objective 1:

To act as a focal point for bringing together community resources to increase access to training...; adding values to existing programs...; providing an access point for labour market information ...[and[.. addresses the training needs of targeted client groups.

Since the formative evaluation, the CSCs as a group can be seen as having made considerable progress on meeting this objective as a whole. As a group they are very much involved in activities that address this multi-faceted objective. They have established both informal and formal means of bringing together a wide range of stakeholders in their respective communities. They are very active in establishing partnerships with other trainers, in both the private and public sectors. However, in a few communities the various training sectors have not been able to come to a resolution of the perceived or actual potential for competition among each other for training opportunities.

The CSCs as a group definitely add value to existing programs and services. The primary means include the use of technology to address the needs of a range of target groups for self-paced, highly flexible training modes. Another means is the structure of operations of CSCs themselves, in that the facilities are open on a very broad schedule, they are very "non-traditional" in terms of the learning environment, and are provided by staff that are uniformly regarded as highly skilled in dealing with consumers. The CSCs have considerably strengthened the provision of labour market information (LMI) since the formative evaluation. Partly this is because there is a great deal more LMI available, in much more accessible and user-friendly formats. Partly this achievement also reflects the fact that CSCs have been awarded contracts for delivering LMI, as both the federal and provincial governments have turned over more of this to community deliverers. More than half of the CSCs participated in the Enhanced LMI component of Strategic Initiatives as well, having received contracts for developing and/or delivering this information to their communities.

Objective 2:

To increase community input and decision-making regarding training and adjustment issues by developing CSC training plans... and establishing consultation...processes.

The CSCs have made progress since the formative evaluation in achieving this objective, but there remains room for improvement. Boards continue to be the central conduit for eliciting community input into CSC training plans. This is a strength, but it also can be limiting if the Board is either too diverse to have the concentration of skills a given CSC needs to meet its own goals for training needs identification, development and delivery; or if the Board has members who feel they must put the interests of the organization they represent ahead of the interests of the CSC. No matter how well the Board or CSC as a whole may operate to develop community-based training plans, because many of the CSCs are located in economically vulnerable areas, it is difficult to plan for advancing economic change through training when the economic opportunities may be decidedly limited. The changes in federal/provincial policies and funding allocations have also changed the context in which CSCs operate and plan. There has been substantial shrinkage of direct and indirect government funding amounts since the CSC/SI was instituted.

Objective 3:

To increase competitiveness of business and industry in the global marketplace and individuals in the labour market by providing services to encourage the use of technology...[for training]...

The CSCs can be seen as meeting this objective to a considerable degree. Though it is asking rather too much of any one program or of an individual CSC to increase "global" competitiveness of industry, the employers who have utilized CSC training services speak very highly of the value of the training for increasing the skill levels of their employee base.

Employees and current training participants interviewed also were very positive about the training they received and the manner with which CSC staff interacted with them. Employees and current participants felt their skills were enhanced and they also reported a more positive sense of self-confidence and a stronger valuation of training as something to include in their future.

The CSCs rely most heavily on the use of computer-based technology for the training they deliver, and the range of training content is impressive. The technology is adapted to the needs of the consumer, whether employer, employee or other individual. All of these activities indicate a very pro-active and creative use by the CSCs of electronic technology to advance training in their communities.

Objective 4

To achieve financial independence from government funding of operational costs by generating revenue and leveraging private sector funds; creating partnerships and collaborating with the community; and administering funds responsibly.

This objective highlights an inherent dilemma of the programming model for the CSCs, in that CSCs are both to strengthen economically vulnerable communities through bridging, brokering, etc., and yet are expected to generate sufficient revenue to become financially independent. Generally speaking, the CSCs which approach independence devote themselves virtually entirely to the development and delivery of services on a revenue generation basis. If they serve individuals who cannot themselves pay for service (i.e., those on IA, EI, etc.) the CSC does so as part of a contract held with a ministry/department to provide services to the client group.

Only one CSC is largely financially independent, though it continues to utilize government (matching) funds for subsidizing some of its revenue generating training. Perhaps two more CSCs have the potential to become permanently independent of government funding for operational costs. Fully three-quarters of the CSCs do not anticipate that they can achieve this objective though most feel that if approximately 30% of their operating costs were government funded they could continue to operate, though on a more modest scale in some cases.

The larger context of being located in small, rural communities, many of which are resource reliant in a time of severe downturn, also substantially constrains the ability of CSCs to generate revenue, no matter how willing or able they are to do so. The changes in government roles in relation to support of training and employment-related programming, and the uncertainties of the role of CSCs in the face of the implementation of the provincial Training Accord also are inhibiting factors to development of revenue generating activities.

Thus, few CSCs have achieved this objective, but the impediments for the most part can be attributed to factors external to their individual operations.

The overall effects of the CSC/SI have been positive. They generally have created a distinctive niche in their communities and have indeed enhanced community involvement and control over training resources. The outcome for the communities has been positive and the outcome for the wide range of programming users has also been very valuable. Human resources have been enhanced at the company and individual levels, and the CSCs are often a real source of pride and increased confidence for those who have been involved with them.

There have been strains created by the establishment of CSCs in some communities. The concerns of other training providers, in both public and private sectors were pronounced at the inception of the CSC/SI, but these have been considerably allayed, especially at the local level through active consultation and collaboration over the last several years. Some unease remains, but it has not been an insuperable impediment to the operations of the CSCs or their community partners.

In terms of whether the CSC model addresses significant needs and whether its overall rationale is supported by the evaluative evidence, it seems clear that the CSCs would not be approached by the community to develop and provide the diverse training the CSCs have come to deliver, nor could the CSCs successfully market the range of courses they generate based on their formal and informal needs identification, unless the previously unmet need was out there. Therefore, it appears that there is a definite need for the type of training the CSCs facilitate – a need for the means or processes the CSC employs and for the diversity of content they offer.

Furthermore, the CSCs operations are very efficient. They rely heavily on (unpaid) needs identification, networking, and governance from their volunteer boards. They may have two staff or 18, but they are generally very economical in their approach to human resource management and to facility use. Most staff are on contracts, the facilities are not luxurious, and the overheads are monitored carefully.

Given these evaluative findings, there is considerable justification for a CSC-type model as a means of meeting the larger policy and programmatic goals of providing community-driven training resources that can meet the distinctive needs for innovative, flexible training – using electronic technologies wherever possible – that will strengthen the economic well-being of the community and its individual members.