Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

Introduction and Background

This report sets out the findings of a formal, comprehensive, formative evaluation of the Assessment, Counselling and Referral Strategic Initiative (ACR/SI) under the Canada/British Columbia Agreement concerning the labour market components of the Canada/British Columbia Strategic Initiatives.

The objective of the ACR/SI was to test and demonstrate an enhanced assessment, counselling and referral system for people on income support. It was expected to improve linkages within and between employment programs, the individual on income support and the labour market.

Integration of service design and delivery was a key aspect of the innovation.

Ultimately, the pilot projects were intended to address the goal of moving individuals on income support (IISs) from dependence to independence and improve their long term attachment to the labour force.

For the period May 1995 to July 1996, the period covered by this review, there were at least 8,497 participants in ACR/SI components. Of this, there were at least 3,426 participants in Starting Points, the one common component. From 2% to 37% of all individuals on income support (IISs) in the 11 pilot project areas attended one or more ACR components.

The ACR/SI was conceived within a broad framework of partnership between the Government of Canada and the Province of British Columbia. The partnership components included the ACR Steering Group and nine local ACR Committees responsible for assessing client needs and service offerings in their communities. On this basis, the local committees prepared "service delivery plans" for local ACR pilot projects which were intended to enhance the locally available assessment, counselling and referral services for recipients of income assistance.

The purpose of the formative evaluation was to make an assessment of how well the ACR/SI was designed and managed, and what factors were likely to affect the efficiency, economy and effectiveness of the program. Also, the ACR Evaluation Steering Committee wanted answers to a list of specific questions about the design and operation of the overall Strategic Initiative and the outcomes of the individual pilot projects. (The list of questions is attached as Appendix A).

The evaluation issues described above were addressed through an examination of documents, on-site interviews, follow-up telephone interviews and the administration of a formative questionnaire to members of the nine local ACR Committees and the service providers involved in the pilot projects.

Based on the results of the data gathering and research methodology described above, detailed case study reports were prepared and circulated to the local ACR committees for comment. In this report, the results of the detailed case studies are summarized.

One of the primary outcomes of the ACR components was a movement to more group counselling and assessment, whether by government staff or contracted service provider. This move facilitated a broader and more immediate provision of service to IISs and, according to some of the commentaries received, enabled better services to be provided because of the group peer support effects that occur in group sessions.

The review of the pilot projects indicated that the components which were developed by the local committees followed the criteria of the ACR/SI very closely. The more significant challenge for the local ACR Committees was to develop enough components to address as many of the ACR criteria as possible, for the range of client needs.

It seems clear that the community-by-community needs assessment and implementation strategy was very effective for ensuring more integrated and coordinated approach to service delivery.

Involvement of the individual IIS in the action planning process was more intense and comprehensive than had been the case in previous programs at individual partner levels. It appeared that both the nature and level of involvement of clients in action planning processes were appropriate.

The ACR/SI gave rise to significant innovation in program delivery for recipients of income assistance, particularly because of the delegation of responsibility for design and development to the local community level, on a partnership basis.

Starting Points was the one ACR component common to all ACR pilot projects. Staff and clients rated the Starting Points program highly, as a helpful program in establishing client motivation.

It is unclear whether the motivational effects of Starting Points are lasting the sustainability of the outcomes for clients appears to depend on whether the expectations for follow-up programs and other outcomes are actually met. In cases where the seven month waiting period under Youth Works led to a reduction in the number of referrals, class sizes were reduced and some classes were cancelled.

A key finding was that many of the referrals into the initial round of ACR components were clients who were most likely to succeed in their training plans and employment efforts. It was therefore unclear to what extent the ACR pilot projects were meeting the needs of less motivated and more challenging recipients of income support. The question to be considered is whether special efforts should be made to seek out and involve IISs who have traditionally avoided interventions and contact with service providers in the communities.

Most of the tracking described in connection with the ACR components was short term in nature, focusing on time frames like six weeks, six months or one year after participants have completed their programs. These time frames are useful, but must be extended to make it possible to assess lasting outcomes given the fact that many individuals on income support "cycle" in and out of the welfare system over a period of months and years.

The overall set of challenges to the local ACR committees and the staff of the partner organizations were extreme, due to organizational and policy changes occurring within the partner organizations. The seven month waiting period rule under the Youth Works component of the new BC Benefits program was applied in some pilots and referral processes were changed. In others, the rule was not applied and the original ACR referral processes remained intact. This caused a strain in some of the partnership relationships for some ACR pilot projects. The most significant effect was the breakdown in the referral process which in turn resulted in fewer participants in ACR components and higher costs per participant for those components.

A lack of precision in the definition of what was expected, particularly in terms of employment outcomes in the short term was evident in the first group of pilots. If there are different outcomes expected over different time periods, (i.e., short, medium and long term) and if there are significant differences in the objectives for specific client sub-groups, (i.e., youth, single parents, new intakes, etc.) then these should be explicitly and clearly stated at the outset.

Breakdowns in the referral process were a regrettable feature of some of the initial pilots and efforts must be made to avoid a repetition of this. Clearly assigning responsibility or "ownership" for referral coupled with ensuring that the necessary resources (staff, time) are in place, is a vital first step in ensuring the success of a province-wide roll-out of ACR.

Due to the existence of formal organizations that extend beyond the ACR partnership, it should be realized that the partnership itself will necessarily be imperfect. In order to deal with the limitations of partnership successfully, it will be important to set an appropriate level of expectation for all of the parties to the ACR/SI, and particularly the local ACR committees.

In the final analysis, partnership at the local ACR committee level of the partner government organizations plus the efforts of the Federal/Provincial Steering Committee served to overcome extreme levels of diversity and produced important pilot projects in expanded services to individuals on income support. The effectiveness of the projects will be evaluated in a subsequent summative evaluation of the Assessment, Counselling and Referral Strategic Initiative.