Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

5.1 Potential for Achievement of the ACR Strategic Initiative Objectives

During the formative evaluation, the evaluation team reviewed the design and operation of each of the ACR components in the pilot projects in relation to the ACR Strategic Initiative objectives established at the outset, to estimate whether the objectives were likely to be achieved, in whole or in part, or whether they were not likely to be achieved because of the design or operation of the components. The evaluation prepared detailed tables for each pilot project setting out the results. In this section of the report, we present our conclusions.

Exhibit 5.1 presents the ACR Strategic Initiative objectives and sub-objectives that were used to assess the individual pilot project components.

---

Exhibit 5.1: Achievement of ACR Strategic Initiative Objectives

---

1. To increase retention of IISs through the incremental stages of training by:

• actively involving participants in defining realistic and achievable career goals.
• providing the necessary counseling supports and follow-up activities.
• developing better reporting mechanisms between agencies and MoEST.
• improving the tracking of participant through training and into employment

---

2. To increase the potential for success of IISs to move through training and into employment by:

• ensuring that training plans reflect individual and labour market needs.
• ensuring that training plans incorporate employability skills and computer literacy skills.
• ensuring that training plans incorporate training that is portable and transferable.

---

3. To increase the career planning/management skills of IISs enabling them independently to build on career goals by:

• ensuring career planning courses/services include several components (e.g., LMI, career exploration, personal development, vocational assessment) and are based on "best practices".
• ensuring curriculum and teaching methods promote individual skill development.

---

4. To increase access to assessment, counseling and referral services for IISs by:

• increasing availability of career planning services at appropriate stages.
• establishing appropriate referral mechanisms between ministries, agencies and institutions.
• increasing the availability of information on training resources and employment patterns, etc.
• ensuring better integration and coordination of community services.

---

In general, all of the objectives and sub-objectives listed above are being addressed in varying degrees by the ACR components of the pilot projects, although it is extremely difficult to evaluate whether there are strengths and weaknesses in the overall set of components because of the extent of diversity and the varying local requirements.

With respect to the first objective of increasing retention of IISs through incremental stages of training, it was not possible to determine whether retention rates had actually improved in the formative evaluation because this data was not available. However, with respect to the individual sub-objectives that were intended to contribute to this outcome, it is clear that many of the ACR components actively involved participants in defining career goals. The processes that were put in place included various kinds of information and activities intended to ensure these goals are more realistic and achievable. The ACR components did attempt to provide necessary counselling supports and follow-up activities. Extensive efforts were made by the contracted service providers to improve reporting mechanisms and make the necessary information available to other partner agencies and MoEST. Short term tracking mechanisms were implemented by many of the service providers but little progress was made on implementing longer-term tracking mechanisms intended to follow the IIS over a longer timeframe, or in relation to the tendency of many IISs to cycle in and out of the system over several years.

With respect to the second objective of increasing the potential for success of IISs to move through training and into employment, most of the efforts of the local ACR committees and service providers focused on attempting to achieve this objective. Almost every component of the ACR/SI produced plans which reflected individual needs and many also included some assessment of labour market requirements. Training plans generally incorporated employability skills as required and some of the components addressed computer literacy skills where resources were available.

It was not possible to identify ACR components which addressed the sub-objective of ensuring that training plans incorporate training that is portable and transferable. In this respect, it is likely that this sub-objective is being addressed to a greater extent than was evident in the analysis but it is likely that the concepts of "portability" and "transferability" have not been sufficiently defined in operational terms to highlight them in the design of specific components.

Much of the effort of the local ACR committees and service providers focused on the objective of increasing the career planning/management skills of IISs, thus enabling them to independently build on career goals. A number of career planning and decision-making components were implemented and many of these addressed career exploration, personal development, and vocational assessment. Some also included labour market information.

It was unclear to the evaluation team whether "best practices" were incorporated into the components or whether a process existed for identifying best practices based on the experience of individual components. Some interviewees suggested that all of the practices that were incorporated into their ACR pilot projects were "best practices". However, there was little support for this other than the individual opinions of those involved. These practices had not been proven using valid output measures or other processes of consensus building around "best practices" concepts.

Virtually every ACR component focused on ensuring that curriculum and teaching methods promoted individual skill development.

The final objective to be addressed by the ACR pilot projects was to increase access to assessment, counselling and referral services for IISs. Career planning services were made more available as a result of the ACR/SI. However, difficulties were encountered in establishing appropriate referral mechanisms between ministries, agencies and institutions. Key difficulties have been discussed previously in the Starting Points section of this report. The ACR pilot components did increase the availability of information on training resources and employment patterns. Finally, some progress was made in ensuring better information and coordination of community services, although, linkages to local economic development agencies and employers were not improved to the extent that may be possible, subject to resource limitations of local staff.

In conclusion, all of the four main objectives listed in Exhibit 5.1 above are being addressed in a relatively well-balanced manner by the overall set of ACR components. Key issues have arisen in the referral mechanisms and tracking and monitoring processes

Our evaluation has resulted in a preliminary conclusion that the ACR/SI components which may be the most essential and useful to IISs, are as follows:

Starting Points-each person interviewed agreed that a needs assessment is useful to most IISs before they embark on further training or seek employment. Starting Points appears to be a good framework for providing these services.

It should be noted that the Starting Points objectives do not address the long-term goal of assisting IISs in moving from welfare to work. Long-term attachment to the workforce may occur at some future point and Starting Points may contribute to this accomplishment, but, it is quite unlikely that Starting Points will suffice for the majority of IISs in their efforts to become employed.

Orientation/Information Session-two of the pilots (West End and Burnaby) provide a 1.5 hour Orientation Session. This session appears to provide useful information to IISs regarding services and resources available, as well as IIS responsibilities.

Career Planning-some form of career planning is being provided in almost every pilot project. This service may be necessary for IISs who require further assistance in developing a viable career plan.

Support Workers-many IISs require assistance in completing training programs or maintaining employment. Support workers provide this needed service to IISs. Services can include coaching, counselling or employment services.

Career Information Resource Centre-a centre which provides resource information and employment services such as access to facsimiles, telephones, computers, photocopiers, etc. appears to be useful to IISs.

Work experience-work experience is beneficial to IISs who are interested in pursuing a particular career and wish to acquire a practical understanding of such work prior to embarking upon further occupational training commitments. Work experience also provides the IISs with contacts in their chosen field.

Given the overall challenges described above, the local ACR committees and Steering Group have achieved a strong partnership in creating the 11 pilot projects and serving the substantial number of individuals on income support identified in this report.

Several conclusions should be drawn from the experience:

• The ACR partnership worked very well to produce integrated, locally specific plans to meet the needs of IISs in the pilot project communities.
• Due to the existence of formal organizations that extend beyond the ACR partnership, it should be realized that the partnership itself will necessarily be imperfect. Each of the individual partners has their own core missions, priorities, policies and ongoing challenges which will take priority over the requirements of the partnership from time to time.
• In order to deal with the limitations of partnership successfully, it will be important to set an appropriate level of expectation for all of the parties to the ACR/SI, and particularly the local ACR committees.
• The effectiveness of a partnership like the one established for ACR will always be subject to the effectiveness of the individual partner organizations so that if one or more of the partners has difficulty with decision-making processes or effective delegation of responsibility and authority, these difficulties will be reflected in the partnership.
• Insofar as a substantial harmonization of policy, procedure, language and strategy are required in order to have an effective partnership such as ACR, these factors have to be secured through a strong executive sponsorship of the partnership at the individual partner organization level.
• In overall terms, although there were many challenges to be overcome, much progress was made by the partners to the ACR/SI in working together to innovate and enhance services available to IISs in the project communities.

The Client and Agency Profile and Service Delivery Planning processes of the ACR pilot projects appeared to be very effective in ensuring a more integrated and coordinated approach to program delivery in the communities.

The pilot projects reflected the criteria established for ACR quite closely with challenges in rural communities to meet as many of the criteria as possible.

Subject to budget and resource constraints the pilot projects were able to close a number of the gaps that existed between the needs for, and availability of resources in the pilot project communities, although some of the more significant problems like pervasive unemployment were not really addressed.

For purposes of the ACR/SI, the process of identifying best practices should become a standard feature of ACR administration.

Best practices can be established by routinely reviewing alternative processes to determine which process is meeting objectives most efficiently and effectively. For a given process, field level information and insights can be developed at the operational level, followed by further refinement through the application of expertise, research findings and facilitation. Next, through additional piloting, the effectiveness of the particular process or component can be further tested. With this information and the consensus of knowledgeable practitioners, a determination of whether a best practice has been identified can be made.