Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

• Clients participating in more intensive Employment Benefits and Support Measures (EBSM) initiatives also use self-serve services to a high degree. Client feedback points to the integrated nature of EBSM and other Human Resources Development Canada (HRDC) employment services and these various activities cannot be viewed in isolation.
• There is a high level of satisfaction with both staff assistance received and programs.
• This is highest for those in training and Self-Employment Assistance (SEA).
• There is high client awareness and acceptance of personal responsibility for job search and planning.
• Clients, however, feel that insufficient information was provided prior to selecting intervention. On a similar note, staff are concerned that some clients, particularly older and less experienced clients are not familiar with or can effectively use the self-serve resources.
• A vast majority of clients are aware of the federal role in funding their intervention and service.
• Approximately 92 percent of francophones are able to access services in their language of preference in designated areas.

This chapter presents a description of the EBSM interventions and other services used by EBSM participants. Participants' use of self-serve and assisted services (e.g., counselling and action plans) is examined, as well as participation in each EBSM. Related issues include satisfaction, attrition rate, availability of bilingual service and awareness of HRDC.

4.1 Self-Serve and Assisted Measures

a) Employment Resource Centres (ERCs)

To complement their intervention, the vast majority of EBSM participants (80 percent) made use of self-serve products at the Human Resources Centre of Canada (HRCC) —job bank kiosks being the most-used service. Employment Assistance Services (EAS) participants, women and visible minorities were most likely to have used the ERC services. Participation in Job Finding Clubs (JFC) and counselling was also positively related to use of ERCs. Utilization of ERCs was also high among short-term group session participants (81 percent).

Use of the ERC by non-EBSM participants (those surveyed in the comparison group) was also high at 72 percent. Specifically, 70 percent of Employment Insurance (EI) claimants who did not use EBSM, indicated they used the ERC. This, taken with EI claimants' participation in EBSM (including short-term group session), suggests that in 1997/98 (and likely beyond), that approximately 74 percent of the active EI claimants used some form of HRDC employment related intervention or service.⁷ There was not adequate information on the size of the underlying Reachback population to estimate reach into this important client population.

Perceived benefits of ERCs cited in other client research include: improved job search skills and understanding of the labour market, assistance in moving into employment and a "psychological boost" to overcome isolation and discouragement.⁸

b) Counselling and Job Finding Club

Half of the participants reported seeing a counsellor or facilitator during their participation in EBSMs. Most likely to have received counselling were older participants, women, and those who were unemployed for a longer period prior to their intervention. Access appears to have declined somewhat over time, with participants who started their intervention early in the study period being more likely to have seen a counsellor. This is likely a reflection of the evolution toward group information sessions used as counselling versus one-on-one interactions earlier in the period. Of those who received counselling, the majority were satisfied with the services they received in terms of courteousness and knowledge of the counsellor. University-educated participants provided lower ratings. Focus group participants indicated a preference for more personalized, one-on-one attention and greater follow-up. The three-week Job Finding Club is well-used, with one in five reporting that they had participated.

c) Action Plans

One of the principles of the EI legislation is to encourage participants to assume more responsibility in their interventions. Action plans, which are designed to encourage greater individual responsibility on the part of clients and commitment to their program, were used in only one-quarter of cases, according to participants. However, clients who had used an action plan reported a very high rate of completion. Individual responsibility may also be generated when participants contribute to the cost of their intervention. This was relatively little-used, however only one in ten training participants reported being asked to contribute to the cost of their training.

The relatively low apparent usage of action plans may, to some extent, reflect a lack of understanding on the part of clients about what an action plan entails. However, even the administrative data indicated that less than one-third of EBSM participants had committed to an action plan. Nevertheless, the qualitative research suggested that the message of individual responsibility had been clearly communicated to many of these participants.

4.2 Program Participation

In addition to a common core of questions related to issues such as attrition and incrementality of assistance, a separate module of program-specific questions was also designed for each EBSM. These items are intended to profile the delivery of the different programs and are presented for each EBSM below.

a) Targeted Wage Subsidy

According to surveyed TWS participants, 73 percent were aware that some of the wages for their job were paid for under the EI program. The majority of TWS participants were employed by private employers; one in ten was employed in the public sector; and six percent worked for a community/non-profit organization. An overwhelming majority of TWS participants (90 percent) worked full-time at their job (30 hours or more per week). In terms of wages, 60 percent of TWS participants were earning from $250 to $499 per week. The average weekly earnings among TWS participants were about $415.

Fifty-eight percent of TWS participants reported that they had been hired by their TWS employer after their subsidy ended, including 48 percent full-time, five percent part-time and four percent seasonal. Of those who had been hired, however, only 56 percent were working at this job at the time of the survey. Of those currently working, the majority (76 percent) were in the same job as when they were funded.

b) Self-Employment Assistance

For clients who completed SEA, the majority of new businesses started were in the following sectors:

• the personal service sector, including services industry (22 percent);
• business services (18 percent);
• percent of businesses were started in education/health/social services sector;
• 10 percent in retail trade.

The above is consistent with the vigorous growth in Ontario's Service Sector.

At the time of the survey, 81 percent of SEA participants reported that their business was still in operation (averaging 13 months operation as of the survey date). Compared to the life expectancy of small businesses, this survival rate is encouraging. Among those whose businesses had stopped operating, the most frequently mentioned reasons were: took another job (25 percent); financial problems such as not being able to raise capital (23 percent); and business not being profitable (19 percent).

A plurality of entrepreneurs noted that their weekly earnings exceeded $500 (41 percent), while 20 percent indicated a range of $250 to $500, and another 28 percent earned less than $250 per week. Average weekly earnings were $670. Nearly 29 percent of SEA operators augmented their self employment income with income from another paid job. For those working at another paid job, the SEA business represented about 23 percent of their total personal earnings.

A majority of clients (63 percent) invested $5,000 or more into their new businesses (including loans and their own equity). The average investment was about $14,000. The majority of SEA clients (84 percent) operate their business all year round. Slightly more than one-fifth of SEA clients (21 percent) reported hiring employees. Those who hired employees, reported having five employees on average.

c) Job Creation Partnership

Compared to TWS, JCP participants were more likely to work for public sector or third sector organizations for their work project. Almost half (47 percent) reported that their work project employer was a community or non-profit organization and 31 percent worked for the public sector. One in five worked for a private sector employers. The vast majority of JCP participants (96 percent) worked full-time at their work projects. While JCP participants receive EI benefits during their work project, employers may "top-up" these benefits with additional wages. According to surveyed participants, 14 percent received a wage in addition to their EI benefits at their job.

d) Training

The majority of training participants (72 percent) decided to take training without the suggestion of an employment counsellor or facilitator. The most common types of training programs in which respondents participated were computer related or information technology courses (34 percent); skilled trade occupations (14 percent); and secretarial/ clerical/administrative positions (16 percent). The majority of clients attended training either at a community college (33 percent) or from a private training institution (38 percent). Six percent of respondents mentioned a non-profit or community organization and another nine percent took training through a local board of education. Most participants were in programs between one and six months in duration. Just over one-third of training participants (35 percent) indicated that their training program included an on-the-job training component. This work placement was, on average, nine weeks in duration.

Under the principles of negotiated financial support, HRCCs decide the amount of financial support a client should receive based on the cost to implement the action plan and the expenses that clients are able to meet on their own. The evaluation results indicate that, for the time period covered in the study, only a minority of clients (10 percent) were asked or required by their employment counsellor to contribute to the cost of their training. This may be an underestimate to the extent that respondents may not have considered contributions to non-tuition related costs such as books, transportation or child care in responding to this item. Of those who had been asked to contribute to the cost of training, the average investment was just over $1,300. Considering both feepayers and training participants who had made a financial contribution toward the training, one in five (22 percent) reported that they had had to borrow to pay the cost of their training.

e) Employment Assistance Services

EAS is perhaps the most difficult of all the EBSMs to specify the types of services clients use as it is more of a funding source than a discreetly defined program. Further, EAS funds support various employment support related activities previously mentioned such as counselling, assessment, Job Search Workshops, short-term employment group sessions and ERCs, not all of which are documented .

This evaluation has focused on case-managed EAS services, which largely comprise assessment and counselling services, job finding clubs and attendance at short-term group sessions. Two-thirds of EAS participants reported that they had spoken to an employment counsellor or facilitator to help plan a back-to-work strategy (significantly higher than EBSM participants overall at 52 percent). EAS participants met with a counsellor/facilitator 5.6 times on average. The setting for the counselling session varied, with 31 percent indicating a one-on-one session, 27 percent a group session and 41 percent reporting both formats. Just over one-third (35 percent) of EAS participants participated in a Job Finding Club (higher than the one in five among EBSM participants overall). The same proportion also reported attending other job search workshops. Four in ten EAS participants developed a return-to-work action plan with their employment counsellor (higher than the 25 percent reported for EBSM participants overall).

For participants attending only short-term group sessions, the survey examined the kinds of topics covered in these sessions and their duration.⁹ Forty-three percent of participants indicated that their session had been three hours or less, while 48 percent reported that their session had lasted between four and eight hours. Seven percent of participants received a short-term group session that was longer than eight hours.

By far the most common kind of information covered in the sessions involved practical job search skills, including interview tips and job lead development. About two-thirds of participants also received information on resumé/cover letter preparation and labour market information. Fewer participants were exposed to issues such as the job loss cycle and transferable skill assessment.

As noted earlier, HRDC's coding of EAS does not identify the specific services received under EAS. Drawing from the survey feedback from EAS clients, these participants receive a broad variety of services. The vast majority of EAS participants (91 percent) reported using at least one service from the ERC. The reported participation in the various types of services offered at ERCs was substantially higher compared to other EBSM participants (with the exception of the job bank kiosks, which had a uniformly high level of use across most groups). For example, participation rates among EAS participants in ERC services included: job bank kiosk (79 percent); information on training and education programs (55 percent); labour market information (49 percent); computers (45 percent); Internet (44 percent); and office equipment (43 percent).

4.3 Completion/Attrition

The majority of clients completed the full period of their programs: 88 percent of training/fee-payer clients completed training programs with attrition at seven percent. Attrition from TWS was higher at 23 percent and 27 percent for JCP. Participants who indicated that they were not in the labour force prior to commencing their intervention had a lower completion rate for training (78 percent), but a slightly higher rate of completion for the work placement programs (TWS and JCP combined) (83 percent).

Among those who did not complete their program, the most frequently mentioned reason was to accept another job (mentioned by 60 percent of training participants who did not complete their program and 72 percent JCP participants). Among TWS participants, reasons for attrition were more likely to do with the job ending prematurely or the participants being asked to leave (32 percent).

4.4 Satisfaction

Overall, EBSM participants expressed a high degree of satisfaction with the service they received and with the specific intervention taken. Training and SEA participants were most satisfied, with about two-thirds of clients being at least very satisfied¹⁰ with their program. TWS and JCP participants were somewhat less satisfied, though the lowest ratings were with the wages or income they received while at their work placement.

Weaknesses in the delivery were evident from both the qualitative and quantitative data in terms of the initial information available to participants to help them to choose their program. These data also indicated that from the perspective of clients, the current delivery of EBSMs is not as responsive to participants with higher education and perhaps to older workers. (In terms of the latter, focus group participants believed that older workers required more personalized attention, and the survey data indicate that older workers are, in fact, more likely to receive counselling than younger participants.) Focus group participants indicated not feeling fully informed about the options available to them and stressed the importance of "connecting" with the system as quickly as possible after the start of an EI claim.

Higher satisfaction ratings on selected elements tended to increase with exposure to counselling and successful program completion. Participants who started their intervention earlier during the period under study (before April 1997) were more satisfied with the speed of services and the availability of services in one place than those who started their program later.

Short-term group session participants were asked to rate their level of satisfaction with various elements of their session. Ratings were high generally, with highest ratings being provided with resumé preparation and lower for discussions on labour market information, the job loss cycle and assessment of personal employment barriers.

4.5 Client Access to Related Services

The socio-economic profile of EBSM participants presented in section two suggests that those who took up the interventions were a reasonable cross section of the intended client population and points to effective client selection. An associated measure is the degree to which the selected clients view the offering of the various EBSM initiatives as being critical to their employment goals.

To this end, the survey examined the extent to which the support provided by HRDC was viewed as incremental — that is, to what extent would the selected participants have been able to obtain a similar service without the assistance or support of HRDC. Overall, HRDC assistance was perceived to be important or incremental by the majority of EBSM participants. This was particularly true for training and JCP participants. For training participants, 88 percent believed they could not have entered training without financial assistance, while 83 percent of JCP participants said they could not have obtained their job had JCP not been available. For fee-payers, TWS and SEA participants, incrementality of support is somewhat lower (between 60 and 70 percent).

The above is a function of a variety of factors awareness of and access to other services, personal resources available to clients, etc. These findings are consistent with EBSM clients' predominant use of HRDC employment related services noted in subsection 4.8 below.

4.6 Awareness of HRDC

The use of third parties to deliver employment services raises issues around the visibility of HRDC as the funder of EBSM. In response, efforts in HRCC offices had been undertaken to increase awareness the HRDC is delivering services. The survey measured the degree of participants' awareness of the role of HRCC in funding employment services. An overwhelming majority of participants (92 percent) were aware that HRDC had funded their program. Given the large proportion of EI claimants on EBSM interventions, this result is not surprising . Awareness was less for reachback clients.

4.7 Bilingual Service

HRDC policy on language of service follows the requirements of the Official Languages Act. Services are provided in designated areas, based on estimated demand. At the time of the evaluation HRDC had designated 16 areas to receive bilingual services.¹¹ The designation as a bilingual area carries with it the requirement to make all services available in both official languages. Beyond the requirement of the designated areas, HRDC's Ontario Region attempts to provide language access in other non-designated areas in order to proactively promote French services.

The evaluation measured the take-up of EBSM by clients who indicated French as their home language. This portion of clients were asked if they received service in the language they preferred. The demographic profile presented earlier suggests that 5 percent of clients using EBSMs had French as their first language spoken at home. For those francophone clients served in designated areas, 92 percent indicated they were able to receive service in the language they preferred. The service received was pre-intervention activities such as initial staff inquiries and information. The language clients preferred to use in an actual intervention was not measured.

4.8 Use of Other Services

EBSM participants were asked whether they used any other employment services available from other organizations. About 13 percent of participants reported that they had used other services. This was confirmed in the focus groups where most participants indicated that HRDC had been their primary source of assistance in finding work. The types of other services that respondents were most likely to have accessed were: information on the local job market; listings of job postings; and job search advice. A private company was the most commonly mentioned source of services followed by community college/university and municipal social services. Use of other services by those attending the short-term group session was comparable at 14 percent as was the percentage of EI claimants who did not use EBSMs.

Footnotes

7	This estimate of reach into the EI population is based on an average 1997/98 EI population of approximately 500,000 regular claimants and take-up rates identified in the evaluation. During the period about 390,000 EI claimants participated in some form of HRDC employment intervention or service (160,000 in EBSM initiatives, including short-term employment group sessions and 230,000 in ERC like services.
8	Sigma-3 Policy Research Inc., Employment Resource Centre Evaluation, Final Report, Submitted to Human Resources Development Canada, 1998.
9	Note that to reduce recall biases, these questions were only asked of participants in sessions since April 1, 1998.
10	Findings reported on satisfaction ratings pertain to percentage of responses within the top two rating categories on a seven-point scale unless otherwise stated.
11	See Appendix D for the list of designated HRCC office areas.