Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

In this chapter, we present findings on the success of the PBMs to date. Two main kinds of outcomes are discussed. In the first part of this chapter, we observe the extent to which the Cda/NB LMDA results targets have been met. In the remaining parts of this Chapter, we present evidence on impacts of the PBMs on participants, employers and communities.

It should be noted that the measures presented in this chapter are preliminary only. Moreover, the short-term nature of the period of investigation in this formative evaluation may favour interventions such as Partners and Entrepreneurs in which employment outcomes are often immediate, and which may be biased against interventions such as job search training assistance which have a longer "gestation" period. More definitive measures will be presented in a summative evaluation, when there will be more time available to detect impacts.

7.1 Results Targets Attainment

The Cda/NB LMDA, like all provincial labour market agreements, specifies results targets for EI reforms in three areas: participants returning to work, active EI claimants served,¹⁶ and EI benefits savings. The first column of Exhibit 7.1 presents the exact targets for the 1997/98 fiscal year for all active benefits and measures under the Cda/NB LMDA. In the second column are presented the actual results as in the 1998 EI Monitoring and Assessment Report.¹⁷ In the third to fifth columns, we use survey data collected for this evaluation to "validate" and update the first of the results (returns to work)¹⁸ and administrative data to look at the other two result indicators. Note that column two figures cover a period up to March 31, 1998, which is why we show employment rates for two different periods in columns three and four: before April 1, 1998 and on or after April 1, 1998. Note as well that total column includes amounts for those whose intervention end date was unknown, which is why it does not represent a sum of the previous two columns.

Note further that the figures in the first two columns of Exhibit 7.1 apply to all EI active benefits and measures under the Cda/NB LMDA including services such as individual counselling interviews and group services¹⁹ which EI clients can use independently of the benefits and measures being evaluated. On the other hand, the figures in the third column apply only to the six benefits and measures under study in this evaluation. For this reason, these figures should not be compared.

View EXHIBIT 7.1

Return to Work Results

With respect to the employment results target (participants returning to work following intervention), column two of panel 1 of Exhibit 7.1 indicates that the 1998 EI Monitoring and Assessment Report found that about 70 per cent of this target was attained (5546/7947). We confirmed this figure using the HRDC Results File administrative data. In the third and fourth columns, we present the results of our validation and update exercise of this result, which consisted of applying, for the two different time periods, (i) three employment rates derived from the survey results, to (ii) the count of the total number of PBM participants from the administrative data (8,881 up to April 1, 1998 and 8,768 on and after April 1, 1998). The three employment measures (with the values for the two different time periods in brackets) are: (1) employed at all following the intervention (71.7 and 51.9 per cent); (2) employed for 12 consecutive weeks following the intervention (70.5 and 49.2 per cent); and (3) currently employed (44 and 37.3 per cent). Note that the second employment measure — employed for 12 consecutive weeks — comes closest to the Cda/NB LMDA measure. Comparing column 2 to column 3 figures, and noting that the latter applies to a set of measures that excludes additional employment services as noted above, indicates that the New Brunswick administrative data (which the former is based on) appear to under-estimate employment outcomes.²⁰ The EI Monitoring and Assessment Report suggested that under-estimation may be due to the fact that the data systems used to track clients and the exchanging of data were not fully operational during the 1997/98 fiscal year. In the fourth column we update the employment counts for the period subsequent to the March 31, 1998 fiscal year and in the fifth column we provide the totals for the two fiscal years.

Active EI Claimant Share Results

In calculating this result, the EI Monitoring and Assessment Report included Rural Experience and EAS participants. Consequently for this evaluation report, these participants were also included (see second row of panel 2 in Exhibit 7.1). The majority of participants in these two programs are reachback clients. The Canada/New Brunswick Labour Market Development Agreement actually states that this accountability result is to be applied to Provincial Benefits only, and not to Measures. Since Rural Experience is funded under Research and Innovation (a Measure) and EAS is a Measure, this accountability result should be calculated excluding these two programs, as presented in the first row of panel 2 in Exhibit 7.1.

With respect to the active claimant share target, the first row of panel 2 in Exhibit 7.1 indicates that this target, too, has not been met. (In column 2, there is no entry because the 1998 EI Monitoring and Assessment Report provided only a national proportion result of 82.5 per cent, the reason being that administrative data systems were said to have not been in place to adequately measure this proportion.) In the 1997/98 fiscal year, 56 per cent of the participants in the Provincial Benefits under study were active EI claimants, based on population administrative data.²¹ Though once again we cannot really compare this result to the target, the proportion who were active claimants in the counselling and information sessions not under study would have to have been exceptionally high for the total share to be on target. Column 4 shows that the share has fallen to 41 per cent in the 1998/99 fiscal year, and the overall percentage of active EI claimants was 47 per cent. Management decisions about the targeting of programs will necessarily impact on the degree to which this (and other) results targets are achieved.

Unpaid EI Benefits Results

To check the third target — unpaid EI benefits — we first show the results presented in the 1998 EI Monitoring and Assessment Report in the third panel, second column of Exhibit 7.1. This report found that for all Cda/NB LMDA benefits and measures, there were $13.1 million in unpaid EI benefits in the 1997/98 fiscal year, which was barely one-half of the target. Once again, we confirmed this figure on the basis of the Results File.

As part of this evaluation, we computed unpaid EI benefits for PBMs as well but excluded services that were used by respondents exclusive of the PBMs under study. The results of this analysis for the 1997/98 and 1998/99 fiscal years and the total for both fiscal years including those with an unknown EI end date, are presented in the third, fourth, and fifth columns, respectively, of the third panel of Exhibit 7.1. In each cell of those columns in that panel, we show two rows of figures, corresponding to two different approaches used to compute unpaid EI benefits. The first is the "standard" HRIB approach based simply on the difference between paid and eligible EI benefits of claimants returning to work before the end of their benefit period.²² The second is the alternative ARC method,²³ which attempts to link unpaid EI benefits explicitly to a return to work. The main difference between the two approaches, then, is that the standard method attributes all unpaid EI benefits to a return to work, regardless of the reason why EI benefits were not paid. In contrast, under the ARC approach, the computation is linked to a return to work (RTW). As it turns out, the ARC method yields a smaller amount of savings because not all unpaid EI benefits can be attributed to a return to work. The standard approach can incorrectly identify a RTW when benefits are reduced for non-work reasons such as severance pay received, vacation pay, disentitlement, overpayments, etc..

To elaborate, under the first approach, unpaid EI benefits were computed by summing up the amount of savings for each participant, as indicated by a variable on the HRIB dataset received from HRDC. This amount was computed (by HRDC) for each individual as the difference between (1) EI benefits eligible and (2) EI benefits paid, which in turn was computed as the benefit rate multiplied by the difference between (1) the entitlement weeks and (2) the difference between the EI weeks paid and weeks paid in the RTW period. For each individual on the file, we compared the dates of each EI claim period to their interventions' start and end dates. Only EI claims overlapping with an intervention under study were included in the calculation; those claims not associated with counselling and group information sessions which were used exclusively by clients did not figure in the calculations. The majority of LMDA participants were single-intervention users, so for these individuals the savings derived from the EI claim were easily linked to the "most recent" intervention. In the case of multiple-intervention users, unpaid benefits associated with EI claims related to the different interventions were added up.

In the total population of 17,548 participants on file, 4,626 clients had EI claims linked to an intervention (i.e., excluding counselling and information session interventions as discussed above) and had reported unpaid EI benefits greater than $0. The total savings for these participants amounted to $14,145,474.

However, this is a figure that covers the better part of two fiscal years, i.e., participants whose intervention ended in either the 1997/98 or the 1998/99 fiscal year. Thus, in row one of columns three and four of the third panel of Exhibit 7.1, we show the savings realized separately for clients from the two fiscal years with known intervention end dates. We show in column three that only $7,933,245 were unpaid EI benefits realized in 1997/98. Adding in the some $62,000 savings (not shown) from individuals with a missing intervention EI end date, still brings the total savings for the 1997/98 fiscal year to less than $8 million. Row one of the fourth column indicates that about $6.15 million were realized in 1998/99 fiscal year, which brings the cumulative total to the some $14.1 million (column 5) including those with unknown intervention end dates as indicated above.

The alternative method of calculating unpaid EI benefits, as suggested by ARC, was based on the HRDC Status Vector file, which contains information on EI eligibility and benefits paid, and the Results File, which contains return to work (RTW) flags. The results of this exercise are presented in the second row of columns three to five in the third panel of Exhibit 7.1. In general terms, savings were computed as the difference between (1) EI benefits paid after the RTW date, and (2) remaining eligible EI benefits. In some cases, wages paid are such that EI benefits could be paid but at a reduced rate.

In order to calculate unpaid EI benefits under the ARC approach, the following detailed steps were taken for each individual:

• the RTW date was subtracted from each of the individual's EI-claims' start date, with the condition that the former date not be later than the latter date;
• the difference between the RTW and EI-claim start dates in step 1 (call it the "pre-period") was compared to the length of entitlement period related to this claim and, where the length of this pre-period was greater than the entitlement period itself, a value 0 was ascribed to the pre-period;
• the pre-period (in weeks) was then multiplied by the respective EI benefit rate (expressed as dollars per week), thus arriving at the total amount of EI benefits paid at the start of the RTW;
• the amount of benefits paid at the time of the RTW was further subtracted from the total EI eligible benefits, thus arriving at the amount of remaining EI benefits in the entitlement period;
• benefits paid up until the RTW were then subtracted from the total amount of benefits paid during the claim (i.e., including benefits paid after the RTW), thus calculating amount of EI benefits paid after the RTW; and
• finally, EI benefits paid after the start of the RTW (from step 5) were subtracted from the remaining eligible EI benefits (from step 4) to arrive at EI benefits unpaid for each individual.

The calculation was initially performed on the 6,144 persons on the Results File who participated in the six PBMs under study (i.e., excluding services that were used exclusive of those PBMs) and matched with the participant population from the main file. Missing information reduced the number of cases to 5,972, however. Further, another 3,800 individuals were assigned zero savings because the date of the RTW was beyond the entitlement period. This left 2,172 individuals who had "valid" dates, i.e., a RTW flag that was dated before the end of their entitlement period, and therefore potentially who could generate savings. Summing the computed savings across these individuals produced a total of $11,117,584 in unpaid EI benefits. Assigning zero savings to clients with negative savings (i.e. clients whose EI benefits had been extended), increases the total amount of savings calculated using ARC method to $11,260,748.

Once again, we differentiated clients for the 1997/98 and 1998/99 fiscal years because the targets specified for this evaluation were for 1997/98. Row two of the fourth column of the third panel of Exhibit 7.1 indicates that just over $6.5 million in unpaid EI benefits were realized in 1997/98 for the six PBMs under study. Adding in the $318,000 for persons whose EI intervention end date is not known (not shown) brings the savings up to barely $7 million for this fiscal year, even more short of target than using the HRIB approach. The fourth and fifth columns of row two of the third panel of Exhibit 7.1 indicate that savings from the 1998/99 fiscal year brought the total saved over the two fiscal years to the $11.3 million²⁴ as mentioned above.

Additional Return to Work Results

To amplify the return to work results, we present Exhibit 7.2, which compares incidence of 12 consecutive weeks of post-intervention employment across program types and claimant status. Though the employment results target was not specified according to program type and claimant status and though it was specified in terms of an absolute number not a percentage incidence, it is instructive to observe how the rate varies across the different groups.²⁵ First, the results indicate that active EI claimants were more likely to occupy jobs lasting 12 consecutive weeks than reachbacks, particularly in SLG and EAS programs. Second, the employment rate was particularly high for Partners and Entrepreneurs.

7.2 Impacts on Participants

Impacts on clients were measured using two approaches: (1) clients' and employers' subjective ratings of the importance of the help they received in obtaining employment; and (2) clients' objective labour-market outcomes, as revealed by their actual status. Both the subjective ratings and objective outcomes of PBM clients are compared with the rated and actual outcomes for the comparison group.

In interpreting the results based on participant survey data as presented in this chapter, the following should be noted. First, the results for participants overall are weighted by sex, age and program type; the participant results presented by program type are weighted by age and sex only; and the comparison group results are weighted by age, sex and the theoretical time of intervention. Note also that the sample size presented in the exhibits is unweighted. If second, the analysis of participant survey response data consisted of cross-tabulating post-intervention labour-market outcomes by type of intervention (i.e., benefit or measure) and by the characteristics of the participants. Participant outcomes are also compared according to EI-claimant and reachback status and compared to comparison group outcomes. Note, however, that, because a comparison group was not selected for the reachback group, the comparison between participants and non-participants is restricted to PBM participants who were active EI claimants. Note also that, while we observe differences in outcomes by characteristics of participants and program type, in such bivariate comparisons we are unable to control for pre-existing differences among participants and between the participant and comparison groups. For this reason, multivariate analysis was conducted where there were controls for these variables; the results of this analysis are presented in Chapter Eight.

The client outcomes we examined are in the following areas: employment, joblessness, job-search behaviour, income-support utilization, and attitudes and skills. As in Chapter Six, participant survey results for all participants and cross-tabulated by claimant status (active versus reachback) are weighted by sex, age and program type; participant survey results cross-tabulated by program type and comparison group survey are weighted by age and sex only. Employer survey results as they pertain to participants are unweighted.

Employment Outcomes

Employment for clients is the ultimate goal of PBM assistance. Employment impacts were measured in this evaluation using evidence gathered in the focus groups and case studies, as well as the employer and client surveys. In this section, we first present qualitative evidence on employment outcomes and then discuss a large number of employment outcomes based on survey evidence as follows: perceived importance of intervention on employment, employment rates, retention rates, non-completion, employment stability, employment status, hours and wages. Included with this discussion are results on related measures of retention, non-completion, and incrementality.

Qualitative Evidence on Employment Impacts

In the focus groups with clients, the general conclusion drawn was that, without the PBMs the clients had participated in, they would not have been in the jobs they were in or have started the businesses they started under the programs. In the focus groups with employers, it was observed that a number of clients had been hired by participating employers following the wage subsidy. It was also thought that many would in the future benefit in this way from improved skills and confidence resulting from their participation in the PBMs. However, a drawback is a lack of jobs in the local economy.

Positive employment outcomes were also identified in at least four of the case studies conducted for this evaluation. In both of the case studies of Adjustment Services Initiative (ASI) "down-side" adjustment situations (plant closings), we found that the committee formed and its recommendations were of benefit to the workers concerned. In one, committee deliberations and recommendations resulted in the establishment of an on-site employment resource centre and in about one-half of affected workers finding alternative employment, some of which was in an area different from their previous career. In addition, several workers returned to get their high-school certificates, which should result in employment in the future. In the other "down-side" adjustment situation, the ASI committee's work led to the establishment of an on-site drop-in centre. About eight in 10 of affected workers were able to return to the workforce.

In two other case studies, employment outcomes for clients were also identified. First, in the SLG case study, benefits from the intervention came in the form of jobs for several clients, some with better jobs. Second, in a case study of an ASI community development adjustment committee, the fact that the participating organization had become more business oriented as a result of the committee's work enabled it to be more socially proactive and to place clients from the community in greater numbers and diversity.

Perceived Importance of Intervention on Employment

The first approach to measuring employment outcomes in the participant survey was to directly ask participants employed at the time of the survey to indicate the extent to which they thought the employment benefit or measure enabled them to obtain their job. Respondents were asked to rate the importance on a seven-point scale ranging from one representing "not at all important" up to seven representing "extremely important", with responses of 6 and 7 re-grouped and identified as "important' in the results presented below. This question was addressed to only those who had a job in the post-intervention period and to those who were not "continuers", i.e., wage-subsidy participants who were hired by their host employer at the end of the subsidy and Entrepreneur participants whose business continued beyond the end of the assistance period. It was assumed that these continuers would have responded "extremely important" to this question, had they been asked it. Including these responses yields the results shown in Exhibit 7.3.

The results indicate that just over one-half of participants indicated (or it was inferred) that their assistance was important (reporting six or seven on a seven-point scale) in getting their job. Entrepreneur and Partners played an important role in obtaining their current job. For the other PBMs, between 38 and 51 per cent of respondents said the program had been helpful in getting their job. Not shown is the fact that the proportion saying the intervention played an important role in obtaining the current or most recent job hardly varies at all by the sex, age and education level of the participant, apart from a somewhat smaller proportion of those with a university degree. Comparing EI claimant to reachbacks (Exhibit 7.4), little difference is observed, implying equal perceived program effectiveness for these two client groups. Comparisons were not made with the comparison group because too few of this group made use of employment assistance services.

Employment Rates

The second way employment impacts were measured using participant-survey results was to observe the actual employment outcomes for participants following their intervention. Exhibit 7.5 presents the employment results for PBM participants and for the comparison group. Overall, the results of this bivariate analysis signal at least an initial advantage for PBM participants on some indicators compared to non-participants (though this positive result is tested more rigorously in the multivariate analysis).

The first panel of Exhibit 7.5 indicates that a majority (61 per cent) of participants was employed at some time since completing the intervention or since the reference date in the case of the comparison group. Again, there was little variation across age, sex, and education sub-groups (not shown). However, there were differences by program type. Of all the program groups, Entrepreneur and Partner participants were, by far, the most likely (96 and 88 per cent) to have had a job in the post-program period. These large proportions are largely attributable to the fact that 87 per cent of Entrepreneur clients reported that their business continued beyond the assistance period and that 57 per cent of Partners clients were working in jobs offered by their host employer (not shown). The results across other benefits and measures are similar (in the 60-65 per cent range), with the exception of SLG participants who were the least likely (49 per cent) to have had a job in the post-intervention period. EI claimant participants were more likely (68 per cent) to have been employed than reachbacks (55 per cent), which is not surprising given the former's more recent experience in the labour market. They were also more likely to be employed than comparison group members were (52 per cent), implying some advantage for PBM participants over participants in other employment assistance interventions.

Other results (not shown) indicate that employment outcomes vary by claimant status within program types. These results indicate that EI claimants are much more likely to be employed following participation in Job Action than reachback participants in Job Action, and more likely to be in jobs lasting at least 12 consecutive weeks. On the other hand, participation in SLG and EAS appears to benefit reachbacks much more than EI claimants.

Employment Stability

There were two measures of employment stability in participant survey results. First, participants were also asked if they had occupied a job that lasted (at least) 12 weeks since leaving the program, as an indicator of the durability or stability of employment. Returning to Exhibit 7.5, panel 2 indicates that the majority (59 per cent) of those who had worked found a job that lasted at least three months. Once again, Entrepreneur and Partners participants are distinguished from other participants by the fact that they were most likely (88 and 83 per cent) to have had fairly stable jobs and SLG participants by the fact that they were least likely (52 per cent) to occupy such jobs. The other interventions had similar levels of stability (54-63 per cent). EI claimants were more likely to have had stable jobs compared to reachbacks (71 versus 49 per cent), and there was some advantage for EI claimants in this respect over the comparison group (71 versus 60 per cent). Finally, older workers were more likely to have experienced 12 consecutive weeks of employment than workers in other age groups were (not shown).

View EXHIBIT 7.5

The second indicator of stability is the number of employers that participants had (panel 3 of Exhibit 7.5). The results indicate that a majority of participants across all programs who had at least one job since the intervention had only one employer during the post-program period. Once again, it was participants in Entrepreneur and Partners were most likely to have had just one employer (91 and 80 per cent). EAS participants were the least likely to have had just one employer (46 per cent). Other survey results (not shown) indicate little variation in these figures across age, sex and education participant groups. Finally, Exhibit 7.5 indicates that there was little difference between EI claimants on the one hand and reachback and comparison group participants on the other.

Employment Status Outcomes

In the participant survey, participants were asked about their employment status at two points in time: (1) in the first week following the intervention, and (2) currently, i.e., at the time of the survey. The results in panel one of Exhibit 7.6 indicate that, overall, about one-third (35 per cent) of PBM participants were employed or self-employed in the immediate term following the end of the program, but the proportion varied considerably across benefits and measures. The highest proportions were recorded for Partners (with 60 per cent in full-time jobs) and Entrepreneur (with 71 per cent in self-employment). These participants were most likely to be employed at this time as a result of their continuous employment with their employer (Partners) or sustained operation of their business (Entrepreneur). Employment shares were similar across the rest of the other PBMs, except for the somewhat higher incidence of part-time employment for EAS participants (13 per cent). As for unemployment share, the results indicate that almost half of participants (44 per cent) were unemployed and looking for work in the week after their program ended. This share varied considerably across PBMs, ranging from just 13 per cent for Entrepreneur to as high as 69 per cent for Rural Experience. There were few differences of note between EI claimants and reachbacks.

There were some noticeable differences in the first week, post-intervention employment status by age, and education (not shown). For example, younger participants (under 30) were the least likely to be out of the labour force (unemployed and not looking for work). Also, men were more likely to be employed full-time and women part-time, reflecting overall labour force patterns. And by education, the results indicate that those with no more than a high-school certificate were the most likely to be unemployed and looking for work, while those with a college education were most likely to be employed full-time. Once again, this reflects overall patterns by education level.

For current employment status (panel 2 of Exhibit 7.6), patterns by PBM program types are similar to the first-week status following intervention. Once again, the most positive results are for Entrepreneur and Partners participants. Only nine per cent of Entrepreneur participants were unemployed and looking for work at the time of the survey compared to rates as high as 46 per cent for Job Action and 55 per cent for Rural Experience. Among the other program groups, the percentage unemployed is similar (in the 24-29 per cent range). Partners participants have the highest incidence of full-time employment.

View EXHIBIT 7.6

Current employment status results are positive for both PBM participants and the comparison group. At the time of the interview, 44 per cent of EI claimant participants were employed (self-employed, full-time employed or part-time employed) compared to 50 per cent for the comparison group, but the incidence of full-time employment was higher among EI-claimant participants (33 versus 25 per cent). While the percentage who were unemployed and looking for work was similar between the two groups (about one-third), the proportion going to school was somewhat greater for EI claimants than for the comparison group (10 versus four per cent).

Comparing the post-intervention employment status results in the first week (Panel 1 of Exhibit 7.6) to that at the time of the survey (Panel 2) indicates some improvement over the post-intervention period. All employment shares (full-time, part-time and self-) rose somewhat between the two points in time. For example, the proportion in a full-time job increased from 25 to 29 per cent. Similarly, the proportion that is unemployed and looking for a job fell between the first two points in time (from 44 to 32 per cent). These results are similar for EI claimants and reachbacks. By program type, however, some full-time employment declines were recorded, e.g., the decline in the incidence of full-time employment for Partners participants (60 to 54 per cent).

Pre-/Post-Intervention Comparisons of Employment Status

Another way employment outcomes were measured using the participant survey results was to observe changes in employment status from before the intervention to after. In Exhibit 7.7 the selected results of cross-tabulating pre-intervention employment status with post-intervention first-week and current employment status are presented. The results show that, although the largest post-intervention proportions for each pre-intervention employment status group are also the largest proportions for the pre-intervention employment status group, some interesting changes did occur as follows:

• Among the unemployed in the week before entering the program, one-quarter (23 per cent in the first week and 26 per cent currently) ended up working full-time and six and nine per cent working part-time in the first week and currently, respectively. Over one-half (54 per cent) remained unemployed, and this proportion fell to 41 per cent over the post-intervention period.
• Among those who had been working part-time before entering the program, about one-fifth had changed their status to full-time after the intervention (20 per cent in the first job and 19 per cent currently). Note, as well, the increase in school attendance for this group over the post-intervention period (two to 21 per cent), an indication that these participants had considered education as a means of improving their employment status.
• A large share of the pre-intervention not-in-the labour force group ended up in full-time jobs. Note again that this proportion increased over the period following intervention (19 to 25 per cent), while the proportion unemployed fell (27 to 13 per cent). Interestingly, the decline in the percentage unemployed is commensurate with the increase in the combined percentages of participants in full-time employment and in school.
• Among those who were students before intervention, a large proportion ended up employed full-time (19 per cent) in the first week following the intervention. This proportion rose considerably to 45 per cent currently indicating a delayed reaction to intervention. Similarly, the proportion unemployed fell from 36 to 13 per cent between the first week and currently.

One other finding (not shown) is that about 70 per cent of those who were not or only somewhat interested in entering the labour before the intervention reported that they were interested after the intervention.

Characteristics of Current or Most Recent Job

In this section, the characteristics of participants' current job (at the time of the survey) or, if without a job at the time of the survey, the latest job (following the intervention) are briefly examined. Excluded were those who never had a job since the intervention or reference date. Tabular results are presented in Appendix D.

First, just under one-fifth (19 per cent) of PBM participants had returned after the intervention to the employer they had prior to it (among those who had a job before), with SLG participants being the most likely to do so (31 per cent). EI claimants were considerably more likely (27 per cent) to do so than reachback participants (11 per cent). Interestingly, however, EI claimants were less likely to return to their previous employer than members of the comparison group (32 per cent) were.

Second, of respondents in their current or most recent job, the majority was working full-time hours. Almost nine in 10 participants worked more than 30 hours per week.

Third, overall, a little over one-half (54 per cent) of PBM participants were working in year-round jobs at the time of the survey. Entrepreneur and Partners participants were most likely to be in year-round jobs (85 and 75 per cent). Conversely, Job Action and particularly Rural Experience participants were more likely to be employed on a casual/contract basis (32 and 50 per cent) or in seasonal jobs (24 and 18 per cent) than those in other PBMs, except for EAS participants who also had a high incidence of seasonal employment (20 per cent).

Fourth, there was considerable variation in weekly earnings across program groups. Mean gross weekly earnings for PBM participants in participants' current/most recent job ranged from as low as $342 for Job Action participants, $366 for Rural Experience participants and $372 for EAS participants to as high as $514 for SLG participants, almost one-half (47 per cent) of whom earned $500 or more a week (Partners participants earned an average of $415 per week and Entrepreneur participants had average weekly earnings of $472). Median wages indicate a similar ranking of programs though with smaller differences among programs. This can largely be explained by the educational profile of clients in the respective program, as discussed in the previous chapter. SLG clients tend to be more highly educated while Job Action clients tend to have no more than a high-school education. EI claimants ($479) earned somewhat more than the reachback group ($401) and the comparison group ($458); however, comparing the median wages across those three groups indicates no difference. Finally, reflecting patterns observed in the labour force overall, female clients earned less than males did, older clients earned more than those in other age groups did, and weekly pay rose with education level (not shown).

View EXHIBIT 7.7

Retention

A further measure of employment success is the extent to which wage-subsidy participants were retained by their host employers at the end of the subsidy.²⁶ In the participant survey, participants who remained with their host employers until the completion of the wage subsidy were asked this question.²⁷ The responses to this question, presented in Exhibit 7.8, indicate that just under one-half of participants (46 per cent) were retained by their host employers. Given the nature of the respective interventions, it is not unexpected that the retention rate was highest for Partners participants (66 per cent).²⁸ In contrast, Job Action and Rural Experience are, by definition, project-based interventions, with little expectation of participants continuing beyond the wage-subsidy period. Indeed, as the exhibit indicates, the proportions retained are only about one-quarter (28 and 23 per cent) for these two interventions. In addition, it is noted that the retention rate is somewhat higher for EI claimants compared to reachback clients (52 versus 45 per cent). Finally, retention declines with age and is similar for males and females (not shown).

Similarly, as the second row of Exhibit 7.8 indicates, Partners participants were most likely to be hired into the same job as they had during the intervention. Of those who were hired by their host employers, over 90 per cent of Partner and Job Action participants were hired into the same job compared to 86 per cent overall. The rate for Rural Experience participants was in the three-quarters range. EI claimants were somewhat more likely to be hired back into the same jobs than reachbacks (92 versus 83 per cent).

Finally, in the third panel of Exhibit 7.8 we observe the types of jobs participants in the different wage-subsidy interventions were hired into. The results indicate that Partners participants were much more likely (81 per cent) to be hired into year-round full-time jobs than participants in the other wage subsidies (42 and 28 per cent). There was little difference between EI claimants and reachbacks in this respect.

What are the reasons participants are not being hired? For this, we turned to the employer survey results. Among the reasons offered by employers for having not hired all workers (or intending not to hire all workers), Exhibit 7.9 indicates that the dominant reason is a lack of resources on the part of employers participating in Job Action and Rural Experience (37 and 30 per cent, respectively). Results not shown indicate that lack of resources was cited most frequently by larger, non-profit, municipal, older, year-round, and low-growth organizations. By industry, this reason was most frequently mentioned by businesses in arts, entertainment and recreation; education, health and social services; and transportation and distribution services. For Partners employers, the main reason is the fact that the jobs were only temporary or seasonal (16 per cent), which also was an important reason for Rural Experience employers (27 per cent). The latter also frequently mentioned lack of work (19 per cent) as a reason for not hiring participants after the subsidy. Lack of work and temporary work reasons were most likely to be mentioned by seasonal businesses, with no clear link with other organizational characteristics (not shown). Other reasons were mentioned typically by only a fraction of employers, with "workers not amenable to training and their attitude" being the most frequently mentioned among these reasons, particularly for Partners employers.

Non-completion

Large numbers of participants not completing their term under the wage subsidy would be an indication of lack of success. As Exhibit 7.10 indicates, about one-third of employers participating in wage-subsidy programs who responded to the employer survey lost some participants before the full duration of the subsidy. Among employers who experienced losses, the most frequent reason for non-completion cited by employers was that the participant voluntarily quit (36 per cent), which was the case for all three programs, but particularly for Rural Experience (42 per cent). Other prominent reasons cited were as follows: found another job and personal problems/illness, particularly for Job Action (20 per cent); not amenable to training/poor attitude, particularly for Partners (21 per cent); incompetence, for Partners and Rural Experience employers (17 and 16 per cent); and wrong skills for the job, particularly for Partners (17 per cent). Among these reasons, it should be said that finding another job would be considered a "good'' reason and not attributable necessarily to the intervention or the experience itself.

Incrementality

It was clearly stated in program documentation that employers are to hire workers under wage subsidy only into incremental jobs, i.e., into jobs that they would not have filled otherwise. In the employer survey, employers were asked to indicate what they would have done if the wage subsidy had not been available. Summing up the responses indicating incrementality, presented in Exhibit 7.11, indicates that about three-quarters (74 per cent) of wage-subsidy transactions were incremental. Incrementality was greatest for Job Action (85 per cent) and lowest for Partners (66 per cent). At the other extreme, about one-fifth (19 per cent) reported that they would have hired someone into the positions regardless of the wage subsidy, implying complete non-incrementality. This was the case particularly for Partners employers.

There were some interesting patterns in incrementality across organizational characteristics (not shown). The percentage of employers who said they would not have hired anybody without the wage subsidy rose gradually by firm size, was particularly high for municipal organizations, rose with the age of the business, and was higher for seasonal and low-growth businesses. The fact that incrementality appeared to be greater for larger firms may be explained by the fact that such firms tend to hire more workers under the programs thereby increasing the chances that at least some of the jobs were incremental. By industry, incrementality was particularly high for public administration and education, health, and social services and particularly low for traditional services, business services and manufacturing.

Another incrementality measure from the employer survey was that provided by a question on whether or not the wage subsidy enabled employers to hire faster than they would have without the subsidy. The results presented in the last row of Exhibit 7.11, indicate that about two-thirds of participating employers reported that the wage subsidy speeded up their hiring. This proportion did not vary greatly across wage-subsidy types, but, as results not shown indicate, tended to be higher in manufacturing (81 per cent) and businesses less than two years old (81 per cent).

Yet another measure of incrementality was that provided by an employer survey question on hiring behaviour after the end of the wage subsidy. Employers were asked to indicate whether or not, after the wage subsidy, they had hired, or intended to hire, other (non-program) workers into the jobs that were occupied by the participant during the wage-subsidy period. This could be viewed as a measure of incrementality (as well as satisfaction with wage-subsidy experience). However, as mentioned above with regard to retention rates, this would be the case only for Partners because Rural Experience and Job Action are project-based benefits and not intended to lead to lasting jobs. The responses to this question (not shown) indicate that just over one-fifth (20 per cent) of Partners employers have done so, or intend to do so, with little variation across wage-subsidy types. Participating government and municipal organizations, older businesses (over 30 years), and health, education and social service organizations were the least likely to have responded in the affirmative to this question.

Joblessness and Job Search Outcomes

In this section, we review the evidence on four post-intervention outcomes: number of weeks jobless, number of weeks searching while jobless, job-search activity, and interest in entering the labour force. "Jobless" individuals are here defined as those officially unemployed (i.e., out of work and looking for work) plus those not in the labour force.

Duration of Jobless Spells

Panel 1 of Exhibit 7.12 presents survey results with respect to the duration of jobless spells following intervention, scaled by the time since the intervention, as reported by respondents. The mean percentage of time not working during the post-program period for PBM participants was 29.1 per cent and the median was almost one-fifth (19.4 per cent). There was much variation across the program groups. Job Action and Rural

Experience participants experienced the most joblessness at over 35 per cent of the time since the intervention (mean and median), with less than 15 per cent of each group experiencing no joblessness. It is also noted that almost three in ten Rural Experience participants (29 per cent) were jobless in one-half of the weeks since their intervention. At the other extreme were Entrepreneur participants who on average were jobless for only eight per cent of the time (median = zero) and who were the most likely to experience no joblessness (78 per cent) and the least likely (six per cent) to have experienced joblessness for over 50 per cent of the post-funding period. This is largely attributable to the fact that most of these persons continued on in their self-employment business after funding had ended. As for differences by sociodemographic characteristics, results (not shown) indicate that the mean percentage of time not working rises with age, declines with education level, and is higher for female clients than for male clients.

Members of the reachback (mean=31.3 per cent) and comparison (mean=45.1 per cent) groups experienced considerably more joblessness than EI claimants did (mean=26.7 per cent). This is also apparent from the differences in the proportions experiencing joblessness for over 50 per cent of the time (27 and 40 per cent versus 18 per cent, respectively).

Duration of Job Search

In this section, we present results for job search while jobless in the post-program period, which is considered to be a positive outcome of the intervention. It should be noted that responses to the respective question were re-coded to include as zero those who were not jobless in the post-program period. Also, as above, responses were scaled by weeks since the intervention.

The percentage of time looking for work during the post-program period (panel 2 of Exhibit 7.12) parallels the above results. Entrepreneur participants were the least likely to look for work while unemployed in terms of mean and median percentage of the time (6.4 and zero per cent) and also had the highest proportion (91 per cent) who were searching for work while unemployed for one-quarter or less of the time since the intervention. Partners and EAS participants also demonstrated good results in this respect. Job Action and Rural Experience participants looked for work for the longest periods of time (means of over 27 per cent of the time). Differences by age, sex and education are in the same direction but muted compared to those observed above for weeks of joblessness (not shown). Finally, reachbacks tended to look somewhat longer for work while jobless than EI claimants. Comparison group members tended to look even longer (16.1 versus 9.3 per cent, medians).

View EXHIBIT 7.12

Job-Search Activity

Those who reported actively looking for work during the post-program period (both those who obtained a job and those who did not) were asked to specify the types of job search methods they used. Exhibit 7.13 presents the responses to this question. The top four methods of job search were the same for all respondent groups, including all program types as well as among EI claimants, reachbacks and the comparison group. Sending résumés or applications, making personal visits to the employer, using a resource centre or job bank, and word of mouth (asking friends and family) were the most important methods of jobs search for all groups. Less than five per cent of respondents used the Internet or participated in a job finding club for their job search.

Although there was little variation in job-search activity across the benefits and measures, some differences did arise, including the following:

• EAS participants were more likely to send resumes and applications compared to other participants and were also more likely than others to rely on newspapers.
• Rural Experience participants were by far the most likely to use word of mouth, family and friends to search for a job.
• Job Action participants, too, were likely to rely on word of mouth.

Some interesting differences in job-search activity also arose according to the sex, age and education level of participants (not shown). The proportions going to a resource centre, making telephone inquiries with employers and using word of mouth all rose with age, but the proportion sending out resumes declined with age. Men and women tended to pursue the same activities, apart from greater relative numbers of women sending out resumes than men did. Finally, those with no more than a high school certificate appear to be much more reluctant to send out resumes than those with a college certificate or a university degree (40 per cent versus 61 and 60 per cent).

View EXHIBIT 7.13

Interest in Entering Labour Force

Finally, those who were jobless (i.e., unemployed or out of the labour force) at the time of the interview were asked to rate their level of interest in entering the labour force in the next 12 months. Interest was gauged on a seven-point scale, ranging from one equal to "not at all interested" up to seven equal to "extremely interested". The re-grouped responses to this question, presented in Exhibit 7.14, indicate that a vast majority (over 90 per cent) of PBM participants were reported to be quite interested (6 and 7 on the scale) in entering the labour force. Other results (not shown) indicate that interest level did not vary much by age, sex and education (not shown); across program types, there were only minor differences as well. As for claim status, differences again were not great, with reachbacks indicating somewhat lower levels of interest in entering the labour force. Parallel ratings by the comparison group show more modest interest in entering the labour force, with only three in four indicating being interested in entering the labour force.

Utilization of Income Support

Another major indicator of program effectiveness is the extent to which interventions have reduced participants' reliance on income support such as employment insurance (EI) and social assistance (SA). Note that the results on EI use are based on the administrative data, which permit the capture of a return to EI after the period of EI associated with the intervention is completed. This is a true measure of EI use that is not possible using the survey results.²⁹

Exhibit 7.15 indicates that about one-quarter (24 per cent) of participants started a new spell of EI following program intervention. Reflecting the pattern of employment experiences observed above, there was wide range in EI use, with as little as three per cent among Entrepreneur participants having received EI to about one-third (32 per cent) of Partners clients and over one-half of Job Action and Rural Experience clients. Reflecting labour market experience patterns, women and younger persons were less likely to have collected EI than those in other sex and age groups (not shown). On average, as panel 2 indicates, participants received EI in eight per cent of the weeks following the intervention, translating to about one month on an annualized basis. Rural Experience and Job Action participants had the highest mean percentage of weeks receiving EI (22.9 and 17.1 per cent).

EI claimants were less likely than reachbacks to have received EI in the post-intervention period (21 versus 32 per cent) and for less of the time following the intervention (5.7 versus 11.1 per cent, means). However, non-participants were somewhat more likely (23 per cent) to have been on EI in the post-intervention period but for a smaller percentage of weeks (3.9 per cent mean).

As for reliance on SA, only a small minority (nine per cent overall) of participants reported receiving social assistance during the post-program period (panel 3 of Exhibit 7.15). Post-program SA-use was highest among Job Action (31 per cent) and EAS (17 per cent) participants and lowest among Partners, Entrepreneur and SLG participants. The pattern was similar in terms of the percentage of weeks receiving SA, with Job Action participants standing out in terms of mean percentage of weeks received SA (12.4 per cent) and the percentage (25 per cent, not shown) receiving SA for more than five per cent of the weeks. As for differences between reachbacks and EI claimants, it is observed that the former, as expected, were much more likely to have used it (15 versus five per cent) and during a greater percentage of the weeks since the intervention (six versus 1.6 per cent, means), though there were no differences in medians between the two groups. Comparison group members were less likely than EI clients to rely on SA (one per cent), and for a smaller percentage of the weeks since the intervention (0.3 versus 1.6 per cent). Not shown is the fact that female clients were twice as likely as male clients to have received SA after their intervention and in a greater percentage of weeks.

View EXHIBIT 7.15

Attitudes, Training and Skills

In this section, the results from the focus groups, case studies, union survey, and employer survey conducted for this evaluation are presented. First, in the focus groups with clients, it was observed that their participation had increased their skills and experience, which, down the road, would increase their chances for employment. In the focus groups with employers, it was generally believed, too, that the program had enhanced the clients' skills and experience. In addition, clients had benefited psychologically from their participation in terms of feeling that "they had accomplished something" and in terms of acquiring "a sense of dignity", some for the first time. Some clients returned to school following their participation, which would in the long run benefit them.

In the case studies, many of the outcomes identified involved improved attitudes and skills for clients. In one "down-side" adjustment situation, committee deliberations and recommendations resulted in not only direct employment for affected workers but in several workers' returning to get their high-school certificates. Increased labour-market intelligence and job-search and resume-writing skills were other positive effects for the workers. In the other "down-side" adjustment situation, the ASI committee's work led to restored optimism and positively altered attitudes to work and learning among affected workers. In the SLG case study conducted for this evaluation benefits from the intervention came in the form of not only jobs for clients but also improved attitudes to work and learning, increased labour-market knowledge, changed attitudes regarding the benefits of contributing to one's own human-capital development, and reduced drop-out rates in schools. A factor contributing to these positive outcomes was the flexibility of the program. In one of the ASI community development case studies, we found that the participating organizations' working closely with the schools enabled the development of more labour-market skills for clients.

In the three EAS case studies, the benefits identified appeared to be mainly psychological. Common outcomes identified were improved attitudes toward work and learning, a strengthened ability to set career goals, and increased confidence to apply for jobs. Impacts of more practical nature included increased job-search skills and job-specific skills. Identified success factors in these case studies were once again local flexibility but also the emphasis on placement.

In the union survey, the two union representatives who responded were asked to assess the impacts of the intervention on participants. Unfortunately, the one respondent who felt qualified to answer these questions was the one representative who did not know which PBM intervention the employer had participated in. Thus, the client outcomes we discuss here cannot be associated with any particular intervention. At any rate, the one representative who responded said that the intervention had positively impacted on clients' attitudes to work, attitudes to learning and training, confidence, attitudes to authority, and job-specific skills. Little or no impact was detected on basic reading and writing skills, computer skills, personal management skills, teamwork skills, job-search skills, and overall level of preparedness and job readiness. The respondent was adamant in his or her belief that the participant was job ready at the end of the intervention. The union representatives identified no negative impacts of the intervention on participants.

In the employer survey, there were two sets of questions on human capital development: training and skills. With respect to training, it would be one of the goals of the wage subsidy benefits to increase the skills of participants so that they may enter employment. In the survey, employers were asked to indicate what sorts of training and related assistance they provided their wage-subsidy participants. The results presented in Exhibit 7.16, which shows the proportions of employers providing the different types of training assistance, indicate that job-specific (83 per cent) and orientation training (76 per cent) were the most common sorts of training provided, which was true for all wage subsidy benefits.³⁰ In another tier were task rotation, personal skills training, mentoring, and job shadowing, all of which were provided by about one-half of employers. Unique among the activities listed in the exhibit is fringe benefits (36 per cent), which is not really associated with human capital development. The least frequently provided assistance was computer training and career counselling (particularly in Rural Experience), and job-search advice (particularly in Partners).

Not shown is the fact that the provision of training is strongly associated with the size of the organization. The fact that the incidence of training rises steeply with organization size clearly reflects long-established patterns from the general labour-market literature. Other organizational characteristics associated with the provision of training are being a municipal organization, being two years or younger, being a year-round business, and already training large proportions of the organization's workforce.

To what extent were skills of participants increased as a result of the training provided and the experience itself? On the employer survey, employers were asked to judge the impact of the wage-subsidy experience on participants. Results from this question presented in Exhibit 7.17 indicate the proportion of employers saying participants' experience under the wage subsidy had a positive impact on them (5, 6 or 7 on a 7-point scale where 1=extremely negative impact and 7=extremely positive impact). The results indicate a majority of employers believed the wage subsidy had a beneficial impact on clients. There was little variation in reported impacts across skills and wage-subsidy benefits, with 75-89 per cent of employers believing that participants benefited from their experience. The only differences of any note are those between Job Action and Partners in attitudes to learning and training (89 versus 74 per cent). Not shown is the fact that few employers believed the experience had a negative impact on clients. The results from another measure, which is also not shown, indicate that 60 per cent of employers believed that participants were job ready following the wage subsidy, with the biggest difference occurring between Partners (67 per cent) and Job Action (54 per cent). Finally, other results (not shown) indicate that skill and attitude impacts did not vary greatly with organization size, sector, age, and industry.

7.3 Impacts on Employers

In this section, we present (unweighted) impact results mainly from the employer survey, as well as the focus groups, case studies, and union survey. In the focus groups with employers, the evidence gathered indicated that employers' participation in the wage-subsidy interventions had benefited their organizations in both quantitative terms (increased production) and qualitative terms (enhanced service delivery).

In contrast to the ASI "down-side" adjustment committees reviewed above which benefited mainly the affected workers, the two case studies of "up-side" employment-development adjustment situations resulted in positive impacts for the participating organization. One case study found that the committee's work contributed to a change in operation style, to increased management and marketing knowledge on the part of the owners, and to the introduction of the use of the Internet in the organization. Increased sales and employment resulted from these changes, but also increased costs in covering a portion of the professional fees of the committee chair. Similarly, in the other "up-side" ASI case study, it was found that the committee led to a large number of positive outcomes for the organization, including a business plan, marketing strategy, reorganized finances, increased confidence in the company's direction, and increased awareness of the existence of business support services. Importantly, the recommended course of action led to increased access to, and sales in, new markets. An identified success factor was "buy-in" from senior management.

In the union survey, the two union respondents were asked to evaluate the impact of the interventions on the organization and its existing workforce. For the most part, the responses of the two representatives were similar. Both thought the intervention had a positive impact on employee morale and workload. One of the two respondents (although not necessarily the same one) also thought the respective program positively affected productivity, employment levels, interest in providing job opportunities to those in need, and the ability of the organization to evaluate new potential employees. Little or no impact was detected on existing employees' wages, advancement opportunities, the organizations' output and revenues, product or service quality, openings for new workers, hiring practices, and relations between management and employees. The union representatives identified no negative impacts of the intervention on the organization and its staff.

In the employer survey, there were two sets of impact questions: one on perceived impacts and one on the costs of participating. First, employers were asked to evaluate the impact of the wage subsidy on their organization along a number of dimensions using a seven-point scale, ranging from 1 (extremely negative impact) to 7 (extremely positive impact). The results presented in Exhibit 7.18 show the percentage of employers reporting 5, 6, or 7, indicating a positive impact. (Few employers believed the wage subsidy had a negative impact (1, 2, or 3) on the organization and the percentages indicating a neutral impact (4) were in the mid-teens for all wage subsidy types). The results indicate that a majority of employers thought that the wage subsidy benefited their organization. The greatest support was reported for benefiting the organization overall (82-86 per cent) and, among specific aspects of the organization, it appears that the workload and productivity of existing employees benefited the most (71-76 per cent). There was little variation across different wage-subsidy measures, with the greatest gap being between Partners and Rural Experience in terms of enhancing the organization's ability to evaluate potential new employees (85 versus 62 per cent).

For this set of measures, the age and the sector of the organization appeared to matter (not shown). For all organizational impact measures, the percentage indicating a positive impact clearly fell with the age of business. And for all measures but workload/ productivity, government organizations were least likely to benefit from their participation in the wage subsidy program. A few differences by industry also emerged: no more than one-half of participating organizations in construction, business services, and public administration indicated that their ability to evaluate new potential employees benefited from their participation in the program.

The second organization-impact measure was provided by a survey question on employer costs of participation. In the employer survey, employers were asked to identify areas where they had incurred significant costs resulting from their participation in the program. The results from this question presented in Exhibit 7.19 indicate that significant costs were incurred most prevalently in two areas: supervision/training and wages (39 and 31 per cent). This was true of all wage-subsidy programs, but particularly with respect to Partners, where close to one half (49 and 45 per cent) of employers incurred significant costs in these two areas. The third most frequently cited significant cost overall (15 per cent) was with respect to benefits; again this was higher for employers participating in Partners (23 per cent). Note that 41 per cent of employers reported that they did not incur any significant costs as a result of participating in the wage subsidy.

As for differences by organizational characteristics, private sector businesses were, by far, the most likely to say that wages represented a significant cost to their participation in the wage subsidy. Businesses with five or more employees most frequently mentioned the time and money spent supervising participants as a significant cost of participation. Non-profit employers were most likely to say that they had experienced no significant costs while participating in the program.

7.4 Impacts on Communities

Qualitative evidence gathered in the key informant interviews, focus groups, and case studies was used as the basis for the discussion in this section. Generally speaking, key informants were not able to detect benefits at the community level. One key informant observed that the local unemployment rate had not fallen during the time the new PBMs have been in place, while another observed that the lack of consultation at the local community level may even have led to negative effects for the community (though what these were was not specified). Still another believed that there was a plethora of employment programs designed to move social assistance recipients (SARs) off social assistance and onto Employment Insurance and that LMDA had not changed this situation. On the other hand, another observer thought that the drop in the number of SARs was attributable partially to the LMDA, but also to the improvement in the economy.

In the focus groups with employers, it was thought that the programs have resulted in enhanced infrastructure for the community. This will serve to attract tourism to the community and thus benefit it in the long term, as well as to expand the activities that the participating community organization can carry out. It was also noted that, during projects where local infrastructure was being built, there had been spin-offs for the local community, e.g., local suppliers had benefited from the material purchases.

The two ASI community development case studies found that communities definitely benefited from the ASI committee's work. In one community, an action plan was recommended and adopted by the community. This led to the decision to open an RCMP station in the community, to hire an economic development manager, to purchase land for an industrial park, and to begin negotiations with major employers to set up shop in the community. In addition, morale in the community was increased and greater ties between the community organization(s) and the town council were established.

In the other community-development ASI case study, both the community organization and the community benefited from the adjustment experience. The committee's recommendations led in the immediate term to the creation of a strategic plan, to hiring business operations managers, and to hiring additional workers to relieve staff shortfalls within the organization. These actions enabled the organization to identify additional revenue sources, enabled it to become more financially self-sufficient, and in general encouraged it to adopt a more business-oriented philosophy. This in turn enabled the organization in turn to be more socially proactive and to place clients from the community in greater numbers and diversity. Working more closely with the schools enabled the development of more labour-market skills for clients. An important consequence of these activities was greater morale in the community.

Finally, in an ASI downside case study, a finding was that the committee helped the community to adjust to the loss of a major local employer.

Footnotes

16	This results target applies only to Provincial Benefits and not to Provincial Measures (Rural Experience and EAS).
17	Submitted to Human Resources Development Canada by the Canada Employment Insurance Commission, December 18, 1998, covering the period up to March 31, 1998.
18	Information to update result target attainment is based on the survey which took place in early 1999.
19	The 1998 EI Monitoring and Assessment Report notes that in the 1997/98 fiscal year there were in excess of 3,300 interventions where individuals made use of these services only. This evaluation is focused on six benefits and measures excluding those interventions, i.e., Partners, Entrepreneur, Job Action, SLG, EAS and Rural Experience.
20	Among those who completed their intervention on or before March 31, 1998, only 43 per cent of this group who said in the survey they had been employed for 12 consecutive weeks in the post-intervention period had a comparative 1997/98 employment outcome flag on the administrative data file. On the other hand, 78 per cent of those who had an employment outcome flag said in the survey they had been employed for 12 consecutive weeks following their intervention
21	We use administrative here because the corresponding question was not asked in the evaluation survey.
22	A claimant is considered to have returned to work if, for 12 consecutive weeks, they receive 25 per cent or less of their maximum EI entitlement within their benefit period. There are of course no paid EI benefits for reachbacks who were not receiving EI benefits at the time of the intervention.
23	This is the approach used by Applied Research Consultants to compute unpaid EI benefits for the Canada/British Columbia LMDA.
24	Technically, we still have not attributed the return to work to a specific intervention. Though we are dealing with participants in interventions, we have not taken into consideration the date of the intervention. For RTWs occurring before the intervention, there are no savings to be derived. Indeed, there were a few such individuals on the file. Subtracting the savings of this group from the computed sum reduces the total saved for the 1997/98 and 1998/99 fiscal years by $543,879 and $58,361, respectively.
25	The 1998 Formative Evaluation of the Employment Benefits and Support Measures - Final Report, for the EBSMs for Canada as a whole, conducted on behalf of HRDC, compared this measure of employment by program type and claimant status in this way.
26	Employers in the employer survey were also asked to indicate the percentage retained of the participants they had in the wage subsidy programs. However, inconsistent reporting by employers led to the conclusion that the percentage retained measure based on the employer survey results would be unreliable. At any rate, it is safe to say that participants individually would be in a better position to recall being retained than employers who would have to recall several different wage-subsidy events.
27	An error in survey coding resulted in a number of Job Action and Rural Experience participants being initially skipped over this question. But there is no reason to believe that those who did not answer this question are any different from those who did, implying that the percentage re-hired for these programs would still be valid. Also, those currently in the program should have been eliminated from this calculation, but as they represent such a small proportion of participants, their inclusion has likely not greatly affected the results.
28	It is interesting to point out that percentage retained by Partners employers computed on the basis of the employer survey results was 59 per cent, which is similar to this rate.
29	The survey simply asked if the participant was on EI following the intervention. The responses to such a question would have been biased against short-term interventions, after which there is more remaining EI entitlement than longer-term interventions.
30	It should also be noted that the participant survey results indicated that 45 per cent of SLG participants received on-the-job training from the employer whom they were placed with while participating in training through the SLG benefit.