Government of Canada | Gouvernement du Canada Government of Canada
    FrançaisContact UsHelpSearchHRDC Site
  EDD'S Home PageWhat's NewHRDC FormsHRDC RegionsQuick Links

·
·
·
·
 
·
·
·
·
·
·
·
 

6. Outcomes of COMPASS


The purpose of this chapter is to lay a solid foundation for the econometric analysis to follow in the next chapter. The descriptive findings on outcome28 are more intuitive than the much more complex econometric analysis; a good understanding of the basic outcomes presented in this chapter will help the reader understand the econometric analysis. A note of caution is in order, though: differences between participants and non-participants cannot be attributed to the Compass Program until the econometric models control for outside influences. That is the business of the next chapter. Findings presented in this chapter must therefore be considered preliminary.


6.1 Recent Labour Market History

For the longitudinal econometric analysis to follow in the next chapter, it is necessary to establish pre-program and post-program labour market activity. The survey asked participants and non-participants to account for their time spent working, unemployed, and in school since 1992. The results are portrayed in Chart 6.1.

Before the Compass Program, non-participants spent more time than participants working and less time in school. Pre-program time spent unemployed (and not in school) was about the same for both groups. For non-participants, time spent unemployed increased from 1992 to 1995 and held steady in 1996. Participants showed a similar pattern until 1995. During 1995 and 1996, when Compass was in full swing, participants' time spent working increased markedly, probably reflecting a combination of participation in the program and post-program employment with the placement employer.

Chart 6.1
Recent Labour Market History

The next chapter will isolate the impact of the program, but for now we can remove the influence of being in the program by looking at how those who finished participating in Compass before 1996 spent their time in 1996. Chart 6.2 shows that participants out of the program before 1996 spent about 43% of 1996 unemployed, 50% working, 5% in school and 2% in school and working. Non-participants referred to Compass before 1996 spent 48% of 1996 unemployed, 47% employed, 4% in school and 2% working and in school. The differences between groups are not significant although they are in the right direction.

Chart 6.2
Labour Market Activity in 1996


6.2 Use of Social Assistance

One of the main objectives of Compass was to reduce reliance on social assistance. As the following chart shows, participants and non-participants have had a history of great reliance on welfare. There was no significant pre-program difference between participants and non-participants. But, during 1995 and 1996, non-participants relied on welfare to a significantly greater extent than did participants.29 This is also evident from the mean monthly amount of social assistance received in 1995 and 1996:

  1995 (SE) 1996 (SE)30
Participants $543 ($19) $490 ($17)
Non-participants $628 ($28) $609 ($25)
  t=2.6, df=694, p<.01 t=3.9, df=707, p<.001

To a certain extent, the results undoubtedly reflect participating in the program, since most of those on placements were no longer receiving welfare. But this doesn't account for the entire effect: Comparing only participants who finished their placement before 1996 with non-participants who were referred to Compass before 1996, we found that participants spent an average of 4.1 months on welfare during 1996, whereas non-participants spent 6.5 months (t=4.8, df=416, p<.001). Before attributing any differences to the program, however, other possible influences must be controlled (see next chapter).

Chart 6.3
Mean Months On Social Assistance, 1993-1996

There were no pre-program differences across options in use of welfare. But, during 1995 some differences emerged, with TTO participants on welfare for 5.2 months, WEO participants for 5.8 months and EDO participants for 7.4 months (F=4.2, df=2/560, p<.02)31. Differences were magnified in 1996, with TTO participants on welfare for 3.7 months, WEO participants for 4.7 months and EDO participants for 6.9 months (F=11.3, df=2/626, p<.001)32. Whether or not this difference was because TTO worked better than WEO or EDO in this respect will be sorted out in the next chapter.


6.3 Use of UI

The target group had consistently spent an average of seven to eight weeks per year on unemployment insurance until 1995 (Chart 6.4). In 1995, the time spent on UI fell by two weeks, probably because most of the group was on social assistance then. Never did the difference between participants and non-participants exceed one week. Note, however, that participants were much more likely to be receiving UI at the time of the survey than were non-participants, as discussed below in the "Current Situation" section.

Chart 6.4
Mean Weeks On UI, 1990-1996


6.4 Post-program Activities

Because not much time has elapsed since most Compass participants completed the program, it is important to determine what they did immediately after finishing. Table 6.1 lists the responses of survey respondents. Just over a quarter of WEO participants and about 45% of TTO participants continued to work with the placement employer after the subsidy ceased. Another 15% of each group started working for another employer. Most of the rest began looking for a job: only 31% of WEO participants and 23% of TTO participants had found one by the time of the survey (not a significant difference)33. As for EDO, only 42% continued in self-employment immediately after their involvement in Compass ended (although as revealed in the next section 72% of all EDO respondents were self-employed at the time of the survey).

There was no significant difference across regions.

Table 6.1 Distribution of Participants by Activity After Compass
Activity All Participants WEO TTO EDO
Continued employment with placement employer 34.7% 25.9% 44.7% 9.8%
Started working for another employer 13.9 15.6 14.2 0.0
Self-employment 2.8 0.0 0.3 41.5
Continued education 2.5 3.8 1.8 0.0
Took a job training program 1.1 1.1 0.9 2.4
Stayed at home with children 2.5 4.9 0.9 0.0
Stayed at home for other reasons 2.3 3.0 2.1 0.0
Started looking for a job 29.1 36.9 25.1 12.2
Other 11.1 8.7 10.1 34.1
x2 = 312.2, df=16, p<.001
phi=.697

As for non-participants, only 19% got a job instead of participating in Compass - about the same proportion of Compass participants who got a job with another employer after the placement. The greatest proportion stayed on welfare. About 15% began looking for a job, but only one in five of them had found one by the time of the survey; and it took them an average of 23 weeks.

Table 6.2 Distribution of Non-participants by Activity Instead of Compass
Activity All Non-participants
Stayed on welfare 23.2%
Started working 19.3
Self-employment 1.3
Continued education 5.2
Took a job training program 4.6
Stayed at home with children 6.5
Stayed at home for other reasons 5.2
Started looking for a job 15.0
Other 19.6


6.5 Current Situation

A crucial test of success is what participants are now doing as compared to those who did not participate. Are participants more likely to be working than non-participants? Are they less likely to be on welfare? Are they less likely to be on UI? Are EDO clients still operating their own businesses? Does participation lead to a greater tendency to upgrade one's education or job training skills? Table 6.3 provides preliminary answers to these questions. The next chapter will control for other factors that may have lead to these results for a more definitive picture.

Before correcting for possible outside influences, it appears that participation in Compass led to a significantly decreased reliance on social assistance and a greater probability of working in a paid job, at least in the short term. At the time of the survey, one-third of participants were on social assistance, versus 57% of non-participants. And 56% of participants were working as opposed to only 37% of non-participants34. The program appeared to have no effect on the probability of upgrading one's education or training. On the other hand, Compass appears to have led to a greater reliance on UI: 25% of participants were on UI at the time of the survey as compared to only 12% of non-participants. This was probably a side-effect of the greater tendency to work in a paid job, but it suggests that the jobs they were getting were unstable.

There is also evidence (again preliminary) that TTO was more effective than the other two options at removing people from social assistance and at getting them into a paid job. Yet TTO participants were much more likely to be on UI at the time of the survey than were those in the other two groups: this is likely largely because TTO placements were long enough to qualify participants for UI, whereas WEO placements were not. About 72% of EDO participants were self-employed at the time of the survey, although half this group was also on social assistance35.

Graphic
View table 6.3

Region had a significant association with two activities. Participants living in the Halifax region were much more likely to be working and much less likely to be on UI than those living elsewhere, most likely because of better opportunities in the large urban area:

Region % Working On UI
Halifax 70.6% 13.4%
Cape Breton 43.8 37.7
North Shore 57.7 25.5
Western 51.4 24.7
  x2 = 16.9, df=3, p<.01 x2= 20.8, df=3, p<.001

Job developers pointed to three general factors that keep some participants unemployed after Compass participation. One was characteristics of those participants who remain unemployed. Many were lacking the motivation to work: they simply had a poor attitude towards work. Or they refused to accept the kind of jobs for which they were qualified: "I find sometimes it's hard to get through to my clients, you're going to do the crappy work before you get the good work." Or they didn't want to be the only one in their peer group who was working: "None of their friends are working, and they are different from everybody else and they don't want to be. They want to be with their friends."

Many who stayed unemployed had barriers that could not be overcome such as lack of child care, and lack of money for job search (e.g., for transportation). Another key obstacle was lack of job search skills: "Most of my clients haven't got a clue how to look for work." One job developer noted that clients who found employment were those who went to job finding clubs, or formally learned job search skills or interview skills.

A second general factor was the poor labour market, especially for those with few marketable skills. But even some highly skilled clients remained unemployed if their expectations were too high.

The third general factor was disincentives built into the social assistance system, especially the FB system. In many cases, the amount of money they could earn working was not attractive enough compared to the amount of money they got on FB to make the effort worth their while. But what it seemed to come down to is "There are no consequences if the client decides not to work."

The situation on the Municipal Social Assistance side was different. "It's grinding poverty." In Kings County for instance, clients can receive as little as $335.00 a month. "That's it. For housing, transportation, food, everything. So if you get them in any job that pays minimum wage, they're happy. And their lifestyle has improved by 100% because their take home pay is nearly $600 a month which is double what they are given."

As for EDO, job developers held that income support for people on EDO was available for too short a period. "We're talking a maximum of 6 months that you can do for income support, if you can get the municipality to carry for 3 months." Economic Renewal said a year is preferable. Good business ideas might fail because the period of support was too short, according to the job developers.

The next three charts reveal the changes in program outcomes over time. On the x-axis of each graph is the number of months since completing the job placement. On the y-axis is the percentage of participants currently (at the time of the survey) working (Chart 6.5), on social assistance (Chart 6.6), and on UI (Chart 6.7). Note that each point on the graphs represent different cases (i.e., it is a cross-sectional rather than a longitudinal analysis). The first graph suggests the effects of the Compass program on employment status do not decline within the first two years of completion (assuming the different cohorts of clients are similar to one another in employability). It also shows that TTO is more effective in this regard throughout these first two years.

Chart 6.5
% Of Clients Currently Working In A Paid Job By Number Of Months Since Completing Compass

Chart 6.6 shows no clear trend for either group respecting use of social assistance. TTO clients start off much less likely to be on social assistance than WEO clients; but by the 7-9 month stage, they have closed the gap considerably. This may be partly because TTO clients are more likely to qualify for UI than are WEO clients.

Chart 6.6
% Of Clients On Social Assistance By Number Of Months Since
Completing Compass

Chart 6.7 suggests a downward trend in use of UI for both groups. The drop until one year after completion may merely reflect exhaustion of entitlement. Except for one period - 10 to 12 months after completion - TTO clients rely on UI to a greater extent than do WEO clients. The higher rate of UI receipt in the first few months largely reflects UI eligibility criteria: the typical 26 week TTO placement is long enough to qualify the person for UI; the typical 16 week WEO placement is not.

Chart 6.7
% Of Clients On UI By Number Of Months Since Completing Compass


6.6 Impact on Earnings

Chart 6.8 shows the earnings history of participants and non-participants before during and after participation in/referral to the program (the chart includes those who had no income)36. Every year prior to Compass, non-participants earned significantly more than participants, which reflects their tendency to have been employed more than participants as already shown. During the year in which participants finished their placement (and in which non-participants were referred to the program), participants earned significantly more, largely due to their placement earnings. In the calendar year after participation/ referral, there was no significant difference in the earnings of participants and non-participants37.

Chart 6.8
Earnings History Before, During And After Compass


6.7 Impact on Educational Upgrading

Since participating in Compass, 14% of participants said they had gone back to school, college or university to upgrade their education38. Since being referred to Compass, 19% of non-participants said they had done the same, a significantly greater proportion (x2 = 5.3, df=1, p<.05). This does not necessarily imply that Compass has had a dampening effect on upgrading, because few participants would have tried to upgrade while they were involved with Compass. That is, the typical non-participant had more time since referral to Compass to upgrade than had the typical participant. For instance, if participants who finished their placement in 1995 are compared to non-participants who were referred in 1996, it turns out that 19% of participants and 16% of non-participants had resumed their education. Thus, subject to the appropriate statistical controls to be applied in the econometric analysis, it appears that Compass had little effect on academic upgrading.

Proportion of participants continuing their education did not differ between options or across regions.


6.8 EDO Outcomes39

Almost all EDO participants (93%) were interested in establishing their own business before becoming involved in Compass. Indeed, 39% had operated their own business before joining Compass. And half the participants had sought assistance from other sources in attempting to start their business before becoming involved in Compass.

There is also evidence that they were well prepared for their foray into self-employment, with 90% saying they had previous training or experience in the field in which they wanted to start their own business. Similarly, about 82% of non-participants who wanted to start their own business had had previous training or experience in the field.

Sixty-four percent of EDO clients responding to the survey received formal training from Compass in entrepreneurial skills and business development. About 81% considered this training to be very useful for starting their own business; the other 19% said it was fairly useful.

Just over half those who received training stated that they got follow-up support from Compass staff once the training ended. Two-thirds found this support very useful, 27% fairly useful, and 7% (representing only one case) not very useful. Over 80% of those who did not receive follow-up support said they had wanted it.

One measure of the stability of a business is whether it is in arrears on its loans. Most (74%) survey respondents who received a stream 2 loan were able to keep up with their loan repayment schedule. This is comparable to estimate of interviewees that about 20% of the remaining EDO businesses were behind on their loan payments.

At the time of the survey, 63% of EDO participants were self-employed in the business they were developing while participating in the Compass Program40. Of the 11 non-participants who tried to start their own business without the help of Compass, 55% were self-employed in that business at the time of the survey. The rate of business survival did not differ significantly between groups (x2 = 0.3, df=1, p>.50), suggesting that Compass made no difference in this respect (but this finding is very tentative considering the small number of cases). Still, most participants needed Compass to get their business off the ground: 82% thought that they would not have been successful in establishing their own business without the help of Compass.

Of those participants who said they were not employed in their own business, 80% never did get the business going. The three businesses that started but failed lasted for 17 weeks on average. Asked why the business is not operating (or never started), the 20 respondents gave eight different reasons: most often mentioned was "no market," by three respondents. Interviewees gave the following reasons for EDO business failures:

  • Training wasn't adequate;
  • Insufficient funding;
  • Insufficient follow-up;
  • Lack of markets for product;
  • Insufficient motivation; and
  • Personal problems.

Most EDO respondents could not or would not answer the question on how much profit their business made in a typical month. Of the 14 participants responding, 10 said there was no profit, and the other four made very little: on average EDO participants reported a $71 profit per month41. Given that the typical participant had been self-employed for only 38 weeks at the time of the survey, small profits are to be expected. Only five non-participants answered and four said there was no profit.

Also, job creation attributable to Compass has thus far been negligible. Three-quarters of businesses started through Compass and still surviving by the time of the survey had hired no other workers42. In total, the 26 surviving businesses represented in the survey had hired nine full-time workers and three part-time workers (apart from themselves)43. Generalizing these findings to the EDO population, it is estimated that the EDO program has helped generate up to 30 jobs, most of them full-time. This may be an over-estimate, if those whom we could not reach to survey were less likely to be operating a successful business than those we did reach.


6.9 Current Attitudes

The follow-up survey repeated a series of questions about attitudes towards work, unemployment, welfare, self-esteem, and life in general from the baseline survey. The responses are presented in the charts presented in Appendix B.

In general participants and non-participants had positive attitudes in all these areas. At the time of the baseline survey, there was little difference between the two groups in any area, although the difference was large enough to reach statistical significance in two cases44. Attitudes did not change all that much between surveys; where there was a significant change, it was always in the hoped-for direction.

The preliminary analysis suggests the Compass program had an impact on only three of the 22 attitudinal variables - and the impact was negative in two of these areas. Subject to the econometric analysis, the program had a positive impact on participants' satisfaction with the work they had done in their lives. But, the program had a slightly negative impact on two areas of self-esteem: "I have as much to contribute as anyone," and "I am able to do things as well as anyone." Participants moved in the right direction for both variables, but non-participants moved significantly more.

There were no significant differences across Compass components on any attitudinal item where the program had an impact.


6.10 Use of the Opportunity Fund

Most interviewees were effusive in their praise for the Opportunity Fund. It was described as a "godsend," a "blessing," "magic," "excellent," "wonderful." "That was wonderful. We were able to assist clients. The feedback has been excellent. I'm going to miss that option the most." It was undoubtedly the most popular part of Compass, and many federal and provincial informants considered it the most useful, and certainly the most cost-effective component.

Why the popularity? Because front-line staff were given the flexibility to make a small amount of money go a long way. "We only had $3750 for the year but that enabled us to do so much." "Whatever that person needed to make that person more job ready, we pretty well could cover it in the Opportunity Fund." According to interviewees, job developers and the clients who received money from the fund (who were surveyed) it was used for drivers licenses, work clothing, safety equipment, insurance, short term training, transportation to an interview, GED, other courses - anything to remove small but significant barriers to employment or training.

We love the Opportunity Fund. We want more. . . And it's not just because it's money, fund money. It's just so flexible. It is well targeted and it's so cost effective. The results are so good that you can see . . . almost immediate gratification happening in people. And 40% of people were able to attach to the labour market directly from that one small intervention -- $100, $200.

There were few sour notes sounded over the fund. What criticism there was centred mainly on the small amount of money available to each ERC. "It needed to be larger. There were many times when someone just needed work boots or a uniform but there wasn't enough money in the Fund." There was one exception: one ERC was "very stingy with it," never paying more than 50% of any item. "We were allocated $3000 for 96/97 but we've only spent $300 of that. In the case of a student, we ask them to pay it back. It builds responsibility."

Finally, one interviewee brought up the potential for abuse since leaving discretion completely in the hands of front-line staff makes its use very subjective.

Survey respondents who got money from the fund received an average of $11345. Three-fifths felt that these funds were very important in enabling them to receive training and/or employment. The rest said the funds were somewhat important. The econometric model will include receipt of the Opportunity Fund to determine its importance in landing a job.


6.11 Conclusion

Focusing on the cardinal objectives of Compass - to lessen reliance on welfare and increase employability - at the level of outcomes as opposed to impact, the program appears to be successful. At the time of the survey, participants were much less likely to be on social assistance and much more likely to be employed than were non-participants. And the pre-program gap in annual earnings in favour of non-participants had been completely closed the year after participation/referral. On the other hand, that participants were twice as likely as non-participants to be on UI at the time of the survey, raises serious questions as to the permanency of the outcomes.

The report now turns to an analysis of impact.


Footnotes

28 Though many consider the concepts of outcome and impact synonymous, there is an important distinction: outcomes refer to measurements of the end state of participants and non-participants with respect to relevant variables such as earnings (e.g., participants $10,000, non-participants $9,000). Impact refers to the difference between the outcome of participants and the outcome of non-participants; that is, the effect of the intervention (e.g., $1,000). This chapter presents the outcomes, the next the impacts. [To Top]
29 1996: t=9.7, df=1125, p<.001.
1995: t=3.4, df=1029, p<.001.
[To Top]
30 Standard errors measure how accurately the sample reflects the population. [To Top]
31 Post-hoc comparisons (Scheffe and Tukey tests) revealed that EDO was significantly higher than TTO in 1995. [To Top]
32 Post-hoc comparisons (Scheffe and Tukey tests) revealed that all three options were significantly different from one another in 1996. [To Top]
33 It took an average of 8 weeks for these participants to find a job (no significant difference across options or regions). [To Top]
34 Moreover, of those working, participants worked significantly more hours per week (35.3 on average) than did non-participants (31.2). There was no difference by option. [To Top]
35 There were too few EDO cases who were self-employed and on social assistance and who answered the questions about the amount of welfare they were getting to determine reliably if the amount of social assistance had changed as a result of participating in EDO. Of the 24 cases reporting the amount of welfare they received in a typical month in 1995 and 1996, the mean monthly welfare cheque actually rose from $677 to $716. The difference was not significant, however, probably due to the small number of cases. [To Top]
36 Standard errors: Participants: 6 years before=$323; 5 years before=$253; 4 years before=$251; 3 years before=$240; 2 years before=$185; 1 year before=$106; during=$501; 1 year after=$815. Non-participants: 6 years before=$392; 5 years before=$272; 4 years before=$283; 3 years before=$290; 2 years before=$234; 1 year before=$160; during=$275; 1 year after=$939. [To Top]
37 For those who reported employment income in the calendar year after participation/referral (i.e., removing 0 income cases) the mean earnings for participants was $7,888 for participants and $7,655 for non-participants (t=0.1 df=195, P>.90). Note the standard errors for the year after Compass are much higher than the other years because of the relatively small number of cases: participants -- $815; non-participants -- $939. [To Top]
38 Standard error=.0138. Thus the margin of error is ± 2.6% 19 times in 20. For non-participants, SE = .0173 for a margin of error of ±3.3% 19 times in 20. [To Top]
39 Note that the findings in this section are based on only 42 participant survey respondents and 12 non-participants. [To Top]
40 Another 9% were self-employed in another business. Interviewees estimated that only 10% of EDO businesses had folded. By comparison, HRDC’s Evaluation of the Self-Employment Assistance Program, reported that 85% of SAR participants in SEA were still operating their own business at the time of the survey (that survey took place an average of 40 weeks after completion of the program versus 29 weeks for EDO respondents in this survey). [To Top]
41 In stark contrast, the typical SAR participant in SEAP reported a profit of $2,222 per month. [To Top]
42 This is comparable to the finding from the SEAP evaluation: 29% of SAR participants hired paid employees. [To Top]
43 The average number of workers hired by participant firms still in operation at the time of the survey was 0.46; the standard error was 0.17. [To Top]
44 Again, when running about two dozen tests with a significance level of .05, we would expect at least one false positive. [To Top]
45 One person claimed to have received $2,000, but we believe this person misunderstood the question. Including this amount, the average goes up to $183. [To Top]


[Previous Page][Table of Contents][Next Page]