Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

This section presents findings on process-related outcomes of the Labour Market Development Agreement (LMDA). It presents details on the labour market information and community employment resources used by respondents on a self-serve basis. It also discusses client self-sufficiency, their attitudes toward work and social assistance and their satisfaction with the Alberta Programs and Services (APSs). We use findings from our survey of comparison group members to contrast with findings from participants. Comparison group members are chosen to match participants prior to the survey. However, we cannot control which comparison group members or participants ultimately respond to surveys. Those who complete our surveys are always less well matched than the initial samples. Regression analyses are able to adjust for these differences. Note, however, that simple comparison across the two groups may be affected by the differences between the ultimate samples.

4.1 Labour Market Information Provision

4.1.1 Labour Market Information Used

The unemployed have a variety of employment resources available to them. In our survey, we asked whether individuals had made use of these resources on their own (i.e., on a self-serve basis). Eighty-three percent had. The most frequent resources used by these self-serve clients were newspapers (87 percent), job banks at government offices (77 percent), job banks at other locations (50 percent) and the Internet (44 percent). Exhibit 2 presents details on the use of employment resources.

Seventy-two percent of comparison group members have made use of employment resources available in their communities on their own. The most frequent resources used by these comparison group members are newspapers (93 percent), job banks at government offices (71 percent) and job banks at other locations (30 percent). Newspapers are used more frequently and job banks less frequently. Compared to the participant group, comparison group members are more likely to use newspapers (93 percent vs. 87 percent) and less likely to use job banks at government offices (71 percent vs. 77 percent) and at other locations (30 percent vs. 50 percent).

Comparison group members more likely to have used employment resources on a self-serve basis are:

• Higher earners, those with household incomes of more than $40,000 (83 percent);
• Higher educated, those with some post-secondary education (82 percent); and
• Younger, those under 34 years of age (80 percent).

Greater use by those with higher income, higher education and more youth is not evident in the participant survey data. This fact suggests that participation has resulted in greater access among those groups who do not normally seek out self-serve tools.

a) Internet

Forty-four percent from our participant survey, but only 23 percent of comparison group members, identified using the Internet for labour market information.

The Internet resources most frequently accessed by the participant group are job listing (other than Electronic Labour Exchange) or résumé posting services (41 percent), the Electronic Labour Exchange (22 percent) and Canada WorkInfoNet (22 percent). Comparable percentages for comparison group members are 36 percent, 8 percent and 28 percent.

b) Printed Career and Work Search Information

Thirty-four percent of the participant survey group, but only 11 percent of comparison group members, identified using printed career and work search information.

The principal items used by those individuals from our participant survey using printed career and work search information are the Job Guide (53 percent), Work Futures/Job Futures (30 percent) and Career Paths (29 percent).

Participant group members using printed career and work search information accessed it from a variety of sites. They are most likely to have found this information at a Career Development Centre (CDC) (33 percent), Human Resources Development Canada (HRDC) or Employment Insurance (EI) office (22 percent), Labour Market Information Centre (12 percent) or at a Canada-Alberta Service Centre (CASC) (11 percent).

c) Provided Technology

Participant group members are also more likely to use technology provided for them by government offices or their third party service provider. Thirty-five percent of the participant group and 22 percent of comparison group members said they used provided technology. The technologies used most frequently by participant group members were computers (74 percent), fax machines (53 percent) and photocopiers (53 percent). These technologies are most frequently used at CDCs (29 percent), private training centres (13 percent), HRDC or EI offices (12 percent), Human Resource Centres of Canada (HRCC) (12 percent) or CASCs (11 percent). Exhibit 3 provides detail on the use of technologies, by participant group members.

d) Participant Group Members' Attitudes Toward the Labour Market
Information Used

Participant group members rated five statements about the labour market information they used on a scale of one to seven. On this scale, one means "strongly disagree" and seven means "strongly agree." They were most likely to find the labour market information easy to understand (5.9), easy to use (5.8) and/or up to date (5.8). They were slightly less likely to find that it met their needs (5.2) or made them more self-sufficient (5.2).

4.1.2 Significant Differences in Use of Labour Market Information and Provided Technology by Participants

Participants who were also self-serve clients and comparison group members were asked questions in their respective surveys, related to their use of Labour Market Information (LMI) and provided technology. We assessed potential differences through regression techniques. Regression models controlled for potential differences in the demographic and other characteristics between participants and comparison group individuals. By adjusting for the effects of other known differences, the differences that remain may be due to the intervention. Significant results are listed in Exhibit 4. These results represent changes that are larger than would be normally associated with chance. Using regression techniques, we found a significant difference⁴ in the use of:

• The Internet. Participants were 19 percent more likely to have used the Internet as a result of having participated in their APSs. Participants who participated in programs other than CEAS (this category was assessed because of the unique nature of CEAS) were 26 percent more likely to use the Internet while CEAS participants only were 14 percent more likely. Other individual programs where there was a significant increase in the use of the Internet were JP with a 40 percent increase, SW with a 24 percent increase and SDS at 15 percent. Active claimants increased their use of the Internet by 18 percent.
• Printed career and work search information. On average, participants were 21 percent more likely to use printed information than they would have been without their APSs. Results were positive and significant for participants taking both CEAS and SW. They were also positive and significant for those taking programs other than CEAS. (As noted earlier, this category was assessed because of the unique nature of CEAS.) Active claimants were 17 percent more likely and reach-back clients 30 percent more likely to use printed career and work search information.
• Provided technology. On average, participants were 11 percent more likely to use provided technology as a result of having participated in APSs. Results were positive and significant for participants taking the individual programs CEAS and SW, and for the participants taking any program other than CEAS. Active claimants were 14 percent more likely to use provided technology.

Participation may be responsible for these differences. Alternatively, some other unmeasured factor, such as motivation, may account for both higher use and participation in an APS.

4.2 Client Self-sufficiency

Client self-sufficiency was examined using the following indicators:

• The development and completion of action or business plans;
• Participant contributions to APS costs; and
• Participant attitudes toward being independent of assistance and/or contributing toward training costs.

Two-thirds of confirmed APS participants (67 percent) have developed action or business plans and 90 percent have completed at least some of the goals set out in these plans. As a result, a majority of participants take part in activities that should lead to their self-sufficiency.

As reported, 43 percent of APS participants who identified costs for their APS contributed to these costs. These include 23 percent who paid part and 20 percent who paid all of their costs. Slightly less than half participate in the costs of their back to work activities.

As reported below, participants rate the importance of becoming independent of Employment Insurance (EI) or Supports for Independence (SFI) very highly (at 6.2 on a seven-point scale). They also rate their willingness to pay for a training course to stay off government support high (at 4.9 on a seven-point scale). As a result, participants appear motivated to becoming self-sufficient.

4.3 Participant Attitudes

4.3.1 Attitudes Toward Themselves and the Labour Market They Face

APS participants were asked whether their attitudes toward work and their futures have become more positive, more negative or stayed the same since their involvement in their APSs. Sixty percent report that their attitudes have become more positive and 30 percent say they have stayed the same. Only 9 percent report that their attitudes have become more negative.

Participants whose attitudes have become more positive are more likely to be:

• Skills Development Upgrading (SDU) (85 percent) participants;
• If finished with their APSs, completed more than one year ago (84 percent);
• Still involved in their APSs (73 percent);
• Residents of northern Alberta (72 percent);
• SW (72 percent) participants; and
• In a household with annual income of more than $40,000 (70 percent).

Those with a more negative attitude are more likely to:

• Have less than a grade 12 education (14 percent);
• Have household income less than $20,000 (14 percent);
• Participate in TOJ (15 percent) or JP (19 percent); and
• Not be a self-serve client (20 percent).

On balance comparison group members are slightly more negative and apprentices slightly more positive than non-apprentice participants in their attitudes to a number of statements about themselves and the labour market they face.

In our survey, statements were assessed on a scale of one to seven. On this scale, one means "strongly disagree" and seven means "strongly agree." Participants agree (5.4) with the statement "I am generally optimistic about my future." They also agree, but less strongly (4.9) that "If it means staying off government support, I would be willing to pay for a training course." Comparison group members are slightly less likely (5.2 and 4.7 respectively) to agree with these statements. Apprentices are more likely (5.8) to agree to the first and equally likely to agree with the second statement.

Non-apprentice participants are less likely to agree that "I would turn down a well paying job if I had to move from my community to get it" (3.3) or that "There are no jobs out there" (2.4). Comparison group members are slightly more in agreement (3.5) that they would turn down a job if they had to move. Apprentices are slightly less in agreement (3.2). Comparison group members agree with non-apprentice participants (both 2.4) in their assessment of the availability of jobs. Apprentices are slightly less likely to agree that there are no jobs out there (2.2).

Participation may also affect attitudes beyond their work and futures. We asked participant and comparison group members: "How important is it to you to be independent of employment insurance benefits or social assistance?" A seven-point scale was used to assess importance where "1" was not very important and "7" was very important.

Statistically significant results were found for participants under all APSs, all APSs excluding CEAS, and under SDS who put higher importance on being independent of EI or SFI benefits. Differences may be attributable to participation since the two groups were drawn to be statistically similar in observable characteristics other than participation.

Note, however, that any differences may be due to other unobservable factors. For example, we cannot rule out the possibility that some hidden factor, such as motivation, is attributable both to participation and to any difference between participants' and comparison group members' responses to these questions. Findings are indicative but not definitive of an attributable impact of the APSs.

4.3.2 Satisfaction

Satisfaction was assessed on a scale of one to seven. On this scale, one means "not at all satisfied" and seven means "very satisfied."

a) Satisfaction

In total, 725 participants have ended their APSs. On average, these participants rate their satisfaction with their APSs at 5.2. APS satisfaction is highest among SDU participants (6.2) and lowest among TOJ participants (3.5). Satisfaction for all other APSs are between 5.0 and 5.3 on this scale.

b) Satisfaction by Apprentices with Training

Apprentice participants rated their satisfaction with the training they received. On average, apprentices rated their training at 5.7.

c) Satisfaction with Current Job

In total, 407 participants were working at the time we interviewed them. On average, participants rated their satisfaction with their jobs at 5.0. Comparison group members working at the time we interviewed them rated their job satisfaction slightly lower than this (4.8) on the same scale.

4.3.3 Usefulness of APS

Participants who have ended their APSs also rated the usefulness of their APSs to the achievement of a number of goals on a scale of one to seven. On this scale, one means "not at all useful" and seven means "extremely useful." They found their APSs to have been most useful at helping them increase their confidence and clarify their goals (4.7 for each). They also found that they helped them to gain specific occupational skills, general job skills and full-time employment (4.1 for each). They found them least useful at improving their earning potential (3.9).

APSs ranked highest in terms of areas of specific usefulness were:

• Gaining confidence: SDU (5.9) and SW (5.2);
• Clarifying career goals: SDU (5.3) and SDS (5.0);
• Getting full-time employment: SW (4.6) and SDS (4.5);
• Getting general job skills: SW (4.6) and SE (4.3);
• Getting specific occupational skills: SDS (5.6); and
• Improving earning potential: SDS (4.6) and SDU (4.4).

4.4 Further Training

About one-quarter of participants whose APSs have ended (27 percent) have taken further training or courses since that time. Seventy-three percent have not. Of those who have taken further training, just over one-half (52 percent) say they are taking it is as a result of their involvement in their APSs. Thus, about 14 percent of participants whose APSs have ended are taking further training as a result of their involvement in their APSs. This is a positive outcome of the APS experience.

4.5 Self-employment

Of the participants who confirmed their involvement in the SE program, three-quarters (81 percent) report having started their own businesses. This includes 35 percent who started their own business before starting their SE program⁵ and 46 percent who started their business after starting their programs. Note that we do not know the extent to which these businesses were operating at the time participants started the SE program. Those started prior to the SE start date may have been operating at a marginal level or not at all when the SE intervention began.

Of the participants who started businesses before they entered the SE program, 61 percent say they made changes to their businesses as a result of their participation in the SE program. Of the participants who started businesses after they entered the SE program, three-quarters agree that their participation in the SE program allowed them to start this business. Therefore, 69 percent report a positive impact (changes to existing business or started a new business) as a result of SE.

At the time of the survey, 74 percent were running their own businesses. Most of the businesses opened by SE participants were started between April and June 1998 (31 percent), July and September 1998 (30 percent) or October and December 1998 (24 percent). Most of the participants currently running their own businesses (58 percent) are still participating in their self-employment programs.

SE participants who have ended their programs give them an overall satisfaction rating of 5.0 (out of seven). They found them most useful in terms of learning how to run their own businesses (4.7), getting more confidence in themselves and their abilities (4.6), clarifying their career goals (4.3), gaining general job skills (4.3) and improving their earning potentials (4.1). They found them somewhat less useful in terms of gaining specific occupational skills (3.2).

Footnotes

4

The level of significance identifies the risk or probability that an estimated result which is judged to be significant is in fact not different from the actual result. For a result to be considered significant, a maximum risk of 5 percent is generally accepted. Smaller levels of significance represent less risk and, therefore, greater certainty that the stated result is significant. We report the level of significance for all results identified as significant.

5

Guidelines for client eligibility (2.1.1) for the SE program identify that the client should have a viable business idea for starting a new business or taking over an existing one where the participant has not had prior ownership. This suggests that the participant should not have started the business prior to participation. A strict interpretation of this guideline would indicate that it is violated in approximately one-third of the cases.