|
|
     |
||
 |
     |
 |
|
By way of summary and conclusion, this closing chapter will present a capsulized response to each evaluation question.52 Chapter/section numbers from this report are provided for the reader who wants more specifics. Relevance1. To what extent are the Employment Benefits and Support Measures (EBSM) delivered under the Labour Market Development Agreement (LMDA) consistent with the EI legislation? With the priorities of Human Resources Development Canada (HRDC) and Nova Scotia (NS)? What are the strengths and weaknesses of the strategic partnership approach? EBSM is consistent with the EI legislation, however, program guidelines such as targeting, selection, and case management are not always closely adhered to, as discussed in Chapters Four and Six. HRDC Nova Scotia is satisfied that it had ample input into the design of the program for Nova Scotia. Informants felt that EBSM is consistent with the priorities of HRDC. Although the LMDA is experiencing some challenges, this is to be expected at the early stages of any partnership. Concerns raised by key informants include a cumbersome committee structure, committee members being on multiple committees, and communication problems. LMDA partners are addressing most of the concerns uncovered in the evaluation by streamlining the committee structure (section 5.2). 2. To what extent are the EBSMs relevant to the employment needs of individuals? To the labour market needs of employers? To the needs of communities? Have adjustments been made to make the EBSMs relevant to these needs? Qualitative evidence suggests that the EBSM are meeting client needs. Most EBSM participants (78 percent) thought the program was good or excellent. Few gave the program a failing grade (4 percent) or a below average grade (5 percent). Clients in every EBSM component gave a mean overall grade of B, except for Employment Assistance Services clients who gave a B-. Although clients were disappointed with the program's help with finding them a permanent job, 88 percent felt fairly or very well prepared for a new job as a result of their intervention. The C grade for finding a permanent job may therefore be a comment on client perception of the job market rather than the program. Every employer interviewed for the case studies was enthused with the Targeted Wage Subsidies option (section 6.5). They also asserted that it helped them meet their labour market needs. HRCC staff were divided on whether employer needs were met. In areas where Targeted Wage Subsidies is employer driven, employer needs were said to be met. Where Targeted Wage Subsidies was client-driven, staff were not sure if employer needs were met. They felt options for on-the-job training were limited. Because of local level flexibility it is fair to conclude that EBSM has good potential to be relevant to the needs of the community. Certainly every community agency representative interviewed in the three communities where case studies took place was pleased with the program and its new emphasis on community capacity building. Design and Delivery3. Do the design and delivery of Part II initiatives reflect service delivery principles and guidelines of the EI Act and LMDA? a)To what extent have areas of joint cooperation minimized unnecessary overlap or duplication within and between governments? Most informants were of the opinion that joint cooperation had minimized unnecessary overlap or duplication between governments, especially for Targeted Wage Subsidies and Self-Employment. A few, however, questioned to what extent harmonization has really occurred. In support of the latter position, there are still various programs to serve the self-employed in Nova Scotia. b)To what extent have Part II initiatives been designed and implemented in partnership with others in the community? Have they led to development or enhancement of partnerships among various levels of government, employers and community groups? Not surprisingly, given the flexibility of the measures, partnerships depend a lot on the particular community. In the Yarmouth area, the HRCC has been very proactive in developing community partnerships with community groups and employers. In other areas such as Antigonish and Sydney, HRCCs seem to be less proactive. HRCC staff in Antigonish do not go out and actively search out potential projects, but they do represent HRDC on a variety of community boards and committees where ideas for projects are often broached (section 6.3). All three communities had examples of successful partnerships with employers and the community. HRCC Antigonish and Yarmouth have Regional Development Authorities that play important intermediary roles in the Local Labour Market Partnerships option. These organizations work with HRDC to identify partners that need help. c)To what extent is there flexibility to allow significant implementation decision-making at the local level? To what extent does the use of Part II initiatives reflect the needs of the community? A recurring theme throughout this evaluation has been the unprecedented level of flexibility at the local level. The strengths and weaknesses of the flexibility have a strong bearing on the strengths and weaknesses of the program. The flexibility enables each HRCC to respond to the needs of its community. d) Are the Part II initiatives accessible in the official language of choice where demand warrants? Were those receiving services in a minority language satisfied? This is not really an issue in Nova Scotia. Survey findings show that only 0.3 percent of EBSM clients spoke French only. Participants most comfortable speaking French were asked in the survey if they received services in the language of their choice. Eleven percent said no (but note this represents only three respondents). 4. What administrative measures have been put in place to collect information on participants, initiatives and third party providers? Is this information adequate for measuring project impacts? If not what improvements are necessary? Overall, key informants were unsatisfied with the monitoring function of EBSM. Regional office and zone offices were worried about the lack of monitoring by HRCCs. HRCC managers were concerned about lack of monitoring of clients on work experience projects. HRCCs were also worried about their own ability to monitor external agencies under contract (section 4.2.4). HRCC staff note that downsizing may have contributed to their inability to monitor agencies and clients. HRCC staff lack the time to visit employer or project sites. Some staff expressed concern about lack of time, and in some cases expertise, to provide financial oversight (to ensure money is spent appropriately). According to virtually all interviewees in the zones and HRCCs, management information tools were inadequate. There is little timely information available about interventions or contracts. This complexity, along with the aforementioned lack of resources dedicated to monitoring, including data monitoring, may well be causing data quality problems. 5. a) To what extent do HRCCs have client targeting and selection? How are targeting and selection criteria determined? b) Are participants representative of these groups? c) To what extent were targets met? d) Are the criteria addressing the needs of the community? Does it result in gaps in services? In most offices targeting and selection criteria exist but staff at some offices do not always adhere to the criteria in their effort to help all clients. It seems that the targeting and selection process is being reviewed and adjusted in some offices, while essentially being omitted in others (section 4.2.5). Concerning gaps in service, most everyone was worried about those who are not eligible for EBSM because they have had no recent EI eligibility. Excluding people who are not on EI attached has large implications for the province. "The decision by the Federal government to get out of training and to narrow the legislation has been hard on Nova Scotia... Many Nova Scotians who are not eligible for EI will end up on social assistance." Among the groups mentioned as "falling between the cracks" by informants: the social assistance client after three years; youth with low education levels; women returning to the labour market; the Black and disabled communities; and employers. Some offices have been able to provide some services through the Labour Market Partnership measure. But these groups cannot be funded for training programs. 6. Is Labour Market Information (LMI) available and accessible to support Part II initiatives and to provide information to clients? To what extent were clients satisfied? Everyone interviewed thought LMI is available and accessible. A few interviewees mentioned that access is a problem for those with reading problems. And several warned that the information becomes outdated quickly, and there must be constant product development. Clients surveyed gave LMI an average grade of B- for both written and computerized material (section 8.2). 7. What are the strengths and weaknesses of the EBSM delivery structure? Are the roles and responsibilities of various partners clear for planning, management, delivery and budgeting? On the basis of interviews and focus groups, the evaluators concluded that flexibility is at once the primary strength and weakness of the delivery structure. HRCCs are happy with their ability to make local decisions. External service providers are also pleased with the discretion local HRCCs have to make decisions without waiting for signatures at every level. There is little doubt that flexibility makes the HRCCs more responsive to local needs. But flexibility can also have its drawbacks. Rules help to ensure the intervention is delivered as designed. Without them, lack of consistency may be introduced between and even within zones. Some HRCC managers and staff confirmed that the lack of consistency among the HRCCs had contributed to some problems. Many of those interviewed for this project indicated that the flexibility of EBSM and devolution of power to local staff could be a challenge to new workers and that even experienced workers need some parameters. (section 4.2.2). Impact9. How satisfied are participants with the benefits and measures provided under Part II initiatives? To what extent did the participants discontinue before the anticipated completion date? What were the main reasons for not completing the action plan? Most participants (78 percent) thought the program was good or excellent. Few gave the program a failing grade (4 percent) or a D (5 percent). The mean overall grade was B (good). Clients in every EBSM component gave a mean overall grade of B, except for Employment Assistance Services clients who gave a B- (section 8.3). Administrative data showed that about a quarter of EBSM clients had no Action Plan at any time, but most of those who did completed it. According to the data, 93 percent of EBSM clients successfully completed their intervention. No single reason predominated for not completing the intervention. Discontinuation was not a major issue in Nova Scotia. Survey findings bolster these results. The vast majority of participants on job placements stayed on the job for the entire period of the placement: 84 percent for Targeted Wage Subsidies and 86 percent for Job Creation Partnerships. Only 11 percent of the Purchase of training trainees failed to finish their course with the reason given most often being they were still in the course (section 8.2). 10. To what extent does EBSM encourage participants to assume more responsibility in the development and implementation of their action plan? Did participants become involved in decisions concerning interventions? To what extent did they contribute to the cost of the intervention? Does it result in more commitment from participants? According to HRCC staff, clients who develop action plans — normally high need clients who require upgrading and/or training — generally take the onus for developing and implementing their action plan. Many come into counseling with a good idea of what they want and, judging by the completion rate, follow through with their plans. Counselors ensure clients know relevant information about future employment prospects and help clients to identify their strengths and weaknesses, but otherwise participants have the primary role in deciding what their needs are. The survey showed that 43 percent of Purchase of training respondents were determined to be able to contribute to the cost of their training: 40 percent actually did so. All but 5 percent of Self-Employment clients made an initial capital investment in their new business (section 8.2). Nearly all informants reported that the client is much more committed to the plan when it is not being entirely financed by HRDC. They have found clients to be more motivated to succeed if they contribute to their intervention. Econometric results suggest that those who contributed to the cost of their intervention did better in terms of earnings than those who did not contribute. 11. To what extent have the Part II initiatives helped participants to reduce their dependency on EI benefits and/or social assistance? Obtain or keep employment? Increase their earnings? Create their own jobs? Have the primary targets in terms of employment been met? The econometric analysis in Chapter 9 suggested that EBSM had no impact on reducing social assistance dependency; however there is some evidence that EBSM modestly reduced EI use, measured in terms of both weeks of benefits and total benefits received Participation in EBSM appears to have slightly increased time spent working — Targeted Wage Subsidies was most effective showing an increase of approximately 1-2 months in time spent working. EBSM was shown to have no significant impact on earnings on most EBSM components. About 70 percent of participants were employed for at least 12 consecutive weeks following participation in EBSM. Self-Employment and Targeted Wage Subsidies clients were most successful in this regard. 12. To what extent did the EBSM assist individuals to become self-reliant in career decision-making, skill enhancement, employment maintenance and job search? Some 23 percent of EBSM participants had gone back to school, college or university since taking part in the program. By comparison, 24 percent of non-participants had taken further education or training since the start of 1997 (section 8.1). When one considers that the question covered a two-year period for all non-participants but 59 weeks for the typical participant, it appears that participants were more inclined to upgrade their skills. The impact analysis attributed a small increment — about 4 percent — to the program in the probability of being engaged in education or training at the time of the survey. EBSM clients surveyed gave a B to the program for improving job skills, a B + for upgrading educational skills, a B- for improving their job search skills, and a B for helping them to develop a career action plan (section 6.8). 13. What EBSM characteristics/program components are associated with improved employability and earnings, further education, or self-employment? What activities/interventions were most effective? For what type of participant? For completers/non-completers? Why do some participants remain unemployed and on income support after the project? The previous chapter presented the impact EBSM as a whole and for each component in terms of employability and earnings, further education, or self-employment (see highlights section of Chapter 9 for a synopsis). The analysis looked at program impact on annual earnings by sex, visible minority status and disability status, and found nothing significant. The reason most frequently offered for being unemployed was that there were no jobs available (see Chart 8.15 for the rest of the reasons). 16. For what reasons do employers participate? Do funding arrangements offer sufficient incentives to participate? What do employers like/not like about the program? Employers who were interviewed indicated they participated for two main reasons: because they need extra help for a period of time; and to subsidize employees that need training on the job to carry out specific duties. Employers were pleased with EBSM (section 6.5). The only area for improvement, in one employer's opinion, is better screening prior to sending the potential employee out. Questions related to specific benefits19. What is the survival rate of Self-Employment (SE) sponsored businesses? The survey found that 68 percent of SE supported enterprises were still in business at the time of the survey. Business development agencies interviewed reported a 76 percent success rate (section 6.7 and 8.2). 20. To what extent does SE result in direct job creation in addition to the self-employment of the participant? About 70 percent of the new businesses employed only the Self-Employment participant when the survey took place. In total, Self-Employment businesses had generated 91 jobs — 51 of them full-time positions — besides providing employment for the entrepreneurs. This may understate the true job creation power of Self-Employment since the survey only reached 54 percent of Self-Employment clients. However, it could be assumed that the remainder of participants were not as successful since many had moved or were otherwise unreachable. Still it is likely that Self-Employment has produced over 100 direct jobs besides the owners' positions. 21. Who is the target group for Targeted Wage Subsidies? Do participants remain employed with employers after the subsidy has ended? In theory, Targeted Wage Subsidies aims to provide job experience for those who have been unemployed for a long time or face special barriers to employment. HRCCs normally consider Targeted Wage Subsidies for moderate need clients who need more than Employment Assistance Services to get a job. The survey found that 62 percent of Targeted Wage Subsidies participants were offered a job after the placement: almost all accepted. However, only 35 percent of these clients were still with the subsidized employer at the time of the survey (section 8.2). 22. Is activity under Targeted Wage Subsidies incremental? There is no direct evidence on whether employers would have hired someone in the absence of the subsidy. 23. What are the level, scope and type of activities taking place under Job Creation Partnerships? With a start date on or after January 1, 1997 and an end date on or before June 30, 1998, there were 294 Job Creation Partnerships in Nova Scotia. Chart 8.8 indicates what types of occupations Job Creation Partnerships clients were working at while being subsidized. Job Creation Partnerships jobs were chiefly in construction, services or administration. This conforms to the type of projects funded through Job Creation Partnerships. Section 6.4 of this report gives several examples of Job Creation Partnerships projects. 24. To what extent have Job Creation Partnerships been successful in fostering partnerships? Who are the local community partners involved? What type of contributions are they making? There is only limited, anecdotal information on the success of Job Creation Partnerships in fostering partnerships. The examples related in section 6.4 demonstrate that partners made contributions to some partnerships, perhaps an indicator of success. However, many employers were community organizations with limited resources for contributing financially (or keeping employees after the subsidy expires). Some 44 percent of the Job Creation Partnerships clients surveyed worked for community organizations. Another 24 percent of Job Creation Partnerships employers were public sector organizations. But 31 percent were private sector employers who may well have topped up the wages. Approximately 38 percent of Job Creation Partnerships jobs survived the end of funding (section 8.2). 27. What occupations are training Purchases clients being trained for? How are training needs identified? Chart 8.6 presents the occupation for which Purchase of training clients were training. Sciences, engineering and health, as well as construction were the most popular fields. Clients often identify their own training needs according to HRCC counselors. Otherwise the counselor and/or various workshops help clients identify their needs. 28. Are clients finding employment in related occupations? Asked how closely related their current job is to the job training they received, 58 percent of Purchase of training participants said very closely related, 17 percent said somewhat related and 26 percent said not at all related (section 8.2). 29. To what extent is Negotiated Financial Assistance used at the local level? How is it used? Are staff adequately prepared to use it? Negotiated financial assistance is used across the province for clients who go on training interventions. The extent of the contribution is very much influenced by the individual's situation, with some not being able to contribute at all. The main reason given for being unable to contribute: they do not have the resources available. Offices vary in their approach to financial negotiations with some offices having a starting point on the negotiations (e.g. client can contribute 40 percent unless they can prove otherwise) while other offices do a financial assessment on the client to determine their ability to contribute to their training. Although most staff who were interviewed had received training on financial negotiations, they acknowledge that there is considerable variation amongst staff on what is expected from the client. 30. To what extent have Purchase of training participants contributed to the cost of their intervention? Was the contribution appropriate to their circumstance? Some 40 percent of Purchase of training participants contributed to the cost of their training. The average course contribution was $1,171; but this figure is skewed by nine clients who paid $4,000 or more for their course. The median contribution was $400 (section 8.2). Counselors would argue that the contribution is always negotiated and depends on the individual circumstances of the client. 32. To what extent are third-party organizations used to deliver employment services to clients? How are HRCCs managing Employment Assistance Services? According to survey data, 67 percent of Employment Assistance Services clients received Employment Assistance Services from an HRCC, 18 percent from a third party, and 15 percent from both (section 8.2). This seems to be at odds with interview and focus group findings that most HRCCs contract out Employment Assistance Services to third party organizations. 33. Does the use of external service deliverers address issues related to capacity at the local level? Has Employment Assistance Services increased access? Has Employment Assistance Services resulted in contributions from other agencies? The main reason given for using external service deliverers was the reduced capacity of HRCCs to carry out their tasks due to less staff (section 4.2.6). Employment Assistance Services is a component mostly delivered by external agencies, in particular services such as teaching job search skills and resume writing. Whether it has increased access is an open question: it has helped maintain access that otherwise might have been forfeited in the face of reductions in resources and staff. The case study reported examples of contributions from external agencies as a result of an Employment Assistance Services contract: mostly in-kind contributions like office space and equipment. A good example is the Human Resource Services Centre at Burridge Campus of the Nova Scotia Community College which delivers Employment Assistance Services to clients in the Yarmouth area. The Centre does all the information group sessions, and Needs Determination interviews with the clients. They do some of the counseling and case management as well. The campus provides the facilities and access to other college resources such as the learning centre. 34. What are the level, type, and scope of activities taking place under Local Labour Market Partnerships? Who are the partners involved? There were 221 Local Labour Market Partnerships in Nova Scotia with a start date on or after January 1, 1997 and an end date on or before June 30, 1998. Local Labour Market Partnerships money has been used to fund a wide variety of projects including: hiring coordinators for groups serving the EBSM target groups, financial support for career fairs, creating community economic development advertising materials, funding research to support community development, and financial support for other regional development activities. Partners include Regional Development Authorities, community agencies, employers, chambers of commerce, and governments. Section 6.3 includes examples of Local Labour Market Partnerships used at case study sites. 35. What are the objectives of Local Labour Market Partnerships? How are they established? To what extent do they meet local labour force priorities? The objectives of Local Labour Market Partnerships are to support community efforts to identify labour market needs, identify the capacity of the community to meet those needs, work with partners in building "community capacity" to address local labour market needs, and minimize duplication and gaps in service. As such, Local Labour Market Partnerships fund research, community planning, and innumerable other activities in the name of community economic development. Antigonish and Yarmouth have Regional Development Authorities that play important intermediary roles in the Local Labour Market Partnerships option. These organizations work with HRDC to identify partners that need help. 36. How are sharing of costs, mandate, mutual responsibilities and accountability for results determined? Are partners satisfied? Local Labour Market Partnerships are negotiated with the partners. Local Labour Market Partnerships are usually handled by the project officers in the HRCCs. Third parties are generally required to submit quarterly reports to the HRCC for monitoring purposes. Partners were satisfied with the Local Labour Market Partnerships and with the HRCC (section 6.3).
| |||||||||
![[To Top]](/web/20060127000449im_/http://www11.hrdc-drhc.gc.ca/edd-img/top.gif){kind=small}
![[Previous Page]](/web/20060127000449im_/http://www11.hrdc-drhc.gc.ca/edd-img/back.gif){kind=small}
![[Table of Contents]](/web/20060127000449im_/http://www11.hrdc-drhc.gc.ca/edd-img/content.gif)