 |
|
Summary and Conclusions
The Outcome Evaluation was governed by a set of evaluation questions which were used to focus our investigation. In this section we summarize both qualitative findings from the Interim Evaluation report and quantitative findings presented in this report to draw our conclusions relating to the key issues guiding the study. 1. Participant Satisfaction Conclusion: Participants were generally satisfied with the project. Qualitative Findings Clients were very satisfied with their ITCY training experience,37 expressing that the services they received met their individual needs and that they had come away with valuable skills and experience. Employers were also generally satisfied with their role and with the clients they trained or hired. They indicated feeling like "partners", engaged in a different and deeper way (e.g., mentoring) than in other TOJ-type programs they had been involved in. Quantitative Findings Clients gave the programs high ratings and were generally satisfied with the services and support they received. Relatively few clients indicated needing help with a problem during training, and where the agency tried to help, the clients were satisfied. 2. Reasons for Discontinuation Conclusion: The main reasons participants discontinued training include: poor motivation/lack of interest; having transportation barriers; getting a job; going back to school; and having family/financial problems. Qualitative Findings Agency staff cited poor motivation/attitude as the main reason clients did not complete. Other reasons included: going back to school, getting a job and having family/health problems. Employers also identified poor motivation/attitude as one reason clients left their training placement, but also noted that many left because there was a poor match between the placement and the client's career interests. Quantitative Findings The main reasons clients gave for dropping out of the program included getting a job and going back to school, and also having transportation, family or financial problems, as well as feeling they were not learning anything useful. 3. Impact of Incentives Conclusion: The provision of financial incentives was instrumental in attracting many clients to the training. The incentive was adequate for most participants; however, it was not needed in some cases, and in other cases was not enough to meet clients' needs and encourage/enable them to finish their training. Qualitative Findings Both clients and agency staff clearly indicated that the incentive attracted many youth to the project. 38 Many clients found the incentive money provided the financial stability that enabled them to train for an extended period of time. On the other hand, some youth attended "just for the money" and were never strongly committed to employment. At the same time, it was noted that certain clients (e.g., single parents) quit because they were not receiving as much as they could on SFI, or they could earn by working at the same job they were training at. In these cases, the amount was not a strong enough incentive for them to stay on and finish their training. Agencies indicated that they felt constrained to work within the prescribed grid and had difficulty individualizing the amount of incentive for each client. The incentive became more of an entitlement than an amount based on need or progress. Quantitative Findings The majority of clients reported receiving money from the agency. Most said it was enough to meet their needs.39 At the same time, however, many indicated they would have handled their needs in other ways, and the majority would not have quit training had they not received the money, suggesting the incentive was not absolutely necessary in all cases. 4. Impact on Home and Family Life Conclusion: The ITCY training had a small impact on clients' home and family life. Qualitative Findings Only a few clients indicated their family life, relationships or recreation had changed much as a result of the training. Clients were more likely to indicate they had less time to socialize/recreate because of the training. Quantitative Findings A minority of clients attributed improvements in family relationships, health or recreation to the training program. 5. Preparation for Self-Sufficiency Conclusion: The ITCYs biggest impact appears to have been on clients' attitude, motivation and self-esteem. Most clients also gained useful work experience and occupational skills, as well as job finding and life skills. The programs were not as successful in providing academic skills and a careerplan. Qualitative Findings Clients typically felt they had made significant strides towards self-sufficiency as a result of the training they received. They were most likely to indicate that the program had made a positive impact on their attitude, motivation, self-esteem and confidence. Others commented on improvements in their job finding skills. Employers were generally satisfied with the clients in terms of their job finding and occupational skills as well as their attitude and motivation, although they felt many clients suffered from low self-esteem and poor life management skills stemming from their dysfunctional upbringing. Employers were less likely to be satisfied with clients in terms of their level of education, seeing this as a barrier to longer-term career development. Employers were least satisfied with clients in terms of their career orientation, noting a lack of career planning and long-term goals. Quantitative Findings The majority of clients noted improvements in their attitude and motivation as a result of the training. The majority also indicated acquiring job finding, occupational and life skills as well as knowledge about employer expectations and the training/ experience required to pursue a long-term career path. Clients were less likely to indicate they acquired academic skills they could use on the job. 6. Achievement of Self-Sufficiency Conclusion: The ITCY programs appeared to have a modest impact on clients' self-sufficiency, as measured by incremental gains in employment and income relative to the comparison group. Qualitative Findings Employers were quite optimistic about the future prospects for the clients they trained/hired, but still felt a significant number (e.g., those with poor attitude/motivation, negative lifestyles, "welfare" mentality) would probably be on social assistance in the future. Quantitative Findings Clients showed a steady increase over time in the proportion employed and in their weekly incomes. During the same period, however, the comparison group also demonstrated similar (although smaller) gains.40 Furthermore, the jobs obtained by program participants were not substantially better than jobs obtained by the comparison group. Consequently, the programs appear to have had only a modest impact on clients' self-sufficiency over what they might have accomplished in the job market on their own. 7. Removal of Barriers Conclusion: The longer clients trained, the better their employment outcomes tended to be; however, the ITCYs were not entirely successful in mitigating the effect of certain barriers to employment facing many clients (e.g., poor motivation, lack of transportation, low education, poor work history/lack of experience, young, female). Qualitative Findings Staff felt there would always be a segment of the youth population who would have difficulty maintaining employment because they lacked the motivation to work. Quantitative Findings The longer program participants trained, the better their employment outcomes tended to be (i.e., percentage employed, average weekly income). Clients who were not working gave a wide variety of reasons. Two of the more common reasons were: - not enough experience/education (Destinations);
- don't need/want to work (Career High).
These reasons were somewhat surprising in view of the fact the clients had just gone through a training program focusing, among other things, on skills and motivation. Other reasons included: in school/training (or looking into school/training), lack of jobs, lack of transportation, moving, waiting for job to start, pregnant/on maternity leave. Characteristics of clients who were least successful in terms of employment income included: younger (15 18), female, low education, poor work history, previous SFI history. 8. Cost Effectiveness Conclusion: The Integrated Training model is valued by case managers who believe it to be a more efficient and effective way to deliver services to highly barriered individuals. While results indicate that program participants fared somewhat better than the control group, it is doubtful that the incremental economic benefits would outweigh the higher costs associated with the training model. Qualitative Findings "Integrated Training", where one agency is able to provide (or broker) a range of services, was seen by stakeholders as a more efficient and effective way to provide employment-oriented services to high risk youth. It was viewed as more seamless and holistic for clients to deal with one agency rather than shuttle from program to program. Referral sources would either have to ignore certain needs because services were not available, or work to move the client from service to service, making case management more expensive. The partnerships forged by the agencies (e.g., with employers, with other service providers) also helped to bring more services to the client without increasing program costs. This aspect of the pilot projects was seen to have "raised the bar" in terms of expectations for other contractors. The pilot experience was helpful in negotiating new contracts, pushing agencies to economize, to provide more comprehensive service, and generally to provide greater value for money. Quantitative Findings41 The ITCY programs were found to have an average training cost of approximately $4,200 per participant.42 The modest incremental income gains by the Program Group would not be expected to yield substantial taxation revenue to government.43 The relatively small involvement of study participants found with the Edmonton Police Service and with EI make it difficult to identify "savings" in these areas that can reliably be attributed to the training programs. More comprehensive data was available relating to the involvement of study participants with SFI; however, the savings identified here were relatively small, such that "payback" of training costs could not likely be projected within a reasonable period of time. Integrated training is relatively expensive in comparison with other employment interventions (e.g., job clubs, placement programs), due in part to the additional costs of providing occupational skill training. 44 The ITCYs were expected to establish strong linkages with industry so that the training would target occupations in demand and provide relevant job skills that would make clients competitive somewhat above the entry level. The survey results indicated that many clients were not using the skills they learned, and felt they could have gotten their current job without the training they received. In addition, the jobs obtained by program participants were not substantially better than jobs obtained by the comparison group. These findings call into question the value added by the occupational skill training component of the programs. 9. Lessons Learned Based on the results from the interim and final Outcome Evaluations, the following lessons have been learned from the Integrated Training Centres for Youth pilot project: There is a need for employment-focused services for youth who have dropped out of school. Existing services typically focus on adults, or provide alternative schooling for youth not coping well in the traditional educational system. Community stakeholders support the concept of Integrated Training. They view it as a more holistic, efficient and effective way to provide the range of services needed by youth at risk. Employers are not loath to become involved as trainers of youth at risk, and are prepared to take on a mentorship role that goes beyond the usual training-on-the-job function. Partnerships can be developed which enhance the services provided: The pilot projects suffered no major jurisdictional obstacles. On the contrary, the partnership between the Federal and Provincial governments resulted in more creative and comprehensive programming than might have been mounted by either partner alone. The ITCYs are able to form partnerships with employers and other community service providers which reduce duplication and increase access to complementary services which otherwise would need to be provided in-house. Incentives help to attract youth to training and to reduce their barriers to training. Incentives also provide an inducement for employers to take on youth to train, and for clients to stay in training for extended periods. The provision of incentives is problematic. It is difficult to implement a system based strictly on need. Also, projects which provide incentives can expect higher than normal dropout rates since the money will attract a certain proportion of participants who are not really committed to pursue long-term employment. Although clients were highly satisfied with their experience with Integrated Training, the programs were not fully successful in mitigating some of the barriers youth at risk face in gaining access to the job market. Integrated Training is a relatively expensive employment intervention. Substantial economic benefits must be realized in order for the government to obtain payback over a reasonable time horizon. The pilot projects did not appear to result in the necessary level of incremental benefits.
|
![[To Top]](/web/20060127011257im_/http://www11.hrdc-drhc.gc.ca/edd-img/top.gif){kind=small}
![[Previous Page]](/web/20060127011257im_/http://www11.hrdc-drhc.gc.ca/edd-img/back.gif){kind=small}
![[Table of Contents]](/web/20060127011257im_/http://www11.hrdc-drhc.gc.ca/edd-img/content.gif)
![[Next Page]](/web/20060127011257im_/http://www11.hrdc-drhc.gc.ca/edd-img/next.gif){kind=small}