Industry Canada engaged KPMG Consulting to conduct a summative evaluation study of the Community Access Program Youth Initiative (CAP YI). The summative evaluation was undertaken in order to obtain program results and identify program improvements. The key steps in this evaluation were the analysis of the youth exit surveys and the delivery agent’s status reports, and interviews with CAP YI personnel and delivery agents.

The findings of this study are presented under the four evaluation issues: need for the CAP YI Program, appropriateness of the CAP YI structure and delivery, results achieved by the CAP YI Program, and suggested improvements to the CAP YI.

Evaluation Issue #1: Need for the CAP YI Program

No comparable youth initiative program(s) within the provinces/territories.

Youth are acquiring relevant work experience.

CAP YI succeeds in contributing to Industry Canada’s objectives under the Connectedness Agenda.

CAP sites are benefiting from the additional resources that helps them to better serve their community base.

Meaningful job opportunities are made available within remote/rural communities.

Evaluation Issue #2: Appropriateness of the CAP YI structure and delivery

Recently, service contracts with delivery agents have been replaced with contribution agreements.

Some delivery agents receive matching funds that enable them to continue with the program even when Industry Canada funding is delayed.

Industry Canada has not been effective at rolling out the program in a timely manner.

Delays in funding affect the credibility of the delivery agents tasked with delivering the program.
Some delivery agents are considering no longer delivering the program.
Many youth miss out on the opportunity to apply to the CAP YI program given its late roll-out.
Many CAP sites are unable to secure youth during the summer period.

Delivery agents are successful at ensuring program awareness among CAP sites.

The main tracking/monitoring tool has not been operational for 2 years.

Youth exit data are unreliable.

Industry Canada is considered flexible in allowing delivery agents to interpret selection criteria.

Limited communication is provided by Industry Canada to delivery agents.

Industry Canada has limited awareness of what is happening in the field.

Evaluation Issue #3: Results achieved by the CAP YI Program

Overall, it would appear that the CAP YI has been successful at meeting its objectives under the Connectedness Agenda and Youth Employment Strategy.

The CAP YI program is successfully providing youth with work experience that facilitates transition to the labour market.

Delivery agents are effectively delivering the program within their province.

CAP sites are provided with a dedicated resource.

Evaluation Issue #4: Suggested Improvements for the CAP YI Program

Timely process is required in the delivery of funds.

Delivery agents should meet to share lessons learned or best practices.

More communication from Industry Canada to delivery agents.

Youth and Delivery agents reports need to be reviewed.

Option of longer work terms – 6 months rather than 12 weeks.

Youth Exit Summary Report

A key element in this study was the analysis of the Youth exit surveys. Challenges arose in obtaining the data for the last three fiscal years. In fact, the consulting team was unable to use the data from the FY00/01 as it was corrupt. For the two years that were available (FY99/00 and 01/02) there were difficulties with duplicate entries, as well the data required significant ‘cleaning’ as there were no screening mechanisms in place (e.g., a respondent could enter more than one response when the question should have allowed for only one). Eventually, the consulting team was able to ‘clean’ the data, and the key findings from the analysis of these surveys are as follows:

Many respondents had noted that the program has contributed to their acquiring Internet, technical and HTML/web page design related skills.

The program has also been credited with assisting those respondents who successfully found employment after their internship with finding their current job. Approximately two-thirds (67.9%) of S&T respondents and 56.4% of Student respondents indicated that the CAP YI program helped them find a job.

Respondents have also indicated acquiring general skills including: oral communication, interpersonal & teamwork, organizational, creative thinking, and decision-making skills.

For FY 01/02, approximately 82.7% of S&T youth and 93.1% of Student youth agree that the CAP YI program has contributed to their career development.

56.6% of S&T youth and 51.4% of Student youth indicated that the CAP YI program has influenced their career path.

CAP YI Summative Evaluation Study (PDF, 351KB, 15 pages)

Community Access Program Youth Initiative Summative Evaluation Study — Management Response (PDF, 78KB, 5 pages)

Note: to read the PDF version, you need Adobe Acrobat Reader on your system. If the Adobe download site is not accessible to you, you can download Acrobat Reader from an accessible page. If the accessibility of PDF is a concern, you can have the file converted to HTML or ASCII text by using one of the access services provide by Adobe.