Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

4.1 Introduction

This chapter addresses first the issues associated with the adequacy of the client tracking and monitoring system used to collect information on participants, and interventions for HRDC’s youth internship programs. Also addressed are design features that were specific to the youth internship programs, the Information component and Human Resource Centres of Canada for Students. The concluding sub-section addresses the roles and responsibilities of HRDC in the context of program delivery of youth initiatives under YES.

4.2 Client Tracking and Monitoring Systems for Youth Internship Programs

At the time the formative evaluation was being conducted, the client tracking and monitoring mechanisms in place to collect information on participants in youth internship programs were found to have the following problems:

• The delivery of youth internship programs was not conducive to the electronic capture of participant information. The HRDC case management process used to deliver other programs, such as employment benefits and support measures (EBSM), is more closely involved with the clients themselves than the process for delivering youth internship programs. Typically, HRDC youth coordinators contract with sponsors or employers who are responsible for capturing participant information. In comparison to other HRDC clients, the internship participants are not case managed by HRDC or sponsors to the same degree.

• At the time of the evaluation, systems (NESS and Contact IV) designed to capture participant information for youth internship programs were either not fully deployed or not fully used to capture required youth outcomes. While NESS was eventually modified to accommodate youth clients, Contact IV has not been deployed to the extent anticipated by HRDC. Contact IV is designed for use by HRDC third-party service providers (employers and sponsors) who usually have a long-term arrangement with HRDC to provide employment services to clients and track client information. Youth internship sponsors and employers tend to have shorter-term relationships with HRDC, assisting fewer clients than traditional HRDC third-party service providers. At the time, participant information was generally not entered in either Contact IV or NESS. Since April 1, 1998, HRDC has required the capture of HRDC’s internship participants information for accountability purposes

• While some participant information was in the systems, some information was also stored in paper project files. At the time of the evaluation, HRCCs intended to enter this information in NESS but this process was either in the preliminary stages or had not yet started in most HRCCs. HRCCs are aware of the importance of entering this information for accountability purposes. The level of effort devoted to ensuring that information systems such as NESS and Contact IV are kept up to date varies somewhat among regions and HRCCs.

• The CJS-1 system at HRDC contains information on all youth projects including information on sponsors, employers, and project costs, as well as the number of participants for each project. In fact, the number of participants is often inaccurate. The number of participants for a project is initially entered in the system when a project is approved. If the number of participants changes after approval, the budget and expenditures are usually changed accordingly; however, the number of participants is often left the same. Past experience and consultation with HRDC Management Information System (MIS) staff indicate that this problem is prevalent, but there is no information available on the degree to which the data is inaccurate.

• Management information systems at HRDC relies on data from CJS-1, NESS, and Contact IV. Information from the MIS system will not be accurate for youth programs until the systems feeding it are updated with all required participant information. During the evaluation field work, HRDC did not appear to have the capacity to update these systems in a timely manner.

• Within the interdepartmental reporting process, the major source of information used to monitor the Youth International and Internships in Science and Technology is the quarterly reports. These reports rely on information provided by YES participating departments. While the available information identifies the numbers of participants and expenditures by program component, participant information such as level of education and employment equity characteristics are not generally available. This could limit the extent to which these youth internship programs can be monitored to ensure adherence to program criteria and could present barriers to planning and policy development.

Participant information on many youth internship program participants was not available at the time of the evaluation. For these missing participants, information pertaining to their participation could only be obtained by contacting both employers and sponsors who then provided lists of participants. While this presents problems for the subsequent evaluation of HRDC’s youth internship programs, the more serious problems relate to the lack of capacity to properly monitor the program to ensure that applicant criteria are being respected, and the lack of information linked to client results to fully guide, policy development and management planning with respect to targeting of clients.

4.3 Other Program Design Features for Specific YES Components

4.3.1 Youth Internship Canada

Most project stakeholders (including HRDC personnel, project sponsors, employers and youth participants) perceive that the YIC is providing the degree of flexibility necessary to meet the needs of participants.

According to the information collected during site visits to YIC projects, an operational constraint repeatedly mentioned by interviewees was delays in the start-up of projects caused by the process of obtaining HRDC approval for the projects, prior to the project’s implementation. While awaiting approval, several projects (four out of six) suffered delays ranging from 3–6 months. These delays were particularly problematic for projects that targeted seasonal occupations, such as the construction industry. In many cases, by the time approval was obtained and projects began, many employers were no longer interested or available to provide work experience for the participants.

4.3.2 Information Component

The survey of YIC employers and sponsors addressed the Information component of YES by asking respondents how they had learned of the program. Both sponsors and employers learned about the program in similar ways — primarily from the HRCC, through a variety of media, through previous participation in the program and through business contacts.

There was some variation in the percentage reporting each source, which appears to reflect the nature of the program design as well as the roles and recruitment practices of sponsors compared to employers. For example, employers reported (Exhibit 4.1) learning of the program from interns and personal contacts more frequently than sponsors, whereas sponsors heard about the program more frequently from the HRCC and through previous participation. Few employers and sponsors mentioned advertising as a source.

Source: HRDC Survey of Employers and Sponsors, 1998
* All other responses not presented were given by less than 5% of sponsors.

The survey of YIC participants examined how participants who had initiated the internship learned of the program. The most frequently mentioned sources are shown in Exhibit 4.2. Most participants who approached employers and informed them about YIC had heard about the program from friends or relatives (38%) or from an HRCC (22%).

Source: HRDC Survey of YIC participants, 1998
¹ All other responses were mentioned by less than 2% of respondents.

Youth Career Info Fairs

The Youth Info Fairs appear to provide an overall benefit to its youth participants. The wide variety of information and industry representation demonstrated to youth the extent of opportunities available to them. Furthermore, as one Info Fair volunteer stated, the wide array of information on government programs and services available to them "demonstrates to youth that there are people available and willing to help them find employment." Although most fair attendees did not feel that links were established with employers, a significant number of participants had the intention of applying to the businesses encountered at the fair. These "soft" links between youth and employers contributed to the benefits provided by the Youth Career Info Fairs.

The fair presents an important opportunity to bring community players together, working for the common objective of assisting the youth of the community in their transition from school to employment. From an intergovernmental point of view, the level of cooperation was high as evidenced by fairs investigated, the large number of booths or displays represented by municipal, provincial and federal governments.

However, some concern exists amongst employers that Youth Info Fairs targeted a crowd that was too young to appreciate the labour market information disseminated to them and to understand the viability and importance of the public and private programs and services available.

From an organizational point of view, sufficient time to properly plan and organize youth job fairs appears to be an important element of the planning phase. One of the Info Fair coordinators contended that improvements could have been made to the event had more time been allotted (approximately nine weeks had been allocated to plan and organize the Info Fair). However, in the absence of additional planning time, a clearly defined vision and specific goals delineated prior to the planning phase of the event may assist in improving the overall organization of future youth fairs. It should also be noted that there does not appear to be any mechanisms in place that would ensure "best practices" are documented and used to ensure all regions can learn from the experiences of others.

Youth Link

Several questions were asked of all respondents (survey of 400 YIC participants) about the HRDC Youth Link publication. This is a booklet that describes federal government programs for youth. It was discovered that:

• 96% of respondents did not obtain a copy;
• Of the 12 participants who did obtain a copy, nine found the information they needed; and
• Six interns obtained their copy from the HRCC, two at a job fair, two requested a copy, and one person said it came in the mail.

4.3.3 Human Resource Centre of Canada for Students (HRCC-S)

In the HRCC-S component, over 800 students are hired to work in the HRCC-S network, which provides summer placement services for students. This component employs students to help other students find summer jobs and offers group information sessions on résumé writing, looking for work, and preparing for an interview.

Analysis of the findings from site visits to three HRCC-S indicated that the mandates, objectives, hiring practices and clientele are generally uniform across these centres. It should be noted, however, that the availability of resources dedicated for student client use is not consistent among the centres. For instance, one of the HRCC-S, whose area of service and number of clients appear to be similar to another neighbouring one, currently has three computers (equipped with software packages and access to the Centre’s job bank), a laser printer and photocopier for student use, while the other HRCC-S, does not possess any resources strictly for student use. Presently, this latter HRCC-S staff utilizes their own HRCC computer systems to assist student clients (i.e. with their résumés, cover letters, etc.). As a result, the availability of their own computers, which would otherwise be used for other essential HRCC-S tasks (i.e. updating job orders, accessing employer client databases, etc.) is restricted, as is the amount of time that can be dedicated to each student client.

The HRCC-S appear to offer a number of benefits to both youth and employers. Specific examples include:

• employment of students to deliver the programs and services of the centre allow for:

— student clients to be served by their peers;
— promotion of students as valuable employees in the community;

• student clients being provided with easy access to understandable information;
• access to resources necessary for seeking employment (computers, printer, fax and résumé writing);
• employer clients being provided with information on minimum work standards;
• availability of an employment counselling service; and,
• availability of a job posting service for employers.

4.4 Roles and Responsibilities

Regions and Local HRCCs

Overall, the regional HRDC key informants reported that the roles and responsibilities for themselves in relation to the local program delivery officers were clear and well understood. In a few instances, regional personnel explained that they dealt with public inquiries that could have been better dealt with at the local HRCC level. In other cases, regional personnel reported that, at times they were asked by program officers to make decisions that should really have been made at the local level. Most of these they attributed to having relatively new program delivery officers at the local level.

All key informants reported good and frequent communication between their regional offices and local HRCCs.

Regions and NHQ

Overall, the key informants described their roles and responsibilities in relation to NHQ as clearly understood from their perspective.

The key informants reported good communication from NHQ. However, some of those interviewed questioned whether regional and local concerns were always well understood at NHQ. Specific areas where it was felt that regional and local concerns were not well understood by NHQ include:

• local labour-market situations;
• the impact of resource shortfalls on program delivery at the local level;
• the impact of Labour Market Development Agreements on youth programming; and
• NHQ making assumptions about youth needs that may no longer be valid.

HRDC and Other Federal Departments

One of the main concerns that over half of the key informants expressed was that the interdepartmental youth strategy, while having useful components, was almost too vast to be clearly understood. Roles and responsibilities in relation to other federal departments became difficult to understand for some local program delivery officers as they received inquiries from youth and sponsors regarding other federal government programs under the Strategy. The majority of representatives reported that local HRCCs found it difficult to keep up with all of the different programming, their sources, and their applicability for different youth and sponsors.

Over half of the key informants interviewed cited communication at the local level and between federal departments as an area for improvement. The suggested improvements were to limit overlap between programs, to ensure that as many target groups as possible were being served, to make program delivery structures more efficient, and to reduce confusion for youth and sponsors.