Government of Canada | Gouvernement du Canada Government of Canada
    FrançaisContact UsHelpSearchHRDC Site
  EDD'S Home PageWhat's NewHRDC FormsHRDC RegionsQuick Links

·
·
·
·
 
·
·
·
·
·
·
·
 

Appendix B: Evaluation Methodology


The following are the five principle lines of evidenced used in the Employment Benefits and Support Measures (EBSM) evaluation:

a) Documentation and Administrative Data Review

A review of program documentation was conducted early in the study to understand the program rationale, design, delivery and intended impacts of EBSMs. In addition, program and administrative data were analyzed to develop a profile of EBSM clients and program activity.

b) Key Informant Interviews

A series of forty key informant interviews were conducted as part of the formative evaluation. Interviews were conducted with regional program officials, Human Resources Centre of Canada (HRCC) managers, project officers, and local delivery partners.

c) Case Studies

Case studies were conducted at three HRCCs in Ontario (metro Toronto, Owen Sound and North Bay). The studies were undertaken to illustrate the different service delivery approaches and activities which have developed in response to local needs. Each case study included eight to ten key informant interviews, direct observation of delivery operations (during an on-site visit) and review of site documentation and administrative data (e.g., management and operations, client data, financial information and local labour market information). In addition, two focus groups with EBSM participants were conducted at each site to obtain more detailed feedback from clients on their experience with EBSMs.

d) Surveys of EBSM Participants, Short-Term Group Session Participants and Non-Participants

In total, 1,444 telephone interviews were conducted with EBSM program participants. The sample was defined to include EBSM participants who completed an EBSM intervention between April 1, 1997 and March 31, 1998. On average, interviews occurred between 46 and 54 weeks following the end of the intervention. Survey sampling was based on a random selection, stratified by program type and by language.

In order to measure the incremental impacts of EBSMs on employment and other outcomes, a comparison group of non-participants was identified. The non-participant cases were selected to serve as a comparison group for participants in all EBSMs (excluding those attending a short-term group session) who were Employment Insurance (EI) claimants or "near" reachback clients (i.e., having completed an EI claim within six months of their intervention). In total, 607 interviews were conducted with non-participants.

The comparison group was selected from National Employment Services System (NESS) files linked to Status Vector and Record of Employment (ROE) data. Non-participants were selected based on a similar EI usage profile as participants (i.e., in terms of the length of time between the start of the EI claim and the beginning of the post-program period and in terms of a minimum period on EI defined as the number of weeks into the claim at which participants started their intervention).

In addition to surveys of case-managed EBSM participants and non-participants, a separate survey was conducted of EI claimants who participated in a short-term group session between April 1997 and March 1999. In total, 659 interviews were completed with this group (310 of whom attended a group information session between April 1, 1997 and March 31, 1998 and 349 of whom attended after April 1, 1998). These data were weighted by gender. (Note: the formative evaluation did not include a comparison group for short-term group sessions; as such, the report does not discuss the incrementality of these group sessions.).


[Previous Page][Table of Contents][Next Page]