Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

As a Strategic Initiatives Project, the Vancouver Child Care Regional Delivery Model Pilot Project (VCCRDMPP) had from its inception a strong commitment to the various evaluation processes implemented during the Project. Strategic Initiatives Projects have participated in three separate sets of evaluation activities: a formative evaluation, a process evaluation and an outcome evaluation. Three evaluation reports were produced during the first three years of the Vancouver Child Care Regional Delivery Model Pilot Project (VCCRDMPP).

The first was A Formative Evaluation - Strategic Initiatives Pilot Project (April 1996) produced by Karyo Communications Inc. involving VCCRDMPP as one of seven BC Strategic Initiatives Projects. The goal of this study was to "document program models and their development and implementation to provide a basis of learning and guidance." Data was collected through a series of focus groups using a series of questions developed by the Ministry of Women's Equality, the government ministry then responsible for child care initiatives in BC.

Process evaluation involved two sets of activities. In July 1996 a report, A Process Evaluation - Strategic Initiatives Pilot Projects, was also developed by Karyo Communication Inc. This report "documents factors that impacted on the delivery of program models and services." As part of this study, one focus group was held with each of the four components of VCCRDMPP using questions developed by the Ministry of Women's Equality and revised in consultation with VCCRDMPP staff. Subsequent to this process evaluation, it was agreed by VCCRDMPP participants and the Ministry that the ongoing quarterly reports submitted by the Project describing Project activities and identifying barriers and emerging issues would constitute the process evaluation for the duration of the Project.

Outcome Evaluation was undertaken beginning in June 1997 when VCCRDMPP engaged an evaluation consultant, Adele Ritch, to conduct evaluation of the Project for Year Three (1997-98) and Year Four (1998-99). In June 1998, VCCRDMPP submitted the Strategic Initiatives Vancouver Child Care Regional Delivery Model Pilot Project Interim Evaluation Report: Year Three. This report described how activities of each of the four VCCRDMPP components had addressed expected outcomes to March 31, 1998. Data was collected through a series of face to face interviews with each of the fourteen RUG members, a written questionnaire completed by each of the RUG members, interviews with other key informants, document review and participant-observation at most RUG meetings. Outcome evaluation was limited by the absence of baseline data for all Project components.

The original plan for the evaluation of the project was to continue with outcome based evaluation throughout the Project's fourth year ending March 1999, and then to submit a final evaluation report. However, in July 1998, a revised Strategic Initiatives Project Framework for Evaluation was developed by the BC Ministry for Children and Families. The Framework now included a Summative Evaluation component which was to assess the degree to which VCCRDMPP had addressed the expected outcomes of individual demonstration projects, the expected outcomes of the Regional Delivery Model/Community Demonstration Projects and the expected outcomes of Child Care Strategic Initiatives Projects. The Summative Evaluation was to include review and analysis of Project evaluation reports and quarterly reports written to date and was to be submitted by January 31, 1999 rather than the planned March 31, 1999.

VCCRDMPP in consultation with its Evaluation Consultant revised the fourth year data collection plan to accommodate the new timelines. Data collection proceeded earlier than had been anticipated, and as a result, not all Year Four Project activities could be included in the Summative Report and some data collection originally planned for year four was not carried out.

Qualitative data collection methodologies were used in all of the reports used for this summative evaluation. These included focus groups, one-to-one interviews in person and by telephone, written questionnaires, document review and participant observation. Data from the following sources was used in the development of this Summative Report:

• A Formative Evaluation - Strategic Initiatives Pilot Project (Karyo Communications, April 1996)
• A Process Evaluation - Strategic Initiatives Pilot Projects (Karyo Communications, July 1996)
• Strategic Initiatives Vancouver Child Care Regional Delivery Model Pilot Project Interim Evaluation Report: Year Three (Ritch, June 1998)
• VCCRDMP Quarterly Reports Years One through Four (Project)
• VCCRDMPP Data collection, April 1, 1998 - September 30, 1998 which included:

RUG:

Levels of Project Outcomes

In keeping with the Strategic Initiatives Evaluation Framework, the following three "levels" of outcomes of VCCRDMPP were assessed:

Expected Outcomes of Individual Demonstration Projects

Because each of the four VCCRDMPP Demonstration Projects had its own unique set of goals, activities and expected outcomes, a separate evaluation plan was developed and implemented for each project. Discussion of each demonstration Project is contained in Section Three of this report which details each Project's activities and outcomes.

Expected Outcomes of the Regional Delivery Model Community Demonstration Projects

The Strategic Initiatives Evaluation Framework also required that each of the Regional Delivery Model/Community Demonstration Projects be assessed according to a common set of expected outcomes. The VCCRDMPP was intended to address some but not all of the following outcomes:

1. Reduced management pressures on individual child care programs
2. Increased overall stability of child care programs
3. Improved regional planning, development and delivery of child care
4. Improved quality particularly in the unlicensed sector
5. Improved cost efficiencies of the child care system and affordability for parents
6. Expanded and improved services in communities that are typically hard to serve

Expected Outcomes of Child Care Strategic Initiatives Projects

As a Child Care Strategic Initiatives Project, the project was assessed according to the degree to which VCCRDMPP met the Expected Outcomes of Child Care Strategic Initiatives Projects. The VCCRDMPP was intended to address some but not all of the following outcomes:

1. Increased affordability including reduced or stabilized costs
2. Improved quality including training and supports for parents and caregivers
3. Increased accessibility including a range of service options that meet family needs
4. Improved responsiveness including a service continuum, multilingual services as well as resources, referrals and supports
5. Improved inclusiveness including equal access to child care through support to families and child care settings
6. Cost efficiency or effectiveness including administrative or management cost efficiencies and client satisfaction