|
|
     |
||
 |
     |
 |
|
Overview Report:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![[Table of Contents]](/web/20060127000211im_/http://www11.hrdc-drhc.gc.ca/edd-img/content.gif) |
The BC Improved Access to Child Care Project is a joint federal-provincial strategic initiative funded and managed by Human Resources Development Canada (HRDC) and the British Columbia Ministry of Women's Equality.
This evaluation study was conducted by Karyo Communications under the direction of the Joint Evaluation Working Group, comprising representatives from the federal and provincial governments.
The Joint Evaluation Working Group would like to thank all who contributed to the study, especially community stakeholders, project representatives, and officials of the Ministry of Women's Equality, Child Care Branch, who gave of their time and experience.
|
This report presents the findings of the formative evaluation of the joint Canada/British Columbia Strategic Initiative: Improved Access to Child Care project1. The formative evaluation addresses issues surrounding the start-up of the project in fiscal year 1995-96. As such, the focus of this report is on issues related to: program design and delivery; program co-ordination; and program systems development.
This report is an aggregation of formative evaluation findings and analysis of individual child care projects that were funded under the CCSI project in the 1995-96 fiscal year. The projects included:
The formative evaluation was conducted between January and March, 1996, by a private consulting firm under contract with the Research, Evaluation, and Intergovernmental Affairs Branch of the Ministry of Women's Equality. Appendix A presents the evaluation questions developed by the Joint Evaluation Working Group 3.
Data collection for the formative evaluation included a review of the background and planning documents of both the Ministry of Women's Equality and the sponsoring societies of each of the individual child care projects. Data were also obtained via focus groups and individual interviews. The interviews were conducted with key community stakeholders from each of the child care projects, current and former staff of the Ministry of Women's Equality involved in the CCSI project, and a representative of Human Resources Development Canada who was involved in this project since its inception.
The formative evaluation is one component of an evaluation framework jointly developed by the JEWG. The other components of the framework are a process evaluation and a summative evaluation. The framework is discussed in detail in Section 1.5, Evaluation Framework: Purpose and Description.
|
|
1.1 Social Security Reform and Federal/Provincial Strategic Initiatives Projects
Assisting individuals in reducing their dependence on income assistance programs is a priority of both the federal government and the government of British Columbia. Recent initiatives by both orders of government demonstrate their support for improving economic self-reliance through increased labour force attachment.
For example, British Columbia's inauguration of the BC Benefits initiative in 1995 heralded the province's commitment to realising its vision of a social safety net that supports individuals facing financial hardship, while at the same time fostering their economic independence over the long term. Fundamental to this commitment, and BC Benefits specifically, is the recognition that the majority of new jobs created will demand highly specific skills, requiring training beyond the high school level.
The recognition of the need for higher skill levels led to BC Benefits linking eligibility for benefits to participation in employment-related education and training programs. It is expected that employment-related education and training programs will provide individuals with the skills they need to succeed in the new labour market.
The province's commitment is further demonstrated by the support afforded families with limited income. Specifically, the BC Benefits (Child Care) Act provides financial support through child care subsidies, which helps parents to access child care services while they work or attend employment-related education and training.
Federally, social security reform has focused primarily on the overhaul of Canada's Unemployment Insurance system. Like British Columbia's BC Benefits initiative, eligibility criteria for Unemployment Insurance benefits have been reformed to encourage individuals to pursue training that will improve their chances of securing employment.
In concert with UI reform, the federal government has explored other courses of action to contribute to the process of social security reform being pursued at both the federal and provincial levels. The Strategic Initiatives Program is an example of federal commitment to social policy reform.
The February 22, 1994, federal budget announced the creation of the Strategic Initiatives Program, a joint federal-provincial/territorial cost-sharing venture. Under this initiative a total of $800 million (later reduced to $400 million) would be allocated in the 1995-96 to 1998-99 budgets.
The projects funded under the Strategic Initiatives Program enable the federal government, in partnership with provincial/territorial governments, to experiment with new and emerging ideas about social security. Strategic Initiative projects include unique approaches to employment and self-employment, training, apprenticeship and learning, child care and family development, earnings and income supplements, work experience, and a host of other new approaches.
Projects funded under the Strategic Initiatives Program are expected to:
improve job opportunities for residents of Canada;
enable those facing serious labour market problems to overcome barriers to successful adjustment; and
reduce dependence on the social security system.
1.2 British Columbia Child Care and The Strategic Initiative Program
For the majority of families, economic well-being and access to child care are inextricably linked. For example, 65 percent of children in British Columbia age 0-5 and 70 percent of children age 6-12 are cared for outside the home because parents are either working or pursuing academic/skills training. For many families, child care costs consume a substantial portion of their annual income. In 1987, single-parent families with children under 6 years of age allocated approximately 12 percent of their total annual income to child care, and two-parent families with children in this age group allocated 4 percent of their total annual income to child care.
Access to affordable child care not only impacts on parental decisions regarding what type of child care arrangement to use, but also poses a major barrier to training and labour force attachment among single parents and low-income dual-earner families. Single parents and low-income dual-earner families face a tough economic and social decision - the choice of staying at home and caring for children with the associated loss of earned income, or paying for child care. For the single parent who does not receive adequate family maintenance, income assistance is usually the only alternative when affordable child care is not available.
The link between access to child care and economic self-reliance is of foremost concern in British Columbia. As previously noted, the financial support for access to child care provided by the BC Benefits initiative demonstrates the importance that the provincial government places on this link.
The British Columbia Ministry of Women's Equality promotes the development of a child care system that is accessible, affordable, and provides quality care to British Columbia's children. In a step aimed toward reaching such a child care system, former Women's Equality Minister Penny Priddy announced in June, 1995 that British Columbia would embark on a joint federal-provincial Strategic Initiative (SI) project. The Minister stated that the CCSI would test new and innovative methods of child care service delivery aimed at promoting accessible, affordable, quality child care.
The Canada/British Columbia contribution agreement represents a four-year, $32 million commitment by both orders of government. The costs are to be shared equally.
To date, a number of SI agreements have been signed across Canada. The CCSI is unique in that it is the only SI to specifically address the issue of improving child care accessibility. The evaluation of this project will provide a comprehensive examination of the issues surrounding the planning, design, delivery, and effectiveness of new and innovative ways of managing and delivering child care services.
The potential contribution of the CCSI to social security reform is best captured in Minister Priddy's announcement of the project's inauguration:
By making more safe, affordable child care choices available to more families, we're helping parents get the child care they need to take advantage of jobs, education and training opportunities to support their families. That in turn will reduce the long-term costs of social programs for taxpayers and keep our economy strong and growing.
1.3 The British Columbia Child Care Vision
Since 1993, responsibility for the child care portfolio has resided with the British Columbia Ministry of Women's Equality. The goal of the child care portfolio is to help provide a continuum of child care options which best suits the needs and preferences of parents in British Columbia. These options range from licensed full- and part-time programs, licensed school-age care, agency supervised and supported family home day care, to resources and support services for parents, caregivers, and child care programs.
British Columbia's comprehensive child care system rests on the philosophical foundation that:
the health and well-being of the child are of paramount importance;
the provision of quality child care is a family issue and is required to meet society's needs;
the responsibility for the provision of child care is shared between parents, caregivers, the community, and all levels of government; and
parents have the responsibility and right to choose the type of care that best suits their family's needs.
Since assuming responsibility for the provincial child care portfolio, the Ministry of Women's Equality has made, during an era of public sector fiscal restraint, significant inroads in a historically under-funded area. For example:
In 1994-95, $3.63 million helped support 31 Child Care Support Programs (CCSPs), and create three new CCSPs in the communities of Smithers, Grand Forks, and Williams Lake, as well as a satellite branch in Bella Coola. CCSPs serve two client groups: parents, and child care providers in the licensed and the licence-not-required sectors. To parents, it provides a resource and referral service on child care choices available in their communities. To child care providers in the licensed and the licence-not-required sectors, CCSPs provide training and support in delivering quality child care.
In 1994-95, the Wage Supplement Initiative resulted in an average wage increase of 10 percent for child care workers at eligible child care centres in the non-profit and private, licensed child care sector, thereby helping to stabilise 31,895 licensed child care spaces.
The provincial Facilities and Equipment grants have helped to create 2,116 new licensed child care spaces, and retained 824 spaces. The Emergency Repair, Replacement and Relocation Grants helped maintain and support 11,404 licensed non-profit group child care spaces. The Infant/Toddler Incentive Grant program has assisted in maintaining 4,000 licensed spaces for infants and toddlers.
In June 1994, the provincial government launched the Child Care: Choices at Work plan which provided $42 million in new money to create more child care choices to meet the needs of working families. The focus of the plan is to renew child care to better meet local needs. The vision of the Child Care: Choices at Work plan is to make child care available at a cost families can afford, and to ensure that children are cared for in safe and healthy environments. BC Improved Access to Child Care is part of this plan for child care renewal (see the accompanying chart for an overview of the CCSI).
1.4 Components of the CCSI Project
By March 1995, approximately $2 million new dollars had been invested in child care through the CCSI project. Table 1 provides a breakdown of monies allocated to projects for start-up and operational costs 4.
Table 1
| 94/95 | 95/96 | 96/97 | |
| Nelson OSA* | $47,416 | $189,551 | $150,972 |
| Terrace OSA | $5,522 | $161,062 | $164,040 |
| Vancouver OSA | - | $153,150 | $300,000 |
| Courtenay/Comox OSA | - | $121,161 | $142,713 |
| OSA Total | $52,938 | $624,924 | $757,725 |
| Oaklands | - | $ 47,600 | - |
| Vancouver Regional Umbrella Group | $7,200 | $240,738 | $261,989 |
| Regional Delivery Model Total | $7,200 | $288,338 | $261, 989 |
| ANNUAL TOTALS | $60,138 | $913,262 | $1,019,714 |
As shown in Table 1, to date $1,435,587 million has been allocated to the One-Stop Access projects, and $557,527 has been allocated to the Regional Delivery Model projects (i.e., Vancouver RUG and Oaklands)5 .
In British Columbia, child care needs, both in terms of the type of care required, and the number of child care spaces required, vary greatly from region to region and, within regions, from community to community. Therefore, to realise its vision of an affordable and accessible child care system that nurtures children's physical, emotional and cognitive development, the province has adopted a decentralised approach.
Decentralisation has resulted in the development of provincial/community partnerships. For example, the provincial government is working with communities in developing models of child care delivery and management that are responsive to local needs.
The purpose of the SI agreement was to test new and innovative methods of child care service delivery. To this end, the CCSI project is testing three separate components:
As will be discussed in Section 2, these components share the common feature of extensive community involvement in, and ownership of, the design and the development of the component's project model.
The following is a cursory explanation of the three project components of the CCSI.
Component 1
Community One-Stop Access (OSA) Centres were developed in partnership with the Ministries of Social Services, Health and Women's Equality, as well as families, caregivers, and communities. The defining feature of OSA Centres is the co-location of a Ministry of Social Services financial assistance worker and a Ministry of Health licensing officer with the services of the community Child Care Support Program. The consolidation of child care services under one roof provides parents and child care providers with a single point of entry where a host of information and resources pertaining to child care are readily available. For example, parents and caregivers are provided with one-stop access to:
information on and assistance with child care subsidy applications;
information on child care licensing and child care grant programs;
child care choices and referrals to services; and
training, resource and equipment lending.
OSA Centres also play an essential role in needs assessment and planning for child care at the local level.
Appendix B presents a logic model which delineates the critical linkages between the services provided by OSA Centres and the expected outcomes of the CCSI (i.e., improved accessibility, affordability, quality, cost-effectiveness, and responsiveness of the child care system).
The co-location of a Ministry of Social Services financial assistance worker was introduced to present parents with the option of applying for child care subsidies in a location other than the local social assistance office. It was believed that parents, especially those in small communities, tended to forego applying for either a partial or full child care subsidy to avoid the stigma of being mistaken for a social assistance recipient. By providing an alternative means of applying for child care subsidies it was hypothesised that more working families would apply for partial or full subsidies. In effect, this change in the application process would make child care more affordable for many working families.
Component 2
Community Demonstration Projects test one or more of the following:
the integration and co-ordination of child care services in a given community, including but not limited to: local planning; development of new and/or satellite services to meet specific community needs; grants administration management; efficiencies such as financial and administrative services; purchasing caregiver and parent library resources and equipment lending; and caregiver and parent training and workshops;
approaches to improving the quality of child care services in a variety of settings, including the unregulated sector;
improved accountability of public funds; and
new and innovative ways of providing child care that deliver local solutions to particular problems.
Component 3
The Supported Child Care Project is designed to meet the needs of more than 3,000 children requiring special needs day care in British Columbia. The BC Ministry of Social Services is conducting this project and its evaluation. The Ministry of Social Services is providing a separate formative evaluation report on this project.
Each OSA Centre and Community Demonstration Project7 is based on a Child Care Support Program (CCSP). By utilising the existing environment, in essence working in concert, OSA Centres, Community Demonstration Projects and CCSPs can move child care toward a system that serves local needs. As noted above, the mandate of CCSPs is to increase accessibility through the recruitment and referral of child care providers, and the provision of child care-related information and resources. In meeting its mandate, CCSPs have established strong links with parents, child care providers, and other community-based organisations. These links position the CCSP where it can, through OSA Centres and Community Demonstration Projects, play an integral role in expanding and stabilising the child care system. The result is that OSA Centres and the Community Demonstration Projects build upon the CCSP core foundation of services, for example:
expanding CCSP services through the extension of geographic boundaries, creation of satellite services, or extended support (i.e., workshops, training, resources) beyond the licensed family day care sector to licensed group care providers; and/or
stabilising the child care system by assisting licensed group child care services with administrative and financial planning and management; providing increased support to the licence-not-required child care sector; and developing child care that is flexible enough to address difficult-to-meet child care needs such as emergency care, rural or seasonal child care.
A complete list of the projects funded in the 96/97 fiscal year can be found in Table 2.
Table 2
| Pilot Project | Pilot Project Description | Funds Allocated 96/97 |
| 100 Mile House In Child's Home Care Pilot Project | Test a model of improving services to children and families living in rural settings by offering care in each child's home. | $72,750 |
| Chilliwack Licence-Not-Required Registration Pilot Project | Test new ways of registering licence-not-required (LNR) caregivers with the desired outcome of improving the accessibility, quality, and accountability in the LNR sector and examining ways to improve service responsiveness and inclusiveness. | $88,068 |
| North Shore Registration and Administration Pilot Project | Test ways to achieve administrative efficiencies in the management of child care through the development of management and administrative services to licensed and LNR child care providers in the North Shore, Sea-to-Sky, and Sunshine Coast regions. | $217,241 |
| Penticton Rural Child Care Project | Test ways of improving child care accessibility for individuals working in the area's seasonal fruit and tourist industries. | $61,126 |
| Surrey Local Solutions to Local Needs Services | Test ways of enhancing child care services for the Surrey community-based on expertise, demographics, and consultation with staff, caregivers and other agencies. | $111,729 |
| TOTAL | $550,914 | |
1.5 Evaluation Framework: Purpose and Description
Evaluation is a critical component of the CCSI project. It assesses the extent to which the project has achieved the goals set out in the Canada/British Columbia contribution agreement, and identifies the successes and failures. Furthermore, the evaluation provides valuable information on issues surrounding the development, implementation, and delivery of the child care models. This information can guide others contemplating similar child care management and delivery models in their community.
Provincially, enhancing accountability for public sector performance is a priority. The project monitoring and evaluation activities intrinsic to the CCSI project fit well with this provincial priority. In addition, the monitoring and evaluation activities will provide all parties with information regarding issues pertaining to planning, implementing, and delivering programs in a partnership arrangement.
The evaluation of the CCSI project is a complex task. For example, the scope and nature of data collected must satisfy the information needs of the federal government, the provincial government and, ultimately, the community stakeholders who designed, implemented, and delivered the child care projects.
The following provides an overview of the information requirements of the three partners:
Community Stakeholders
At the community level, each project manager requires specific information that will help them assess which aspects of their project are working well, and identify areas where improvements to their "model" are required (i.e., the type of service offered, the provision or delivery of a service).
Provincial Government
The CCSI consists of a number of discrete, qualitatively different projects, each aimed at addressing specific child care accessibility issues currently being faced in British Columbia 8 . To make informed policy and program decisions regarding future funding and/or expansion of particular projects, the province requires information specific to each project.
Federal Government
The federal government requires information regarding the overall accomplishments of the CCSI, and clarity regarding the factors to which successes and failures are attributable.
To satisfy the information needs of all three partners, the evaluation framework developed for the CCSI identified three major evaluation projects:
A formative evaluation, the subject of this report, investigates the issues surrounding the planning, design, and processes leading up to the implementation of the CCSI project.
The formative evaluation addresses the issues of planning, design and implementation from both a macro and community level. On a macro level the formative evaluation addresses a number of core issues. These core evaluation issues are common to all evaluations of Strategic Initiatives across Canada. Data from the OSA Centre projects and Community Demonstration Projects will be aggregated to address the core evaluation issues and questions.
To provide information for the decision-making process at the provincial or community level, the formative evaluation also assesses the challenges and opportunities experienced in the planning, design and implementation of projects at the community level. In addition, evaluation on a project-by-project basis is required because issues arise that are relevant only to a specific OSA Centre project or Community Demonstration Project.
Specific project information will enable program managers implementing similar projects to anticipate potential barriers, and in turn allow for the formulation of a plan of action to meet these challenges should they arise. Similarly, documenting processes that worked well will inform program managers which activities or processes should be included in their program planning and development plan. A summary of key findings in this regard is presented in the concluding section of this report.
A process evaluation will examine the successes, as well as the challenges and barriers affecting the delivery of services following project implementation. As with the formative evaluation, the process evaluation will address the issues on both a community level (i.e., on a project-by-project basis) and a macro level (i.e., aggregation of projects).
The purpose of the process evaluation activity is twofold. First, the delivery of services must be evaluated. Central to the evaluation will be the assessment of factors that impact on the project's ability to improve the accessibility, quality, affordability, inclusiveness, and responsiveness of the child care system. The second purpose of the evaluation process is to provide feedback to project managers regarding which aspects of their child care service delivery are operating successfully, and which aspects need adjustment or re-designing.
A summative evaluation will be conducted at the end of the initiative, in 1999-2000. It will measure the incremental outcomes, impacts and cost-effectiveness of the CCSI. The key issues to be addressed include:
|
In September 1995, the CCSI Joint Evaluation Working Group drafted the evaluation issues and questions to be addressed in the formative evaluation. These evaluation questions pertain to the core issues of planning, design, and development.
As noted previously, the core evaluation issues and questions are common to all evaluations of Strategic Initiatives across Canada. Transforming these core issues and questions to reflect the nature and scope of the CCSI was the task of the JEWG. To this end, the formative evaluation was designed around the three themes of program design and delivery, co-ordination, and systems development.
Presented below is an analysis of these three thematic issues.
2.1 Program Design and Delivery
2.1.1 Program Relevance
In 1993, a submission to Cabinet projected that approximately 330,000 non-parental child care spaces were required to meet British Columbia's child care needs. In light of this estimate, the relevance of the CCSI is clear.
As discussed earlier, local child care needs vary from region to region and from community to community. To develop program models that were responsive and relevant to local needs, the Ministry of Women's Equality, Child Care Branch, solicited proposals from Child Care Support Programs and their sponsoring societies. Approved program models were funded as projects under the CCSI.
The evaluation found that two activities occurred during the initial development phase for those program models where there was consensus regarding the types of child care services and activities to be provided. One was the use of public consultation to solicit input on child care needs from the community (i.e., caregivers, parents, child care-related agencies and committees). The second involved using the findings of community child care needs assessments. A review of the information collected found that the most significant impact of these two activities was that they helped to identify priority areas of child care services and activities for individual communities.
The importance of community consultation during the model development phase is evidenced by the problems encountered by one of the projects 9. Evaluation findings specific to this project concluded that:
there was a lack of a needs assessment, lack of involvement in planning by a community-based advisory committee, and lack of consultations with community stakeholders;
consensus regarding the child care activities was difficult to reach, and there were significant difficulties in developing a workable proposal;
once the project activities had been specified, staff found that the activities were difficult to implement, and once implemented, difficult to administer; and
the sponsoring society was not located in the community, making communication a difficult process, and in consequence, effective logistical planning was compromised.
The problems initially encountered by this project were alleviated once a Steering Committee with broad community representation became involved in the development process, and the sponsoring society had been shifted to a local community organisation with strong ties to the existing child care community.
The evaluation findings clearly show that community involvement early in the process is not only key to determining which child care services and activities are required in the community, but is key to effective implementation and administration of the project.
2.1.2 Program Policy Guidelines
As noted in Section 1, a priority of British Columbia is to promote and foster a child care system that is accessible, affordable, cost-effective, of high quality, and responsive to local needs.
i) Piloting New Approaches
Prior to the Canada/British Columbia SI partnership, the provincial Child Care: Choices at Work initiative provided the province the primary opportunity to pilot approaches that would promote and foster an improved child care system. The advent of the CCSI has increased the opportunities for testing new and innovative approaches to child care across British Columbia. For example, the joint funding arrangement of the CCSI has resulted in an increase in the number of child care projects undertaken. This increase has alleviated some coverage problems associated with the provincial Child Care: Choices at Work initiative. Specifically, the additional projects helped to ensure regional representation and equality at both community and parent levels.
In addition, the Strategic Initiative Program provides for funding of original research into child care. It is anticipated that this research will form the foundation of new child care policies and programs across Canada, at provincial and national levels.
ii) Project Planning
As a prelude to the June 1995 signing of the Canada/British Columbia SI Contribution Agreement, the federal and provincial governments reached, in June 1994, an agreement in principle to enter into a SI partnership. Immediately following the agreement in principle, the project planning process was initiated by the province, one year in advance of having a signed agreement.
The early start to project planning was not initially viewed as problematic in that the projects proposed for the CCSI were already designed and scheduled for implementation under the province's Child Care: Choices at Work plan.
However, in negotiations with the federal government during 1994, the project model parameters and expectations for community participation initially defined by the province changed. This change in parameters and expectations, coupled with the province's goal to implement the projects quickly, resulted in a number of issues and challenges at the community level. Interviews with community stakeholders identified the following issues and challenges:
Short timeframes for the submission of proposals
The timeframe for developing project models and submitting project proposals was approximately six months. Community stakeholders perceived this to be insufficient. Reasons cited by the stakeholders included the requirement and need for extensive community consultation and a lack of resources to prepare the proposal. In response to the resource issue, the Ministry of Women's Equality provided funding for groups to hire a consultant to develop and write the proposal.
A lack of clarity with respect to proposal requirements
Proposal requirements changed as project parameters and expectations were modified. This resulted in revisions having to be made to submitted proposals, including revisions to budgets. Community stakeholders found this process to be frustrating.
Delays in the review and approval process
The review of proposals and approval of funding for projects by the Ministry of Women's Equality were delayed. As a result, communities experienced delays in implementing their projects. While the delays did not affect the project activities and services that were ultimately implemented, or the project's potential contribution to child care, they caused feelings of frustration among community stakeholders toward the Ministry.
Ambiguity surrounding funding
Without having the Canada/British Columbia contribution agreement in place, the Ministry of Women's Equality could not provide any assurances regarding what kinds of activities would be funded and what items would be eligible. Community stakeholders stated that they were not informed that proposed funding levels could be subject to change once the contribution agreement was signed. After the signing of the agreement, proposed project budgets had to be revised to reflect the available funding.
The ambiguity surrounding funding created concern and anxiety among stakeholders. However, the level of funding received by each of the projects, once the contribution agreement was signed, was substantial in this historically under-funded area (see Table 1). In addition, the project proposals, which were accepted and implemented, have retained, or improved upon, the majority of services and activities initially proposed. For example, the OSA Centres have witnessed a proliferation of co-located community-responsive child care-related services not initially included in the project proposal.
The lack of a signed Canada/British Columbia contribution agreement also had significant impacts at the corporate level. Interviews with former and current staff of the Ministry of Women's Equality indicated that there existed, during this period, a difficult and ambiguous working climate. This, in combination with staff turnover, resulted in communication difficulties between the community stakeholders and the Ministry of Women's Equality.
Ideally, longer timeframes for the planning and implementation of the child care projects would have been desired. However, the province believed that the funding provided through the Strategic Initiative agreement, specifically the $32 million in new funding for child care, was at risk. Therefore, the province believed it imperative that it demonstrate its commitment to leading a Strategic Initiative project on child care. The province believed this commitment would best be demonstrated by the immediate start-up of the types of projects slated for implementation under the earlier provincial Child Care: Choices at Work plan.
In addition, it would have been preferable if the project planning and implementation activities had been initiated after the Canada/British Columbia contribution agreement had been signed. However, as noted above, the planning and implementation activities had already commenced under the province's Child Care: Choices at Work plan.
Since signing the Canada/British Columbia contribution agreement, and since the stabilisation of staff at Child Care Branch, the Ministry of Women's Equality has:
provided project personnel with information packages on the overall objectives of the Strategic Initiatives Program;
identified internal deadlines for project proposal submission and implementation, and provided this information to all appropriate parties;
with the involvement of project personnel, undertaken an extensive review of its processes for overall project development and contract management for Strategic Initiatives projects;
assigned responsibility for projects to individual staff members so that project personnel would have consistency in their contact with the Ministry; and
moved more quickly on contract negotiation with the community by reducing the Ministry's level of control over the more minute aspects of project management and provided clear direction of the areas where the Ministry could be most flexible.
To date, all of the 1995-96 funded projects have been successfully implemented and are delivering services to their community.
The process and protocol surrounding the development of project proposals and submission for funding in the 1996-97 fiscal year has proceeded smoothly. New community groups have not faced the issues and challenges their predecessors experienced.
iii) Project Selection
The Ministry of Women's Equality has in place a process for approving child care project proposals. This includes:
the Child Care Branch, using a point system, reviews the extent to which the proposed project activities support the Strategic Initiatives goals of improved accessibility, affordability, quality, responsiveness, inclusiveness, and cost-effectiveness of the child care system;
the Child Care Branch assesses the degree to which the model represents a new and innovative approach to delivering and managing child care services; and
approval by the Child Care Policy Team 10. Before approval, the Child Care Policy Team identifies issues that require clarification, and/or requests changes to the proposed documents. The Ministry of Women's Equality staff, in turn, relay these requests for clarification back to the community groups for action before final approval is given by the Child Care Policy Team.
There is no evidence to suggest that this process has been anything other than efficient.
2.2 Co-ordination
2.2.1 Information Sharing and Communication
Initially, information sharing with project stakeholders was slow. Several months passed between the signing of the contribution agreement and informing the communities of the terms of agreement for which they would be accountable.
Information gathered from focus groups and interviews shows that the sharing of this document with all stakeholders was of paramount importance in developing clear expectations regarding program policies, activities and objectives. In addition, the sharing of the document paved the way for successful strategic planning and effective program development procedures at both the corporate and community levels.
Regarding the OSA Centre component of the CCSI, there were two additional issues that had a significant impact on the planning and development phases. These issues were the co-location of unionised workers with non-unionised workers, and the province's ability to provide "full-time equivalents" (FTEs) for co-location.
The co-location of Ministries of Social Services' financial assistance workers (FAW) and Health's licensing officers (LOs) at OSA Centres presented a unique challenge to the already complex issue of unionised employees working alongside workers who are not members of a union. Specifically, FAWs and LOs are members of the British Columbia Government Employees Union (BCGEU). However, the Penticton and Vancouver OSA clerical staff, who would be scheduling their appointments with parents and care providers, are not unionised. Under the BCGEU collective agreement, this arrangement constitutes "contracting out" work to non-union personnel. This contravention of the BCGEU collective agreement is an issue that requires a resolution if FAW and LO services are to co-locate under the OSA Centre roof.
To date, the issue of how to co-locate union and non-union staffs remains under discussion. Various union certification models are being explored. The importance of involvement of all key stakeholders in these discussions is highlighted by the confusion that exists with respect to alternative certification models and union policy in general. This experience demonstrates that the complexity of collective agreements and sundry certification models requires clear communication among stakeholders. Effective communication could be achieved through meetings with all stakeholders in attendance so that all leave with the same knowledge and common understanding of union policies and issues.
The second issue to emerge with respect to the co-location of the Ministry of Health and Social Services staff was that of limited human resources in the British Columbia public sector. Specifically, FAWs and LOs already carry heavy caseloads and the addition of OSA responsibilities would increase their caseload substantially. The Social Services Ministry indicated that it was not feasible to have existing staff co-locate at an OSA Centre.
This issue has been temporarily resolved by the Ministry of Women's Equality continuing to fund a .5 FTE per OSA Centre. This issue of ongoing funding has not been resolved and is under discussion at the inter-ministerial level.
2.2.2 Staff Turnover
During the first year of the CCSI project there were numerous changes in staffing. As of August 1995, the Ministry of Women's Equality saw the introduction of a new director for the Child Care Branch, a new SI Child Care Renewal team, a new Assistant Deputy Minister of programs, and a new evaluation analyst. This lack of continuity in staff had ramifications at the community level. Community stakeholders were frustrated at the time expended to explain their project to new staff.
Since August 1995, staffing has remained consistent. New staff have invested considerable time in getting to know the various projects and community groups. The community groups have stated that they have found the new staff to be sympathetic to the frustrations of the past and willing to work closely with them to ensure the process works smoothly in the future.
The issue of staff turnover highlights the need for comprehensive documentation of program activities and processes. Without this background information, including successes and challenges, new staff will have to begin at square one. This in turn would place increased workloads on community stakeholders as they bring new staff up to speed, resulting in stakeholder frustration with the Ministry.
2.2.3 Roles and Responsibilities
The formative evaluation identified two key areas where the roles and responsibilities of the various players were initially unclear or not carried out as effectively as expected. They are as follows:
Implementation Co-ordinatorsTo assist communities in implementing their projects, the Ministry provided each community with the services of an implementation co-ordinator. The role of the co-ordinators was to develop program policies, and to manage and resolve emerging issues.
Community stakeholders and the Ministry of Women's Equality, Child Care Branch agree that the implementation co-ordinators experienced difficulties in carrying out their assigned duties. At the community level, these difficulties resulted in delays in project development.
The problem with the implementation co-ordinators can be traced to a public hiring freeze. Initially, the Ministry had planned to hire individuals with considerable expertise and knowledge of the child care issues in each of the project communities. However, because of the hiring freeze, the Ministry had to turn to secondment of existing provincial employees to staff the implementation co-ordinator positions. Unfortunately, the seconded staff did not have sufficient experience in applying the knowledge and skills required to do the job of implementation co-ordinator.
After a few months it became necessary to remove the implementation co-ordinators. The Ministry of Women's Equality, Child Care Branch then hired an individual with extensive knowledge in corporate program development and implementation, community-based program planning and development, social policy and child care programs, and policy development. CCSI project staff indicated that their experiences in implementation improved following the hiring of this individual.
Informal Policy and Procedures Issues
The issue of how to apply informal policy and procedure guidelines at OSA Centres arose regarding child care subsidy records. The Ministry of Social Services' informal "second set of eyes" policy dictates that, when an applicant is interviewed for financial assistance, two Ministry staff members must verify that the person who is applying for assistance is indeed the individual in question.
Given that the financial assistance worker (FAW) is the only Ministry staff member to work out of the OSA Centre, there was initial concern regarding how to implement this procedure. Community groups wanted protocols on this issue resolved at the inter-ministerial level in Victoria, rather than at the local level. This issue was ultimately resolved with the Ministry of Social Services agreeing that an OSA Centre staff member (such as the receptionist) could assume the role of the "second set of eyes".
The emergence of this issue highlights the need for partners to be fully aware of the informal policies that have become de rigeur with respect to an organisation's workplace environment. Organisations should have in place a plan of communicating formal as well as informal policies so that program planning can proceed in an informed and efficient fashion.
2.3 Status of Systems Development
i) Evaluation Framework
The complex nature of the task of collecting information that addresses the needs of each of the three partners (see section 1.5, Evaluation Framework: Purpose and Description) became evident during an initial attempt at developing an evaluation framework.
The evaluation process started in the fall of 1994 with the establishment of an evaluation committee 11. The process was initiated with a discussion of a draft evaluation framework developed by Human Resources Development Canada. This draft framework addressed core evaluation issues12 common to the evaluation of all SIs. The JEWG worked on an initial evaluation framework in October 1994. The initial framework was the basis for the common evaluation framework developed in the summer of 1995 by the consultant under contract to the Ministry of Women's Equality.
This common evaluation framework would provide an overall assessment of the CCSI project. It was designed for both the OSA projects and the Regional Delivery Model Community Demonstration projects. Based on this framework, a range of baseline data were identified and presented for collection.
The use of a common framework for CCSI projects proved problematic. The project stakeholders for the Vancouver OSA project and the Regional Delivery Model Community Demonstration projects stated that the evaluation questions, and in turn the baseline data that were to be collected, were not relevant to their projects. Therefore, their position was that the data collected would provide neither an accurate assessment of their project's accomplishments nor the information they needed to monitor their projects' progress.
Stakeholders of other CCSI projects stated that the evaluation component was introduced prematurely, before the Ministry of Women's Equality had approved their final proposal.
In addition, as noted in section 2.2.1, Information Sharing and Communication, information regarding the federal/provincial contribution agreement was not released until several months after the signing. Therefore, project stakeholders did not fully understand the goals of the Strategic Initiatives Program and how the evaluation was linked to measuring their project's impact against these objectives.
As a consequence of this state of affairs, project stakeholders felt that they had not been adequately consulted with respect to the evaluation component, and interpreted this as reflecting an unequal partnership between themselves and the province. As a result, many of the projects did not initially participate in the evaluation, specifically in the collection of baseline data.
As a consequence, and in recognition of the differing nature and scope of the projects and the particular information needs of both the province and the project stakeholders, the JEWG determined that separate evaluation frameworks were required. These frameworks would capture the differences in activities delivered by each of the projects, and ultimately the differences in the goals project activities were designed to achieve.
In the fall of 1995, in collaboration with the JEWG, the Research, Evaluation, and Intergovernmental Affairs Branch of the Ministry of Women's Equality took the lead role in the evaluation component of the CCSI project13 .
To provide clarity with respect to the evaluation component, all SI funded projects were provided with an information package consisting of the federal/provincial contribution agreement, and a plain language synopsis outlining the intent of the evaluation and the activities comprising the evaluation plan.
In addition, project stakeholders were invited to a one-day working meeting. The purpose of the meeting was twofold. First, it was intended to address any questions stakeholders had regarding the evaluation components. The second purpose was to provide an opportunity for the stakeholders to suggest possible outcome indicators of improved quality, accessibility, affordability, responsiveness, and cost-efficiency that would be pertinent to their project's evaluation.
Following the one-day working meeting, new evaluation frameworks were developed, and meetings with essential representatives of each of the projects were held in order to review and finalise the frameworks. The end products were project-specific evaluation frameworks that would provide:
an assessment of the impact of the CCSI project in relation to the objectives specified in the Canada/British Columbia contribution agreement; and,
specific information needed by the province and project stakeholders in order to make informed program and policy decisions.
Due to the success of this process, all new child care projects to be funded under the CCSI project will receive the evaluation information package, and stakeholder involvement in reviewing and finalising the evaluation framework for each new project will be continued.
ii) Outcome Indicators
Projects funded under the CCSI project are aimed at achieving one or more of the following outcomes: improved accessibility, affordability, quality, responsiveness, inclusiveness and cost-effectiveness of the child care system. The evaluation frameworks developed for the 1995-96 funded projects have incorporated multiple measures of these outcomes for use in the summative evaluation. Moreover, both quantitative and qualitative measures are being utilised14. All data collection procedures adhere to the rules and regulations of the British Columbia Freedom of Information and Privacy Act.
Surveys of parents will be conducted to determine the impact that the child care projects had:
in assisting them to pursue employment-related training and education;
in assisting them to enter and remain in the labour force; and
in reducing their dependency on income assistance and/or social assistance programs.
Furthermore, all new caregivers recruited to provide services for a CCSI project will be asked if becoming a child care provider was a means by which they moved from a state of unemployment to that of labour force attachment. This information will determine the impact improved access to affordable child care has on creating employment in the child care sector.
iii) Project Monitoring Database and Baseline Data Collection
To date, project monitoring databases have been installed for all the 1995-96 funded projects. All projects have been requested to collect the data on an on-going basis and to submit the data to the Research, Evaluation and Intergovernmental Affairs Branch of the Ministry of Women's Equality on a quarterly basis to coincide with the requisite quarterly reports to the federal government. To date, all the OSA Centre projects have been provided with the Carefinder computer software to facilitate this data collection process. Baseline data collection using this database has been completed for all the 1995-96 funded projects.
The project monitoring data provide information regarding:
project outputs (i.e., frequency of service/activity provision);
the utilisation rate of the project services/activities; and
project outcomes (needed for the summative evaluation).
Project outputs and utilisation rates give project managers feedback regarding which aspects of their project need adjustment or re-designing in order to meet their stated objectives of improved accessibility, affordability, quality, responsiveness, inclusiveness, or cost-effectiveness.
iv) Process Evaluation
The process evaluation is scheduled to take place six months following project implementation (i.e., commencement of service delivery), and then annually thereafter15 The process evaluation questions are designed to encourage project managers to monitor and document the challenges, issues and barriers (or successes) encountered in delivering project activities. This, in combination with the project monitoring data, will encourage project managers to closely supervise and regulate their project's outputs.
v) Summative Evaluation
The summative evaluation is scheduled to occur during fiscal year 1998-99. The evaluation will measure the incremental outcomes, impacts and cost-effectiveness of the CCSI. The report will be organised around the outcome themes of improved accessibility, affordability, quality, responsiveness, inclusiveness, and cost-efficiency.
The summative evaluation will address issues on both a community level (i.e., on a project-by-project basis) and a macro level (i.e., aggregation of projects). On a macro level, data from those projects where identified outcome indicators have been collected will be combined. Changes between baseline and post-project implementation will be calculated; this will represent the incremental change to the outcome indicators. Analyses will be conducted to explain the observed changes and discuss the impacts. Additional analysis will be completed to determine if the CCSI was the most cost-effective approach to improve the accessibility, quality, affordability, inclusiveness, and responsiveness of the child care system.
At the community level, the analysis will examine the outcome indicators by project. The nature and scope of the analysis will be determined at a later date. However, an example would be comparative analysis, as in the following scenario:
project A has the specific goal of increasing "in child's home"16 spaces;
projects B and C have the goal of increasing spaces in general, perhaps using different approaches; and
the analyses would be to compare the outcomes, impacts and cost-effectiveness of project A with those of projects B and C; and of project B with those of project C.
Using a similar procedure, judgements of relative success between projects with similar specific aims will be made17 .
|
Early consultation at the community level and the commission of community needs assessments were necessary for more effective project-model design development. This was in addition to effective logistical planning around the implementation and administration of project activities.
The lack of a signed Canada/British Columbia Strategic Initiatives contribution agreement prior to the initiation of project planning at the community level was a factor contributing to many of the challenges faced by both community and Ministry of Women's Equality stakeholders. Ministry staff could not provide the direction, information, and assurances needed by community stakeholders in order to develop and implement their pilot project models in an effective and efficient manner.
Not involving the British Columbia Government Employees Union during the conceptual stages of developing the One-Stop Access Centre model resulted in unexpected employee union-membership issues. Some projects could not resolve these issues and therefore abandoned the concept of co-location of CCSP services with a Ministry of Social Services financial assistance worker and a Ministry of Health licensing officer. The formative evaluation brought to the fore the need for proactive involvement, and a partnership approach to communicating complex information so that all partners would have the same understanding of the issues and polices surrounding union certification.
The issues of the availability of a full-time financial assistance worker to co-locate at One-Stop Access Centres has not yet been resolved. Currently, the Ministry of Women's Equality is funding one .5 FTE position per One-Stop Access Centre.
One-Stop Access Centres failed to anticipate challenges surrounding the acquisition of suitable space. Such challenges included lack of available space, lack of affordable space, and zoning laws that precluded the situation of a community service venture in a residential neighbourhood.
In evaluating qualitatively different projects, separate evaluation frameworks incorporating questions specific to the project activities are required. Although there are many common issues and objectives among the various CCSI projects, it was found that the use of a generic evaluation framework in which some questions were not directly relevant to specific pilot project activities resulted in a lack of "buy into the evaluation" by the communities. Furthermore, perceptions among the community that they were not equal partners with the Ministry of Women's Equality with respect to developing the evaluation component were fuelled by their perceived lack of opportunity for input into the evaluation framework.
Project monitoring systems have been implemented for all 1995/96 funded projects, and will provide data needed by project managers to monitor service/activity delivery, and provide data needed to assess project outcomes.
|
View Indicators, Data Sources, Type and Collection Intervals
|
View B. C. Child Care Renewal Strategic Initiative
View Management and Administration Model: Oakland Pilot Project
View Regional Management Model: Regional Umbrella Group (RUG)
|
|
| Last Modified: 1999-06-22 | Important Notices |
![[To Top]](/web/20060127000211im_/http://www11.hrdc-drhc.gc.ca/edd-img/top.gif){kind=small}