|
|
     |
||
 |
     |
 |
|
This chapter explores project outcomes: the extent to which they achieved their goals, end products developed, application of results beyond the original project, lessons learned, and survival rate of projects after CCIF funding ended. 5.1 Achievement of Project Goals Results of project self-evaluations and of the survey speak to the issue of goal achievement. Some 99% of those returning project self-evaluations said that most of the project objectives and expected outcomes had been met. Table 5.1 shows the extent to which survey respondents believed their project achieved its goals. Two aspects of the table stand out. First, the majority of project representatives felt they totally achieved most of their goals. For every goal, at least 80% said it was totally or partially achieved. Never did a substantial proportion of respondents (maximum of 3%) say that a goal was not at all achieved. The other interesting aspect of the results is that most of the goals listed in the table applied to most of the projects. Only three goals — improve working conditions of child care providers, increase the general availability of care, and increase child care opportunities — were cited as not applicable by a majority of respondents. These results differed in predictable ways by main activity of the project. For example, few research projects aimed to increase availability of child care or enhance professionalism in the field, but virtually all aimed to provide baseline information for the future and increase knowledge of child care issues. On the other hand, most development projects endeavoured to enhance professionalism in the field and to meet the training needs of providers, but few aspired to provide baseline information for the future. Demonstration projects were more likely than other types to aim to increase the availability of care and to increase child care opportunities. According to respondents, several factors prevented projects from totally achieving their goals. Mentioned most frequently were lack of time (6% of project representatives said this), lack of appropriate facilities (5%), and new needs identified through the project (3%). Asked what they considered to have been the most successful aspect of the project, most said that their goals had been successfully achieved. Many said they provided a needed service or product, others proved a need existed, others cited a positive response to their service or product, and others mentioned improvements in networking or infrastructure. Most often cited as the least successful aspect of the project was inability to build on the results, because other on-going funding could not be secured, or because parents resisted paying more when CCIF funding ended, or because the project was too short-lived, or due to poor dissemination of results. This calls into question the longer term impact of many CCIF projects. As far as unexpected outcomes are concerned, most respondents identified positive surprises, many involving positive reactions to the service or product, or network building, raising the profile of the organization or service, or a high degree of demand for the product. Some respondents cited something negative that was unexpected. No single negative factor predominated, although a few respondents complained that parents were not using the services as much as they had indicated they would. 5.2 End Products Table 5.2 lists the end products developed through CCIF projects. Final reports accounted for 63% of the end products, publications for 32%, and audio-video products for only 5% of projects. The plurality (25%) of end products focused on professional development/training of child care staff. About 12% were aimed at parent education, and 12% were research reports or publications. Table 5.2 CCIF End Products
Source: CIS November 1994 (34 missing cases) Usefulness of End Products Interviewees were asked how useful the end products were for policy makers, service providers, and parents. There were two basic responses provided by CCIF staff. One was that no one really knew how useful the end products were since there had been no systematic analysis of the end products. CCIF used to send a questionnaire to projects that had produced something but the practice had stopped. By this school of thought, there was plenty of useful information available to parents, service providers, and policy makers, but how much it was used was unknown. Some CCIF staff were dubious that the information was reaching parents. The same group was doubtful that much of the information, though potentially valuable, was used by policy-makers, especially at the provincial level. The more common point of view on the usefulness of end products was 'It depends on the project.' Some end products were judged to be very useful, principally for service providers. One informant stated that service providers never had information before and the end products gave them the tools to work with (e.g., guides, manuals, training, etc.) which saves them time and money. Some end products, particularly national research studies, were considered to be valuable input into the child care policy-making process. And this group thought that 'Parents are a lot more aware of what's out there' in terms of child care and could make more informed choices. 5.3 Application of Results Beyond the Original Project Three-quarters of the demonstration projects had been visited by people interested in developing a similar program. One project representative asserted that 900 people had visited for this purpose; another claimed 250 people had visited; and six others said that over 100 people did so. The average (mean) number of people visiting was 71, the median number 25. And perhaps the ultimate compliment to the project and measure of usefulness of the project was that 39% of the models of service tested by the demonstration projects were adopted by another organization or individual, according to survey respondents. Most representatives of development projects thought their work would directly benefit child care providers. All groups listed in Table 5.3 were said to have directly or indirectly benefited by the majority of these projects. Table 5.3 Intended Beneficiaries of Development Projects
Source: Survey of Project Sponsors N=109 Over 73% of research project representatives surveyed said their findings had been used. The following types of institutions were said to have used the projects' research findings:
Approximately 78% of respondents of projects that enhanced information services held that their project's products had been used by other organizations or individuals. The number of individuals reported to have used the product or service devised ranged from 10 to 10,000 with an average (mean) of 1,265. Every CCIF official interviewed had examples of results being applied beyond the original project. Examples follow.
Provincial representatives had little knowledge about whether the results of any CCIF project had been used elsewhere; they certainly could provide no information on frequency of use. And they offered no reasons why some results may not have been used. By contrast, all NGO representatives had knowledge of project results — usually their own projects — that had been used by other organizations. Respondents tended to cite well known studies such as the National Child Care Survey, the infant videos, or the Meadow Lake project as ones that have had broad application. Also mentioned by the provinces were the national rural network recently developed in Manitoba, a survey funded by CCIF to find out why employees were not using a work-site child care, a training program by the Alberta Family Home Day Care Association for home day care providers, a national survey of child care staff salaries, and modules from the distance education project in Newfoundland. Other projects listed by NGOs follow:
NGOs also mentioned products they had used that originated from other CCIF projects including the distance education package developed in Newfoundland, products from SpeciaLink including videos, and training material and videos from the project 'Making Friends.' Types of organizations/individuals using the products were community colleges, child care organizations, parents, government, and community health nurses. 5.4 Analysis of Project Results There is a synopsis of projects in a catalogue of resources but no analysis has been done. CCIF funded the Canadian Child Care Federation to produce a book that summarized certain projects. They produced two publications — one short and another with a one-page description of those projects that had findings or results. The publications were disseminated to child care organizations, provincial governments, universities/colleges and larger resource centres and libraries. According to one interviewee 'There was no conscious effort to analyze the results of projects in one area (rural care, for example) to find the overall lessons learned. It would have been a good idea to do it.' Another interviewee added 'There was no analysis of projects as they finished and no system to ensure they were building blocks. There was a lack of planning and assessment resources.' 5.5 Lessons Learned From Projects There was no one theme to draw from responses of CCIF interviewees respecting lessons learned from each area of activity. Indeed, no two people cited the same lesson from any area (except two people said that more research is needed). They each had different ideas derived from their experience with CCIF for defining future programs like CCIF. Accordingly, their responses will be listed (in no particular order). Demonstration Projects Among the lessons learned from demonstration projects:
Another theme emerging from the peer reviewers was the importance of having independent evaluators assess at least some of the larger demonstration projects. One reviewer put it very clearly when she recommended that more careful consideration should be given to evaluating demonstration projects funded by CCIF. She stated that the evaluators should work cooperatively with the people who developed the program, but that the evaluators should be independent of the project. The advantages of having an independent evaluation group are that it increases the likelihood that crucial information will be documented, it enhances the credibility of the report, and it increases the ability with which one can assess whether the project being demonstrated can be implemented in other settings. This viewpoint was shared by another reviewer who qualified her review of a project by pointing out that it was difficult to evaluate the impacts of this project because she had to rely entirely upon information supplied to her by the staff who had implemented the project, and there was no source of information independent of the project. Case study demonstration project representatives listed several very specific lessons learned from their projects. Among them: be aware of provincial licensing requirements and how they might restrict serving the community's needs; people should get ideas from their community about what services are needed and remember that everything, no matter how small, makes a difference; it is necessary to work closely with the parents who are going to use the day care to find out the type of programming they want; it is important to involve the community to see what other resources are available in the community and to work together; be aware of available funding before starting to set up a centre; be aware of bureaucracy on every level and what hoops you have to jump through; always be ready to be willing to change to meet community needs; care passionately about the program, and be prepared to work long, hard hours for little pay; it is important to try to bring new ideas to the program, and to try and be continually innovative; the staff involved in the program should care deeply about children. Development Projects Among the lessons learned from development projects:
Among the specific lessons cited by case study development project representatives: a project of this nature should be volunteer-driven as this will result in a strong commitment to work together; one of the best ways to facilitate home based child care would be to have a parents' association in tandem with the child care providers which could oversee each necessary stage of the provision of care process, as well as function as an intermediary between the family and the child care provider; ensure that others are aware of the project — be an active communicator; be wise not to raise expectations which cannot be met; place focus on those individuals who provide the service — there is value in anchoring a project at the grassroots level; be prepared for hard work; realize that a project of this nature requires a substantial amount of money. Research Projects Among the lessons learned from research projects:
5.6 Survival of Projects After Funding Ended Continued funding is only pertinent for some types of projects. It does not apply to research projects, or most needs assessments (though a small percentage of needs assessments were the basis for demonstration projects funded by CCIF). Development projects such as a training manual or seminar were not meant to be on-going. The issue is important chiefly for demonstration projects. For these projects, CCIF endeavoured to fund projects that were in line with provincial policies. The express intention was to fund only those demonstration projects that could secure funding from other sources once CCIF funding expired. 'The program tried not to create orphans but this was sometimes difficult to do.' Proposals were required to show a continuation plan. Because demonstration projects created the expectation of future funding, some provinces were not keen on them (e.g., Quebec, the territories). Given the policy to fund only those projects that had a commitment of on-going funding, it is not surprising to learn that most demonstration projects did secure funding after CCIF, according to staff. Estimates of the proportion that continued varied by province. In Ontario and B.C., for instance, the large majority of demonstration projects had secured funding from alternate sources. The proportion was lower, around half, for on-reserve projects. According to those who returned a project self-evaluation, 85% of all projects survived the end of funding11. Buttressing this, 85% of demonstration projects surveyed continued delivering services after CCIF funding expired. Thus, the short-term survival rate is impressive. Within one year, however, only 54% were still operating. And by the time the CCIF program ended, only 46% of demonstration projects were still in operation. Generally, these projects were picked up by the provincial government. 'The provinces are the only ones likely to fund projects since there is a lack of interest and/or money on the part of other organizations.' Occasionally a municipality continued the funding. Several projects involving service delivery were able to sustain themselves through fund raising, parent fees, provincial contributions and contributions from major corporations. Also NGOs may pick up parts of a project. In rare instances projects were able to remain completely intact through fund raising and parent fees. Except for the Well Beings and 'La garde au domicile du parent' projects12, all case study projects attempted to secure funding from alternate sources as CCIF funding was coming to an end. For the most part, the sponsoring organizations have approached provincial and federal departments for funding. The only non-government sources mentioned were small private foundations, the United Way, a union, and corporations. Two projects, Maggie's and Child Care Connection-Nova Scotia, got small corporate donations. The aboriginal case study projects have tended to rely on the federal government because most of the provinces will not fully fund on-reserve projects. As of late June, 1995, the bands were still attempting to obtain funding to continue their projects. Also, the Assembly of First Nations is lobbying for more money for aboriginal child care programs. Among the federal departments/ initiatives approached by aboriginal case study projects: the Department of Indian Affairs, Canada Employment Projects, the First Nations Child Care Initiative Fund and Aboriginal Head Start, Brighter Futures, and Child Care Visions. The bands did explore the possibility of funding from the provincial governments, but have gotten nothing or too little to run the day care centre. Two bands obtained gaming licences and run bingo games and lotteries to help fund the centres. None of the aboriginal projects has secured stable funding as yet. But the new federal programs give them cause for optimism. Some non-native case study projects have fared better. Some have obtained enough provincial funding to operate the child care centre (e.g., Lakeview). Club 6-12 depends mainly on fees, with some provincial support. Others have secured partial provincial funding for their development projects (e.g., CCC-NS). One got bridging money from the federal program Brighter Futures, and one from Child Care Visions. CCC-NS earns some money (16% of its original operating budget) from the organization's own initiatives (e.g. corporate sponsorships, workshop revenues, consultations, publications, and 'piggy-backing' onto activities conducted by other organizations). SpeciaLink managed to get funding from various sources: Health Promotions, Children's Mental Health, Disabled Person's Secretariat, IWK - Children's Hospital Foundation, Enterprise Cape Breton Corporation (part of Atlantic Canada Opportunities Agency), Canadian Union of Postal Workers, and small private foundations. One project, the Westcoast Child Care Resource Centre, was set up in anticipation of alternate funding. In the first year, the project received a substantial portion of the total funding contribution; in the second year, it received somewhat less; and, in the final year, still less than the previous year. The rationale for this was based on the idea that the centre would acquire alternate funding during the course of its development - and it did. In the first year, municipal and provincial governments provided some funding. In the second year, additional funds were obtained from casino proceeds. In addition, the volunteer board initiated several of its own fund raising efforts. Funding from CCIF was used to support staff positions and some operating expenses, while monies received from the additional funding sources were used for the development of all of the organization's services (including the library). 5.7 Conclusion In general, the evidence seems to suggest that most CCIF projects accomplished the goals they set for themselves. Moreover, many — though it is not possible to specify the proportion — produced potentially useful end products. The key question is how many end products are actually used by others? Survey results suggest that most products/services have been used by others beyond the original project. Some interviewees, however, expressed some doubt that most end products have been useful to others. But other informants felt that many products have been useful, though they tended to point to a fairly small group of particularly successful projects (usually those selected for case study). One factor that has limited the usefulness of CCIF projects is that there has never been an overall analysis of CCIF projects. Such an analysis should be done to systematically derive lessons from the body of CCIF projects, and to identify areas that received too much or too little attention.
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![[To Top]](/web/20060127011359im_/http://www11.hrdc-drhc.gc.ca/edd-img/top.gif){kind=small}
![[Previous Page]](/web/20060127011359im_/http://www11.hrdc-drhc.gc.ca/edd-img/back.gif){kind=small}
![[Table of Contents]](/web/20060127011359im_/http://www11.hrdc-drhc.gc.ca/edd-img/content.gif)
![[Next Page]](/web/20060127011359im_/http://www11.hrdc-drhc.gc.ca/edd-img/next.gif){kind=small}