Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

The Canada Education Savings Grant (CESG) Program was introduced in 1998 to encourage Canadians to save for the post-secondary education (PSE) of children. The program provides a grant of 20 percent on the first $2,000 of annual contributions to Registered Education Savings Plans (RESPs) for children up to the age of 17. The CESG is administered by the Learning and Literacy Directorate of Human Resources Development Canada (HRDC).

Over the first four years of the program (1998/99 to 2001/02), close to $1 billion dollars was paid out in grants, with a total of $318 million being paid out in 2000/01.

Evaluation Issues and Approach

This formative evaluation of the CESG was conducted between January and October 2002. Its purpose was to provide reliable information on program relevance, program design and delivery, and to consider the early signs of program impacts.

Evaluating programs, such as the CESG, raises a number of challenges. The CESG is designed to encourage early and sustained participation in RESPs as a way to reduce financial barriers to PSE, increase PSE access and participation, and reduce the financial burden of PSE. Achieving these objectives involves working through groups and individuals beyond the direct influence of the CESG Program. As a result, in individual instances, specific outcomes of the CESG are not easy to track and measure. While these challenges will be more of a consideration for the summative evaluation of the program, they are also relevant to the consideration of the early signs of program impacts.

The methodology used by the formative evaluation recognized these challenges and attempted to address them in a number of ways. The evaluation approach emphasized the use of multiple lines of evidence thereby allowing for findings from one approach to be corroborated by findings from other lines of evidence. Quantitative and qualitative information was gathered from a range of groups (i.e. RESP subscribers, non-subscribers, promoters/trustees, and informants). A review of program documents, administrative data and the literature on PSE financing and access was undertaken to help develop and inform the methodology and lines of enquiry for the various lines of evidence.

In summary, the evaluation included the following key components:

• A survey of 1,998 RESP subscribers;
• A survey of 1,001 non-subscribers;
• A survey of 37 promoters/trustees; (i.e. financial institutions)
• 20 informant interviews (with eight government representatives, six researchers/non-government organizations, and six financial experts);
• A review/analysis of program documents and the literature on PSE financing and access; and
• The use of administrative and other data.

The evaluation approach had certain shortcomings that should be noted. First, inaccuracies in the contact information available on RESP subscribers created difficulties in surveying subscribers. Second the study was not designed to measure program incrementality. Thus, the net impact of CESG on RESP take-up and contributions could not be fully measured. Third, the small sample size of the survey of promoters/trustees and the informant interviews means that the data/information collected from these sources may not be representative.

Evaluation Findings

Program Relevance :

A number of factors act as potential barriers to PSE participation.

The literature review indicated that factors impeding PSE include the costs of PSE, parents having low levels of education, and a child having poor school performance. At the same time, Canadians have a strong interest in PSE and both RESP subscribers and non-subscribers have high expectations for the education of their children. Most subscribers and most non-subscribers expect their beneficiaries/children to attend either university (78 percent of subscribers and 60 percent of non-subscribers) or college (29 percent and 43 percent, respectively).

Less than half of Canadian households with children under the age of 18 had saved for the future education of their children, with some groups putting aside more than others.

Results from the 1999 Survey of Approaches to Education Planning (SAEP) indicated that 17 percent of households, and 45 percent of those with children under 18, had savings for PSE. Further analysis (multivariate) of the SAEP data indicated that the incidence of saving for PSE is particularly high for those with a university degree, those with an annual income of $80,000 or more, and residents of the Atlantic and Prairie Provinces.

There is uncertainty about the cost of post-secondary education and how much savings will be required.

Although some subscribers and non-subscribers have fairly realistic expectations of the cost of PSE, a large proportion of both groups (18 percent of subscribers, 26 percent of non-subscribers) could not or did not respond to the question on what they thought the total annual cost of PSE would be when their child enters PSE. There is also uncertainty about how much of the cost of the child's PSE will be covered by their savings (22 percent of non-subscribers and 13 percent of subscribers could not or did not respond to the survey question in this area).

The CESG is a key program in Canada designed to encourage adults to save for the future PSE of children through a combination of tax-sheltered income-earning savings and grant.

Repayable government and private student loans and study grants and non-repayable government and private scholarships, grants and bursaries are directed at youth, typically in disadvantaged positions and seek to facilitate their participation in PSE at the time of entry or assist debt repayment after completion of PSE.

Characteristics of RESP Subscribers:

The evaluation identified a number of basic characteristics of subscribers.

RESP subscribers have been contributing to RESPs for an average of six years. Almost all have one or two plans and deal with only one promoter/trustee (financial institution). Most have one or two beneficiaries. The average grant allotted in 2000 (the last complete year for which CESG data were available at the time of the evaluation) was $389 per subscriber.

Key factors affecting CESG take-up include parent's education, age and school aspirations for their children, the child's performance in school, and province of residence.

Multivariate analysis identified certain characteristics of subscribers as being influential in predicting RESP take-up, after controlling for the influence of other factors. Among the strongest predictors of whether or not parents will contribute to an RESP on behalf of the child are the child's school performance and their parents' expectations that they will attend university. Other predictive factors include parents' education, their age (over 35 years) and not living in Quebec. The lower RESP take-up rate in Quebec is likely attributable to the province's publicly funded college system (CEGEP) and relatively low university tuition fees for Quebec residents. The effect of income, although significant, is weaker than these other factors.

The foregoing analysis indicates the importance to undertake segmented analysis of the factors affecting take-up for various characteristics of subscribers (e.g., education, age, income) in a summative evaluation. Such an analysis would possibly reveal important factors that predict RESP take-up among different subscriber groups, and by so doing, indicate promotional and marketing methods that would be important in targeting the program.

RESP contributions rise with income and are significantly lower than the population share for parents with low household income and higher for those with high household income.

The share of RESP contributing households during the period of 1998 to 2001, was very low for households with $20,000 or less of pre-tax income (8.6 percent) in comparison to their share of all households (33.6 percent). Households in the $20,000 - $39,999 income category have a share of RESP contribution that is slightly less than their share of households while households in the $40,000 - $79,999 categories have a modestly higher share of RESP contribution than their share of all households. For households above $80,000, their share of RESP contributing households is much higher (36.2 percent) than their share of all households (16.5 percent).

Design and Delivery:

Awareness of the CESG and its rules is low in lower income groups and rises with income level.

Eight-five percent of those making RESP contributions were aware that they were receiving a grant, but only half (48 percent) of non-subscribers had heard about the CESG. Lack of awareness of the CESG was associated with lower income and education levels, as well as living in Quebec.

Government promotion materials are not particularly effective in reaching subscribers and potential subscribers.

Few subscribers (4 percent) and non-subscribers (11 percent) identified government material as their source of awareness about the CESG, although these numbers may not include some of those who heard about the program from government-sponsored advertisements in newspapers or on television.

Less than half of subscribers are satisfied with the ease in finding and understanding information about the CESG, and only 62 percent of promoters/trustees are satisfied with this information.

The majority of RESP subscribers are satisfied with service delivery, although satisfaction was lower for clients of scholarship foundations.

The majority of subscribers are satisfied with all aspects of service delivery. In particular, three-quarters or more are satisfied with the ability of HRDC staff to serve them in the language of their choice and the response time of staff to answer requests by e-mail.

A majority of subscribers are also satisfied with most aspects of program delivery by promoters and trustees. Particular strengths include courtesy and staff knowledge of rules regarding the regularity and amount of the RESP contribution. However, satisfaction with the courtesy and knowledge of staff and satisfaction with service fees was significantly lower for clients of scholarship foundations.

Most promoters/trustees are satisfied with HRDC program delivery, although some areas were identified for improvement.

Most promoters/trustees are satisfied with all aspects of HRDC program delivery to them. Areas where satisfaction is particularly high include the speed with which grants are issued, and the courtesy and language ability of HRDC staff. Response time of staff at the HRDC Call Centre received the lowest rating from Promoters/Trustees.

Promoters/trustees exhibited only a modest level of understanding of reporting requirements. Fewer than half of promoters/trustees surveyed said they understood their reporting requirements under the CESG to a large extent, and several found them difficult to implement. Despite saying that partners' roles and responsibilities were clearly specified and understood, most informants felt that roles and responsibilities needed further clarification. As well, large numbers of promoters/trustees were not satisfied with the information and training provided to them on roles and responsibilities.

A number of factors were identified as affecting program delivery.

Factors identified as affecting program delivery included the potential for the grant to be seen as insignificant, insufficient human resources to manage the program, confusion in the public's mind over the respective roles of HRDC and Canada Customs and Revenue Agency (CCRA), and the fees financial institutions charge subscribers.

Few of those consulted identified the rules associated with RESPs as barriers to program delivery.

Administrative data systems are effective in delivering grants, although they were not well suited to developing the sample frame for the survey of RESP subscribers and to developing a comparison group for evaluation purposes.

The administrative data systems were seen as particularly effective in delivering the grants. Most promoters/trustees viewed the security measures as adequate for encrypting and transferring client data to HRDC.

Inaccuracies in the contact information available for RESP subscribers made it difficult to develop and conduct the survey of subscribers.

Early Signs of Program Impacts:

The available evidence indicates that the number of individuals contributing to RESPs increased significantly when the CESG was introduced.

The linked CESG-LAD database reveals a significant increase in savings for PSE. The proportion of taxpayers with children under 19 years of age and who contributed to a RESP rose from 4.1 percent in 1998 (when the CESG was just getting under way) to 6.2 percent in 1999 (when the CESG was fully operational), and to 7.2 percent in 2000. Other lines of evidence also indicate that the program had an impact on contribution levels. From the survey of CESG subscribers, 72 percent of subscribers indicated that the program had an important effect on their decision to open an RESP account and 23 percent said it was somewhat important. Analysis of the linked CESG-LAD database also revealed that contributions to RRSPs did not decrease between 1998 and 1999.

The foregoing analysis indicates that the introduction of the CESG had an impact on PSE savings. However, the study methodology did not allow for a determination of CESG's net impact on PSE savings; nor was it possible to assess the extent to which savings were transferred from other investment vehicles to RESPs.

There is evidence that the CESG encouraged some subscribers to contribute more to RESPS.

CESG administrative data indicate that average assisted contributions (attracting the grant) rose from $1,640 in 1998 to $2,105 in 1999, although average contributions fell somewhat to $1,945 in 2000.

About 40 percent of all subscribers surveyed reported that the program positively influenced their contributions to RESP. About half of those with RESPs since the program's introduction said they contribute more to RESPs than they would contribute without the grant.

The foregoing provides evidence of the potential that CESG led to an increase in savings for PSE. However, the study methodology does not allow for a determination of the extent to which increases in contribution levels are attributable to the program. Similar to the discussion on the incremental effect of CESG on the number of RESP contributions, the summative evaluation will focus on measuring the net impact of the program on the contribution level.

Limitations:

There is a need for more targeting at lower-income families and for more effective promotional materials to reach targeted audiences.

Awareness of the true cost of PSE, the benefits of attaining higher levels of education and the benefits of saving for the future PSE of children could be increased. The fact that half of non-subscribers said they would contribute to an RESP if they knew they would receive a grant for doing so suggests more effective promotion could increase RESP up-take. Promotion of the program would likely have a limited impact, however, for those families earning less than $20,000 with relatively little discretionary income.

There is a need to do more to clarify reporting requirements and roles and responsibilities of delivery partners.

The evaluation indicates that there is a need to further clarify the roles and responsibilities of delivery partners, particularly those of HRDC and the CCRA. There is also a need for more effort to improve understanding and facilitate implementation of program reporting requirements on the part of promoters and trustees.

Program delivery could be improved by considering ways to improve the CESG Call Centre.

Another way to help improve delivery is to consider ways to improve the response time of the CESG Call Centre.

Consideration should be given to ensure the availability of subscribers' contact information.

Ensuring the availability of accurate subscribers contact information would help to support the evaluation process and the assessment of program impacts in the summative evaluation.

The summative evaluation should include a concerted effort to measure the net impacts of the program on PSE savings and RESP take-up.

The formative evaluation has shown that the introduction of CESG in 1998 was accompanied by increases in RESP holdings and contribution levels. However, the extent to which these changes were due to the introduction of CESG could not be determined, as the study was not designed to measure the net impact of CESG. As the determination of CESG's net impact with respect to RESP holdings and contribution levels are important areas of research for the summative evaluation planned for 2004, exploratory data and modelling work will be undertaken in 2003 to determine the best methods to use.

• An expert panel composed of econometricians, financial experts, HRDC program and research officials as well as representatives from other government departments (e.g., Finance) will review and provide advice on the CESG methodological plan and the findings from the exploratory data and modelling work. The work will be completed in 2003.

The incremental effect of the CESG on the number of RESPs will be addressed in the summative evaluation.

Incrementality in program evaluation terms, attempts to answer the following question: "(1) Did the program intervention make an overall difference with respect to the intended result?; and (2) If yes, what was the extent of the difference?"

With respect to the CESG, the desired incremental program effect is illustrated in Exhibit 7.1, page 46. The difference between the projected trend line and the actual post-program observation of RESP contribution (either in terms of total amount, or number of new accounts) after the introduction of CESG, can be attributed to the program itself. A₁, A₂, A₃ are observations of past RESP contributions, A₄ is the projected contribution of RESP without CESG and B₁ is the actual contribution of RESP after the introduction of CESG. The difference between B₁ and A₄ is the estimated program incremental effect at point t₂.