Evaluation of the National Child Benefit Initiative: Synthesis Report - February 2005

5. Evaluation Findings

Many of the evaluation findings reported in this synthesis report have, in whole or in part, already been referred to in earlier annual progress reports on the NCB Initiative issued under the authority of Federal/Provincial/Territorial Ministers Responsible for Social Services.

This is the first time, however, that the full breadth of the analysis has been integrated and reported as a comprehensive whole. In addition, this synthesis report also incorporates the findings of further analyses undertaken but not, as yet, reported. The latter includes analyses pertaining to the three objectives of the NCB, with a particular focus on the objective of labour force attachment including the NCB Initiative's impact in overcoming the "Welfare Wall", and, as well, its impact on recipients' labour market behaviour with respect to workers already in the workforce.

The evaluation findings presented in this synthesis report examine the extent to which the NCB Initiative has achieved its stated objectives as well as the cost-effectiveness of the Initiative. Section 5.1 examines the impact of the NCB Initiative on reducing the incidence and depth of child poverty, providing an overview of the methodologies employed and the evaluation findings. Section 5.2 presents evidence of the NCB's impact on promoting attachment to the workforce, both in terms of addressing the "Welfare Wall" and on the labour market behaviour of recipients already in the workforce. Section 5.3 examines the harmonization of federal/provincial/territorial activities as result of the NCB Initiative, identifying key synergistic effects. Finally, Section 5.4 addresses cost-effectiveness issues related to the NCB Initiative.

5.1 Measuring NCB Objectives Achievement:
(a) Reducing the Depth of Child Poverty

The measurement of the impact of the NCB Initiative on reducing the number of families with children living in low-income conditions is a particularly difficult undertaking. The standard evaluation methodology (involving an analysis of the experience of those receiving assistance versus a comparison group of similar individuals who did not) is simply not an option in this case. This is due to the fact that all families with children in the income range qualify for benefits so that it is impossible to isolate a similar group who did not receive the benefit. In addition, a second option, involving a time-series analysis of income and labour market effects (pre- and post-NCB), did not prove feasible at the commencement of the current evaluation process due to difficulties in arranging the linkage of administrative data bases.

In light of these constraints, a range of alternative methodologies was employed to produce an overall assessment of the NCB Initiative.

The evaluation used two main approaches to examine the impact of the NCB on reducing the depth and incidence of child poverty: (a) simulations of the gross impacts of the cash portion of the NCB Initiative¹⁰; (b) a net impact analysis of the entire NCB Initiative comparing the labour market behaviour of NCB Initiative recipients with the experience of a reference group of individuals with similar characteristics but without children.¹¹ Although both types of analysis have differing strengths and weaknesses, when taken together, they produce estimates that provide important insights and an adequate, though approximate, perspective for accountability purposes.

5.1.1 Simulations of NCB Initiative's Impacts

With the simulation approach, the measurement of the income benefits of the NCB Initiative was undertaken by comparing the differences in low-income impact indicators under two different federal/provincial/territorial child benefits structures. Two separate sets of simulations were undertaken, one by the Centre for the Study of Living Standards (CSLS) (1996-99)¹² and other simulations generated by the Social Policy Directorate at HRDC (2000).¹³ Simulations for the year 2000 looked at:

The actual program structure; and
A simulated structure without the NCB Initiative essentially based on the support arrangements in place prior to the introduction of the Initiative.

Key characteristics of the two child benefit structures are identified in Exhibit 7 below.

Exhibit 7
Comparison of Two Federal/Provincial/Territorial Child Benefit Structures in 2000
Structure 1: Without NCB Initiative	Structure 2: With NCB Initiative
Maintain the 1996 Working Income Supplement (WIS) structure	Introduce the NCB Supplement
No adjustments to provincial/territorial income support programs for increases in the NCB Supplement	Introduce adjustments to provincial/territorial income support programs for increases in the NCB Supplement
No provincial/territorial reinvestment programs and additional investments in income benefits directly related to the NCB Initiative	Introduce provincial/territorial reinvestment programs and additional investments in child benefits and earned income supplements*
* In 2000, these reinvestment programs (income benefits) represented approximately $345 million or over 70 percent of provincial/ territorial and First Nations reinvestment and investment strategies.

The application of the above methodology to data from the Statistics Canada Survey of Labour and Income Dynamics for the year 2000 made it possible to assess the impact of the income benefits from the NCB Initiative.

This measurement approach has the advantage of isolating the simulated impact of the income benefits under the Initiative on the outcome indicators selected while keeping other socio-economic variables, such as the levels of unemployment or earnings, unchanged.

On the other hand, this impact measurement framework cannot capture changes in the economic and labour market behaviour of low-income families with children which may have been caused by the NCB Initiative itself. However, it does provide a base which can be built-upon to "factor-in" such additional considerations. Sections 5.1.2 and 5.1.3, below, describe the outcomes of this measurement process.

5.1.2 Simulation Findings: The NCB has had a positive impact on families with children living in low-income

Based on post-tax low-income cut-off (LICO)¹⁴ the CSLS simulations estimated that between 1996 and 1999 the NCB Supplement had resulted in a reduction in the number of families with children living below LICO (i.e., the low-income rate fell 4.6 percent) as well as a reduction in the low-income gap for families with children (i.e., the low-income gap declined by 8.7 percent).¹⁵ The estimated impact of the NCB Supplement on both the low-income rate and low-income gap appear to be somewhat greater for two parent families than for single-parent families.¹⁶

The results of the Social Policy simulations indicate a similar pattern (Exhibit 8). In 2000 the NCB Initiative was assessed as being responsible for preventing an estimated 22,900 families with 55,000 children from being considered as living in low-income. In percentage terms, there was a 5.1 percent reduction in the number of families with children living in low-income conditions.¹⁷

Exhibit 8
Change in Incidence of Low-income Among Families by Family Type due to the NCB: January 2000 to December 2000 (Post-Tax Low-income Cut-Off measure)
	One-Parent Families	Two-Parent Families	All Families
Decline in Number of Children Living in Low-income	16,100	37,200	55,000
Decline in Number of Families with Low-incomes	8,600	14,300	22,900
Percentage Change in Number of Families with Low-incomes	-4.1%	-6.0%	-5.1%
Decline in Incidence of Low-income Among Families with Children*	-1.2	-0.5	-0.6
* Decline in incidence of low-income is expressed in percentage points. Source: Based on Statistics Canada special tabulations from the Survey of Labour Income Dynamics 2000.

The results of the simulations were further examined to identify changes in the depth (severity) of low-income conditions (Exhibit 9 below). Again, the NCB Initiative was found to have a positive impact — reducing the depth of low-income by 9.6 percent for all families. The impact was higher for two parent families where the depth of low-income was reduced by 11.0 percent.¹⁸

Exhibit 9
Changes in the Depth of Low-income Among Families Remaining in Low-income Due to the NCB: January 2000 to December 2000
Post-Tax Low-income Cut-Off measure	One-Parent Families	Two-Parent Families	All Families
Decline in Low-income Gap (in millions of dollars)	$100	$220	$320
Percentage Change in the Low-Income Gap	-7.6%	-11.0%	-9.6%
Source: Based on Statistics Canada special tabulations from the Survey of Labour Income Dynamics 2000.

5.1.3 Additional evidence corroborates the simulation findings

The estimated NCB Initiative impacts derived through the simulation analyses provide an important base-line of the income effects from the NCB Initiative. However, they need to be extended to take account of any NCB induced changes in employment behaviour.

To the extent that the Initiative has been successful in moving families off social assistance (as per Section 5.2.2 below) and into employment, additional employment-generated income gains will accrue. On the other hand, there is some evidence of off-setting reductions in job attachment on the part of parents already in the work-force.

Additional information regarding overall changes in labour market behaviour induced by the Initiative (and accompanying income effects) was produced using comparisons of changes in overall labour market experience between NCB recipients and a matched reference group. Using Statistics Canada's Survey of Labour Income Dynamics, changes in the labour market experience and resulting incomes of low-income families with children in receipt of the CCTB were compared with changes in the experience of similar low-income individuals without children across the period 1996-2001.¹⁹ This "difference-in-difference" methodology was recommended by the Evaluability Assessment Report as the preferred approach to estimating the NCB's net impacts on labour supply. However, a peer review of the particular approach used by the consultant questioned the use of a comparison group of individuals without children. Accordingly, the results from this net impact analysis should be viewed as indicative only. In addition, because the estimates are based on the differences in outcomes between the two groups, they do not isolate the impact of any one feature of the NCB. Rather, they indicate what impact the entire NCB (cash benefits, social assistance recovery and provincial/territorial reinvestments) had on the family's labour supply, income and low-income status. However, the findings indicate that the NCB has had a positive effect on reducing the impact and depth of poverty, both for families on assistance and for employed families.

The comparison based evidence from the SLID (Annex 1, Document 7) corroborates the overall direction of the positive income impacts identified by the HRDC and CSLS simulation analysis (Annex 1, Documents 1 to 5). Consequently the combined evidence supports the conclusion that there has been an overall positive impact as a result of the Initiative in alleviating low-income conditions.

5.1.4 The scale of, and benefits from, provincial/territorial reinvestments etc. have the potential to be very significant, but their impact could not be established

Savings-induced reinvestment and investment activities point to a considerable injection of funds to provide further support to low-income families (both direct financial assistance and in-kind services). As reported, the NCB-induced savings have potentially added a further 20 percent by way of reinvestment to the NCB's impact.

In an attempt to identify further the potential positive impacts flowing from these reinvestments, the current evaluation examined two of the areas identified in Exhibit 5: (a) Supplementary Health Benefits; and (b) Child Benefit/Earned Income Supplements.²⁰

Overall, the evaluation found that a lack of comprehensive data on investment and reinvestment greatly limits any analysis of impacts.²¹ In addition to the data gaps many of these reinvestment programs are difficult to evaluate because of the inability to link program participation to the intended NCB outcomes.²²

In general, statistics produced on provincial/territorial reinvestments following the introduction of NCB would have benefited from the development of consistent baseline data²³ (i.e., prior to the introduction of NCB) — so as to identify the extent to which "new" program activities have actually occurred as a result of the Initiative.

It is also a key finding of the current evaluation that, in the main, NCB-supported reinvestment programs undertaken by provincial/territorial governments do not have adequate data provisions to ensure basic information necessary for performance measurement purposes (e.g., data on take-up rates, participant characteristics, and the like).²⁴ This becomes another area where cost-effectiveness considerations necessitate that sufficient ongoing data provisions are put in place in order to examine, and demonstrate by means of evidence, the extent to which the program is achieving its objectives in an efficient way, given the budgetary resources available.

5.1.5 While there has been progress, the evidence indicates the need for continued attention to child poverty

Over the first four years of implementation of the NCB Initiative (1998-2001) the proportion of children in low-income families declined year-over-year (see Exhibit 10). The declines in both the rate and absolute levels of child poverty can be attributed to a number of factors in addition to the impact of the NCB Initiative including general improvements in the economy, lower unemployment rates, and, possibly, to increased provincial/territorial minimum wage provisions.

Exhibit 10
Children (under 18 years of age) in Families with Low-income (1997-2001)
	1997	1998	1998	2000	2001
Post Tax LICO	16.0%	13.6%	13.5%	12.5%	11.4%
Source: Statistics Canada and Prairie Research Associates, Module 1 Final Report, December 23, 2003, p.17.

However, the data in Exhibit 10 also underscore the continued entrenchment of child poverty. Even with progress to date resulting from initiatives on several fronts, over three quarters of a million of Canada's children continue to experience a low-income existence.

Without the support of the NCB, the situation would be appreciably worse as measured by the LICO described in footnote 12. This supports the continued relevance of the program's rationale, since it would appear that general improvements in the economy are not sufficient, in themselves, to address children in families with low-income.

5.2 Evaluating NCB Objectives Achievement:
(b) Promoting Attachment to the Workforce

5.2.1 Program Design: The NCB's design features have made work more attractive to social assistance recipients

The evaluation undertook a detailed analysis of the NCB Initiative's program design to examine its impacts in relation to the alternative of remaining on social assistance.²⁵ This is an important aspect since it is estimated, that at any given time, about one-third of low-income families with children are social assistance recipients.²⁶ This analysis found that the design of the Initiative has indeed served to increase the attractiveness of work relative to social assistance. The evaluation identified several elements of the NCB Initiative that create financial incentives and in-kind benefits to make work more attractive than social assistance. These include:

Increasing the net incomes of low-income families with working members relative to those on social assistance;
Adjusting the income-support provided for children through social assistance programs in order to lower the level of earnings at which parents with children are better off working; and
Providing in-kind benefits (e.g., supplementary health benefits provided by provincial and territorial governments) to the working poor with children thereby reducing the implicit tax rate on moving from welfare to work.²⁷

It is important to note that the estimated optimal impacts of these design features on improving work incentives are based mainly on situations where there is a dollar-for-dollar offset between the NCB Supplement and social assistance payments for families remaining on welfare (Exhibit 11).

Overall, the potential impact of the Initiative's design on the movement from welfare to work is reflected in its impact on the changes in disposable income while on social assistance versus the alternative of full-time minimum wage employment comparing pre- and post-NCB (1997 and 2001). The overall findings are summarized in Exhibit 11 below, covering four family types.

Exhibit 11
Average Annual Difference in Disposable Income between Full-time Minimum Wage Employment and Social Assistance (1997 and 2001)
Family Type*	1997	2001	Percentage Points Change	NCB Contribution
Single Parent, 1 child, age 4	+3.8%	+12.7%	+ 8.9	55%
Single Parent, 2 children, 10 & 13	-8.5%	+2.0%	+10.5	72%
Single Parent, 3 children, 4,10,13	-13.0%	-0.9%	+12.1	80%
Two Parent, 2 children, 10 & 13	+30.8%	+37.9%	+7.1	27%
* This is the unweighted average for 11 jurisdictions. Results vary by province. + indicates disposable income from minimum wage is higher than social assistance income. - indicates disposable income from minimum wage is lower than social assistance income. Note: Disposable income is defined as the amount of an individual's income left after taxes and fixed costs (such as rent, car payments, etc.) which are available for spending and saving. Source: This information was derived from the Module 3 project entitled The NCB and Incomes from Employment and Social Assistance by Province and Territory, 1995-2001. These were produced in the NCB Progress Report: 2002, p. 55. Although the study covered the period between 1995 and 2001, the results are presented only from 1997 to 2001 in order to more accurately reflect the period immediately prior to the inception of the NCB.

As Exhibit 11 indicates in 1997, under the previous program arrangements, if welfare recipients left social assistance for full-time minimum wage employment, single-parent families with two or three children faced a decline in income of 8.5 percent and 13.0 percent, respectively. At the same time, single parents with one child experienced only a slight income gain and two-earner families had a stronger income increase.

By 2001, with the NCB Initiative in place, this picture had changed significantly. The gap in income levels between social assistance recipients and those working at minimum wage had improved substantially in favour of those in the workforce for most family types. Under the NCB Initiative, annual income from full-time employment at the minimum wage (supplemented by income transfers) improved by an average of $3,200 compared to income from social assistance.²⁸ Only the single parent family with three children experienced a slight loss in disposable income when leaving social assistance for work.

A number of factors contributed to the increase in disposable income when social assistance payments were replaced by full-time minimum wage employment during the post-1997 period. However, an important factor was the retention of the NCB Supplement when working versus the offsetting of this feature while on social assistance (see last column in Exhibit 11). Other factors that contributed to this change relate to a number of important initiatives by provincial and territorial governments. Such measures included the decline in social assistance budgets over this time period and increases in provincial minimum wage provisions. Changes in tax provisions also occurred.

As demonstrated above, the overall design features of the NCB Initiative have created financial incentives and in-kind benefits to move people from welfare to work. However, these features were not universally applied across jurisdictions (See Exhibit 12) as a variety of offset arrangements have evolved under the NCB Initiative.²⁹ The distribution of offset arrangements among provinces and territories is outlined in Exhibit 12. The offset approach which has evolved under arrangement III appears to contain fewer incentives for families to leave social assistance, since the full extent of the NCB Supplement is not deducted from social assistance recipients.

Exhibit 12
Provincial/Territorial Approaches to Adjusting Social Assistance and Child Benefits
I. Social Assistance Offset Approach	II. Integrated Child Benefit Approach With Adjustment	III. Integrated and Non-Integrated Child Benefit Approach Without Adjustment
Prince Edward Island Ontario Yukon Northwest Territories Nunavut Manitoba³⁰ Alberta	Saskatchewan British Columbia	Newfoundland & Labrador³¹ Nova Scotia³²

An important conclusion identified in a study commissioned by the NCB Evaluation Working Group noted the following:

"An increasing number of jurisdictions (Newfoundland, Nova Scotia, New Brunswick and Manitoba)...are either passing the NCB Supplement on to all low-income residents or using some other replacement formula, which blunts the power of the adjustment to serve as a work incentive."³³

Under any of the administrative models used by provinces and territories to recover NCB Supplement, the extent to which the offset is effective in reducing the "Welfare Wall" is diminished when the amount of the offset is reduced. These findings have important implications in terms of the potential or scope to enhance the cost-effectiveness of the NCB Initiative in the future.

5.2.2 The NCB Initiative has demonstrated some success in addressing the problems of the "Welfare Wall"

The technical literature reviewed as part of the current evaluation is replete with instances where governments across advanced industrialized countries have encountered difficulties in overcoming social assistance dependence due to the "Welfare Wall". This situation occurs where movements off social assistance to employment can be accompanied by a loss of welfare income and a loss of services and other in-kind benefits. The NCB Initiative is designed to overcome such problems by means of both financial incentives (to defray such losses) and expanding the range of services available for those with a job attachment who are not receiving social assistance.

Notwithstanding the problems noted in Section 5.2.1 above, where certain offset arrangements with provinces and territories are likely to have reduced the Initiative's incentive effects, the results produced by the NCB's design features are mainly positive. This assessment is based on multiple lines of evidence including: the time series analyses of social assistance caseload data;³⁴ the survival analyses of social assistance caseload data;³⁵ and the cross-sectional regression analyses of NCB Supplement recipients' levels of reported employment.³⁶

Impact on Social Assistance Caseload

A set of three separate case studies undertaken with individual provincial governments demonstrate that the NCB has had significant success in addressing the problems of the "Welfare Wall".

Time series analyses were undertaken of social assistance caseloads in Alberta, Saskatchewan and Newfoundland.³⁷ Controlling for other changes in provincial programs and ongoing changes in the economy, the analyses indicated that between the second quarter of 1999 and the first quarter of 2000 the NCB Initiative has been associated with cumulative caseload reductions of approximately 6 percent in Saskatchewan, 10 percent in Alberta and 3 percent in Newfoundland.³⁸

The above findings are generally consistent with the importance of offset arrangements in contributing to overcoming the "Welfare Wall." However, it is important to emphasize that, in all three cases, positive outcomes are identified. It would also be interesting to examine the extent to which regional differences in unemployment conditions might also affect these outcomes.

Impact on Duration on Social Assistance

A parallel set of three case studies³⁹ were completed of the social assistance caseloads of British Columbia, Manitoba, and Saskatchewan using individual-level monthly caseload data spanning at least a three-year period of time before and after the introduction of the NCB.

Survival regression analyses were carried out on the monthly caseload data. Controlling for individual characteristics such as age, level of education, presence of young children, length of prior time on assistance, other program effects and regional unemployment rates, the analyses indicated that the NCB-induced change in the gap between income from work and welfare resulted in higher rates of leaving social assistance only for single parent families in Manitoba (+9 percent). For both single parent families and couples with children in Saskatchewan and British Columbia, the restructuring of social assistance benefits did not lead to families spending less time on assistance.

While these survival analyses show that the NCB did not lead to higher rates of leaving and re-entering social assistance, the time series analyses indicate that it did reduce the size of the monthly caseload, thus suggesting that the NCB's impact was on reducing the rate of new entries onto assistance.

In summary the time-series analysis and the survival analysis highlight that the NCB Initiative represents a new and innovative approach to assisting low-income families. That initial results are mixed may reflect the fact that the analysis covers only the initial period of implementation of the NCB and may not be capturing its full impacts.

Impacts on the Labour Supply of Families on Social Assistance

The third line of evidence comes from the combined NCB Supplement recipient survey and T1 tax form dataset of 2,446 individuals⁴⁰.

Eight hundred and seventeen of these respondents had some amount of income from social assistance in 2000. The level of net (of social assistance recovery) NCB Supplement benefits received during that year was correlated with the number of hours they worked in 2001, controlling for other key characteristics like level of education, presence of a disability, age, marital status, aboriginal/visible minority status and provincial unemployment rates. The results of the regression analysis showed that receipt of the NCB Supplement did not reduce their level of work effort. For these individuals, the NCB had no negative effect on work effort.

5.2.3 While the NCB has shown generally positive effects in promoting labour market attachment for social assistance recipients, it may lead to reduced levels of employment among parents already in the labour force

In addition to social assistance recipients, a further question to be examined is the effect of the NCB Initiative on the job attachment of those low-income families where one or both parents are already employed. It is estimated that up to two-thirds of the recipients of the NCB Supplement fall into this group at any given time. The "Assessment of the Net Impact Analysis of the NCB Supplement on Labour Force Attachment of Parents" formed an input to the current evaluation. The findings raise the possibility that, for such workers, the additional financial support provided by the NCB Supplement is being accompanied by a move to reduced hours of work (ranging from 8 to 12 percent) — in the form of increased part-time work.⁴¹

Insight into the reasons why some parents may use the NCB income to reduce their labour supply comes from the survey of NCB Supplement recipients and the follow-up focus groups. The survey identified a range of factors that affect parents' decision to work. They included issues such as: general family responsibilities, the need to maintain a balance between work and parenting, and the availability and costs of childcare. Subsequent interviews with focus group participants revealed that some parents not on social assistance made the choice to stay at home or work part-time because of the cost of child care while others did so because they believe that their child needs parenting more than extra family income. Thus, a possible explanation for reduced job attachment by some NCB-eligible working parents is that the additional disposable income made available through the NCB Supplement, enables parents to spend more time with their children.

To the extent that this is occurring as a result of the Initiative's income effects, it is important to emphasize that individuals are making choices with respect to trade-offs and adjustments involved. The trade-offs identified by the clientele survey raise the question of the kinds of public policies needed to promote work and parenting.

5.3 Measuring Objectives Achievement:
(c) Harmonization of federal/provincial/territorial activities

Administratively, the NCB Initiative stands in clear contrast with the earlier program activities that were in place to assist children in low-income families under the preceding arrangements. Under the previous arrangements there was a lack of strategic co-ordination between different governments within the Canadian federation (and in some cases even between departments within individual governments). This lack of co-ordination was reflected not only in the planning and implementation of program activities, but (equally important) in a lack of complementarity in program design provisions.

As a result, program activities frequently functioned independently from one another (or operated at cross-purposes), failing to exploit important synergies to improve overall effectiveness of results — from the viewpoint of governance, taken as a whole.

The essential basis of the harmonization approach within the current Initiative is a concerted effort by both orders of government (within the Social Union Framework Agreement). This concerted effort extends to the establishment of complementary program design features, based on:

a) The differing roles which governments play within the Canadian federation,
b) Comparative expertise among governments,
c) Economies of scale, and
d) Local flexibility to respond to differing regional conditions.

Within the above, the Initiative's key program design features established to produce synergies were:

The offsets between (a) payments made to families under the federal NCB Supplement, and (b) additional financial support provisions and in-kind benefits available under provincial/territorial social assistance.⁴² This had the effect of linking and integrating programming between the two orders of government in an unprecedented way, by means of strengthening incentives towards the achievement of program objectives; and
The reinvestment by provinces and territories in complementary programs and services resulting from saving accruing to them as a result of the above-noted offsets. This program design provision had the result of leveraging the above noted linkage to produce further important synergies, yet at the same time maintaining local flexibility.

5.3.1 Key Evidence of Synergistic Effects

The most important evidence of synergistic effects accompanying the Initiative lies in the progress achieved in addressing issues related to the "Welfare Wall" (Section 5.2.2 above). Paralleling this, the NCB Initiative has also produced progress in reducing overlap and duplication among governments, and in streamlining operations.

In terms of administrative objectives, a number of structural design improvements in the way child benefits are delivered have been introduced as a result of the NCB Initiative. In addition, the initiative has brought about a more integrated program framework between governments. For example, in many jurisdictions federally-delivered child payments under the NCB are now combined with provincial/territorial child benefits into a single integrated system.

Based on feedback from the clientele survey and focus groups undertaken as part of this evaluation, the evidence indicates that NCB child benefits are administered in a relatively simple, non-intrusive, and non-stigmatizing manner. Feedback from families assisted indicated that they had encountered no serious difficulties in obtaining benefit payments (although there was evidence that further improvements could be made in explaining the dollar-for-dollar offset to social assistance recipients).

Further, a survey of program managers requesting their assessment of any difficulties or complexities they encountered in implementing the integrated approach also produced overall positive feedback concerning administrative procedures.

A report by the Organization for Economic Co-operation and Development, The Economic Survey of Canada (2001), refers to the NCB Initiative as a noteworthy achievement in improving efficiency and coordination among federal/provincial/territorial programs. Another report, Provincial and Territorial Reinvestment Initiatives Case Studies,⁴³ indicates that the NCB Initiative is an important development in establishing a cost-effective model of program delivery. It notes that Canada Revenue Agency (CRA) has responded well to the diverse program delivery needs of provincial and territorial child benefits and earned income supplement programs established under the NCB Initiative.

Federal budgetary expenditures through the NCB Supplement, along with reinvestment funds made available through provincial savings in social assistance benefit payments, have made it possible to extend child benefits to all low-income families with children in participating jurisdictions. The NCB Supplement helps families with children to participate in the labour market and increase their disposable income. Unlike social assistance, the NCB Initiative does not require families to divest themselves of other resources before receiving benefits. The evaluation of provincial/territorial reinvestment programs found that reinvestment funds made available through the Initiative are creating new opportunities for provincial/territorial policy development and social programming.

The NCB Initiative has provided the federal, provincial and territorial governments with an opportunity to pursue a coordinated approach to the delivery of child benefits. Joint work by federal, provincial and territorial partners on the design of NCB benefits, and the establishment of more transparent processes related to the development, sharing and reporting of data, have led to improved program design.

5.4 Cost-Effectiveness Issues

A key question posed by the NCB evaluation framework was: "Is the NCB Initiative, defined as the NCB Supplement, the provincial/territorial reinvestments/investments and the replacement of the Social Assistance child benefits by the NCB Supplement (Social Assistance replacement), a cost-effective way to achieve the NCB's stated objectives?"

By way of addressing this question, the following cost-effectiveness framework was developed which identifies the various kinds of cost effectiveness questions which can be posed of the Initiative.

Exhibit 13
A General Cost-Effectiveness Framework for the NCB
		No comparison		Comparison
Partial Equilibrium	Administrative	A	What is the administrative cost of the CCTB and NCB programs?	B	What is the cost saving of delivering cash benefits using the CRA platform relative to jurisdictional administration?
Partial Equilibrium	Program	C	What are the total costs of providing the NCB initiative, including the NCB Supplement as well as provincial/territorial reinvestments and investments?	D	What is the effectiveness of predecessor programs in reducing the depth of child poverty compared to the NCB initiative? What are the potential efficiencies (cost savings) by replacing predecessor programs with the single comprehensive NCB initiative? What are the potential efficiencies (cost savings) resulting from improvements (if any) in targeting child benefit expenditures on poverty groups by replacing the predecessor programs with the NCB initiative?
General Equilibrium	Total	E	What is the impact of the employment-related provisions under the NCB initiative on reducing child poverty and reducing the costs by government on other transfer programs directed to low-income families such as SA and employment insurance? What is the overall impact on the economy of the transfer of income from higher to lower-income tax-filers?

The framework classifies the questions as administrative or program-related or as using a "no comparison" or "comparison" process. Moving from left to right or from top to bottom in the framework increases the difficulty or generality of the question being posed.

5.4.1 Existing data limitations precluded a formal cost-effectiveness assessment of the NCB

The evaluation work determined that there are a number of challenges in undertaking a cost effectiveness assessment of the NCB.

With respect to determining the costs of administering the CCTB and whether the federal tax platform represents a more efficient method of delivering Provincial/Territorial cash benefits (Exhibit 13, Cells A and B), the key challenge uncovered by the evaluation work was the use of a common tax platform and administrative unit to deliver several programs. In the case of CRA, a common tax platform is used to deliver a number of cash transfers and the same administrative unit manages the CCTB and GST rebate. CRA typically does not track detailed, segregated cost data. As a result, it was not possible to determine the administrative costs of delivering just the CCTB. Similarly, the cost-effectiveness assessment undertaken by the evaluation determined that, for the two provincial programs that administered as separate entities (the Alberta Family Employment Tax Credit and the Saskatchewan Employment Supplement programs), it was not possible to derive accurate, meaningful unit costs for their delivery and thereby effect useful cost comparisons.

The issue of establishing the total costs of the NCB component programs (Exhibit 13, Cell C) has been addressed, in part, by the annual reporting by jurisdictions for the NCB Progress Reports. However, there are several limitations of these data. First, for those provincial/territorial programs which were in existence prior to the inception of the NCB, not all jurisdictions have provided historical information on funding levels. Accordingly, it is not clear what level of NCB funding was provided to these programs. Secondly, many of the NCB reinvestment programs receive funding from other F/P/T Initiatives, such as the Early Childhood Development and Early Learning and Child Care and nongovernmental/voluntary agencies. To date, no inventory has been undertaken of all of the funding being channelled to these types of reinvestment programs. Such an inventory would be a useful step in determining whether the NCB has led to reductions in, or increased the level of, overlap and duplication of programs and services directed to low income families with children.

Determining the relative cost-effectiveness of the NCB versus alternative approaches to achieving the first two objectives of the NCB (Exhibit 13, Cell D) would require agreement on the comparison program(s), common outcomes for the NCB Initiative and a testing method that controls for differences in context for the NCB and the comparison programs. The cost effectiveness review concluded that the immediate predecessor of the NCB — the CTB+WIS — was the only appropriate counterfactual program to look at. Its relative effectiveness in addressing the first two objectives of the NCB could be assessed by using longitudinal data sets that spanned the duration of both programs and allowed for the construction of a comparison group of non-participants. Currently, only the Longitudinal Administrative Databank (LAD) and the Survey of Labour and Income Dynamics (SLID) offer any potential to track key outcomes. Each has its respective strengths and weaknesses in terms of such things as data elements and sample size. However, a major challenge with both is the absence of a strictly comparable group of families with children who did not receive either program.

Finally, assessing the tax incidence and net economic impact of the NCB (Exhibit 13, Cell E) would require the use of micro-simulation models. However, the empirical requirements for micro-simulation are formidable and would require, at a minimum, estimates of the following kinds of behavioural relationships:

Estimates of the work response of NCB Supplement clients to the cash benefit, social assistance replacement and various work support measures from the jurisdictions;
Estimates of the work response of higher-income households arising because the existing marginal tax level; and
The net gain/loss to the economy arising from changes in gross domestic output from the changed work response along the income/household distribution.

5.4.2 Indirect Evidence Suggests that the NCB is a Cost-effective approach to achieving its stated objectives

There are a number of indications that the NCB is a cost effective approach to meeting its objectives:

The fact that most jurisdictions have chosen to use the CRA tax platform as the method of delivering their own child benefit and earnings supplement programs indicates that they deem it to be a more efficient method of delivering these programs;
The use of the federal tax platform to deliver provincial and territorial cash benefits has harmonized the federal and provincial components, to the general satisfaction of recipients. Managers described the implementation of this process as successful;
The linking of social assistance and the NCB Supplement through the recovery process has required and promoted advancements with respect to technical information for data sharing and eligibility determination; and
The key design feature of the NCB (recovery of the NCB Supplement from families on assistance and the use of that income to fund reinvestment programs which promote labour force attachment) has led to a reduction in the welfare wall. In turn, there is evidence that it has enabled social assistance recipients to leave welfare.

5.4.3 Summary Cost-Effectiveness Issues

In summary, it is important to underline that very extensive and detailed operational data on costs and performance measures would be required to address the full range of cost-effectiveness concerns. Currently, such data are not sufficiently developed in the case of NCB Initiative, including data related to the operations of the CRA. The evaluation activity has confirmed these outstanding data deficiencies.

The data issue is a pressing one if governments are to successfully meet their commitment to principles such as transparency, accountability, efficiency and effectiveness. In large part, the issue relates to the need for extensive advance planning when programs are in the initial developmental stage. There is also a requirement for governments to commit sufficient resources to develop and maintain the data systems required to address cost-effectiveness.

In making the above points, however, it should also be noted that it would be unfair to single out the NCB Initiative for undue criticism, since the problems identified are widespread across many government programs. Nevertheless, improvements to data collection and availability are required since, without adequate cost data and accompanying results information, there can be no real transparent results-driven accountability.

Notwithstanding the above, the current evaluation findings do provide evidence that there are grounds for optimism concerning the core performance of the NCB Initiative in cost-effectiveness terms. Compared with the previous arrangements, the evidence on current improvements in addressing the "Welfare Wall", reducing the extent and severity of low-income conditions, plus the benefits from harmonization — all attest to significant gains.

The current evaluation, notwithstanding the data limitations encountered, has also enabled the identification of five key program areas which would clearly benefit from scrutiny by policy/program decision-makers in a cost-effectiveness context.

These areas of potential concern are as follows:

There is a need to examine the Initiative's dollar-for-dollar offset arrangements. It may well be that there are sound reasons for less than full dollar-for-dollar adjustments. On the other hand, given the Initiative's current employment objectives, the issue of incentives and possible disincentives regarding job attachment is an important one.
It is essential that data systems and measurement procedures be put in place to analyze and demonstrate the results achieved by provinces and territories from their investments and reinvestments. Very little clientele uptake and results information is currently available. On the face of it, it is clear that provincial and territorial investments/reinvestments measures are addressing important needs. However, it is important to be able to analyze and demonstrate the results achieved. Moreover, the findings from such results-driven performance may in some cases, point to areas where further improvements could occur.
The measurement of savings and the method of reporting on reinvestments need to be examined and improved. There is a requirement to measure and report transparently on levels of savings versus levels of reinvestment, etc.
The investment and reinvestment information reported would benefit from a rigorous, across-the-board set of baseline data for all provinces and territories. Such baseline data are essential to demonstrate and confirm the degree of incremental activity which the NCB Initiative has achieved.
The effects of the NCB Initiative on possible reductions in job attachment merits priority investigation. As noted earlier in the text, the evaluation evidence in this area is very tentative. The importance of these issues suggests that it should be further examined as a high priority.

Finally, it bears repeating that all of the areas identified above should be regarded in terms of the potential for still further improvement with respect to the NCB Initiative, which has already achieved significant positive results. In addition, the five points above demonstrate the importance of evaluation feedback information in improving results performance in the NCB Initiative.

¹⁰ See Annex 1, Documents 4 and 5 for more details.

¹¹ See Annex 1, Document 7 for more details.

¹² See Annex 1, Document 4; Centre for the Study of Living Standards. The impact of the National Child Benefit Supplement on the low-income status of Canadians families with children: The SPSD/M results. Also see Annex 1, Document 5: The National Child Benefit Impact on Income levels of Canadian Families with Children: HRDC Simulation Results (Social Policy Directorate of HRDC).

¹³ See NCB Progress Report: 2002, Section 7.

¹⁴ The Low-Income Cut-Off (LICO) is the income level where a family spends 20 percentage points more than the average family on basic needs, including food, shelter and clothing. LICOs vary by the size of the family and the population of the area of residence. The LICO is not an absolute measure of poverty, rather these statistics are often used to study relative low income in Canada.

¹⁵ See Annex 1, Document 4 for more details.

¹⁶ See Annex 1, Document 4 for more details.

¹⁷ See NCB Progress Report: 2002, p. 50, Table 14.

¹⁸ See NCB Progress Report: 2002, p. 52, Table 16.

¹⁹ See Annex 1, Document 7 for more details.

²⁰ See Annex 1, Documents 15 and 16.

²¹ See Annex 1, Document 13.

²² Module I Technical Report 9 — Cost Effectiveness Framework, pp. 6-9.

²³ Module I Technical Report 9 — Cost Effectiveness Framework April 2003, p.1. notes that data limitations preclude a cost effectiveness analysis.

²⁴ See Annex 1, Document 13. See also Provincial/Territorial Reinvestment Case Studies Module II, Final Report March 2003, pp 41-42.

²⁵ See Annex 1, Document 21 for more details.

²⁶ This is a conservative estimate based on analysis of the SLID (Survey of Labour and Income Dynamics) using $32,000 as the low-income cut-off.

²⁷ See Annex 1, Documents 13 to 17 for more details.

²⁸ See Annex 1, Document 21.

²⁹ In New Brunswick (1998-1999), the province chose not to adjust social assistance payments by the amount of the NCB Supplement and continues with this approach. In 2000-2001 and 2001-2002, New Brunswick did not recover the NCB Supplement increase to families receiving social assistance.

³⁰ Manitoba used this approach for a portion of its social assistance caseload. In July 2000, Manitoba stopped recovering increases in the NCB Supplement. In July 2001, Manitoba stopped recovering the NCB Supplement for children six and under. In January 2003, Manitoba stopped recovering the NCB Supplement for children eleven and under. Effective January 2004, Manitoba stopped the recovery for all children on social assistance.

³¹ In 1998-1999, Newfoundland and Labrador chose not to adjust social assistance payments by the full amount of the NCB Supplement. Newfoundland and Labrador redesigned its income support program in 1999-2000, with the introduction of the Newfoundland and Labrador Child Benefit as the provincial reinvestment initiative. In 2000-2001 and 2001-2002, Newfoundland and Labrador did not recover the NCB Supplement increase to families receiving social assistance.

³² Nova Scotia restructured its social assistance regime in 2001 and paid children's benefits via the Nova Scotia Child Benefit. In July 2002, the NCB Supplement was passed on in full to all clients in Nova Scotia.

³³ For more details refer to Prairie Research Associates, Module 1: Final Report, December 23, 2003, p. iv.

³⁴ See Annex 1, Documents 1 to 3.

³⁵ See Annex 1, Documents 18 to 20.

³⁶ See Annex 1, Document 9.

³⁷ See Annex 1, Documents 1 to 3.

³⁸ The figures cited for the three provinces were a sub-set of those contained in Annex 1, Documents 1 to 3 and have been generated for comparison purposes.

³⁹ See Annex 1, Documents 18 to 20.

⁴⁰ See Annex 1, Document 9.

⁴¹ See Annex 1, Document 9.

⁴² See Annex 1, Document 13.

⁴³ See Annex 1, Document 13 to 17.

Contents