|
|
     |
||
 |
     |
 |
|
Five approaches are proposed for monitoring31 and evaluation of major social programs administered by HRDC: composite social indicators; a variant of social accounting, namely Input-Output frameworks with social accounting dimensions; microsimulation analysis based on agreed-upon social benchmarks (expectations) from such programs; a small sample panel survey; and case studies. Some might be employed to measure the independent effects of federal contributions — the impacts of CHST and the enriched federal Child Tax Benefit (a component of the federal-provincial National Child Benefit System). Others might be utilised for the measurement of the joint effects of complementary federal and provincial programs (of seniors' safety-net programs like Old Age Security, the Seniors' Benefit, which will come into existence in 2001, the Canada Pension Plan retirement and disability components). But most of the discussion is devoted to the measurement of the effects of CHST. 3.1 Composite and Partial Social Indicators The Genuine Progress Indicator (GPI), the Index of Social Health (ISH) and similar measures32 represent significant attempts to explore the feasibility of developing partial and composite indices of social improvement for Canada and the provinces. So is the Human Development Index (HDI) of the United Nations. The GPI-type measures focusing on CHST objectives might be further developed to track on a continuum the improvements in those aspects of `quality of life' partially attributable to the joint or complementary federal-provincial expenditures of the CHST type. But these would not be CHST or joint CHST and complementary provincial program attribution effects. HDI-type indices could also be adapted as another line of contextual evidence to compare against the performance of the GPI-type measures, and it is relatively simple to construct. Statistics Canada has the databases and experience with this kind of GPI work. And although the GPI was developed as a Canada-wide index, similar provincial indices could also be constructed. Their advantages lie in their ease of construction and ease of understanding. Their limitation is that they could not provide statements of absolute or relative program impacts (attribution) for federal expenditures on particular programs, like CHST. Their contribution would be in tracking the effects of overall systems of programs and tax expenditures, as well as the economy, on society as a whole. Monitoring the Effects of CHST A fully articulated GPI-type index, with many dimensions, would be more suitable to the measurement of all federal and provincial tax and expenditure programs and the overall economy, rather than sectoral ones like CHST-welfare and CHST-post-secondary education (PSE) expenditures or the corresponding provincial social assistance and provincial PSE spending. A fully articulated GPI would not be appropriate for the estimation of the well-being impacts on a province-by-province basis attributable solely to single-purpose federal or provincial expenditures (welfare, post-secondary education, or health)33. It would nevertheless provide contextual evidence of overall socio-economic trends. What would be particularly helpful is a partial GPI, or HDI indicators, to try to track (not estimate) the singular effects of CHST and complementary provincial spending. This option would be to focus on developing indices of variables immediately impacted by the CHST, namely of welfare, education, maybe even health aspects of CHST administered by Health Canada (as well as the effects of economic and social conditions and other government policies). This would be helpful even if it did not produce direct attribution effects. For example a `welfare' subindex might include such variables as trends in the number of households by type (number of adults, children, other dependants); by region (province); by some acceptable measure of poverty (e.g., Statistics Canada's Low Income Cut-Offs ); by their employment experience in the last 12 months and its stability (full time, part time); and by recent dependence on Employment Insurance or provincial social assistance. A PSE subindex might include the number of dependent or independent youth in post-secondary education by age; sex; type of household and level of income; their level of indebtedness; region; type of studies being pursued; level of participation in federal student-loan program; employment history and stability of affected youth (PSE students); and their household characteristics (living on their own or with parents). Another option would be to develop a broader type of more focused GPI measure for the purpose of tracking all CHST and provincial-related expenditures (by HRDC, Health Canada), as well as complementary provincial spending on a province-by-province basis. In the opinion of Health Canada, the key determinants of population health, namely education, employment opportunities and income support (via welfare) lie outside its policy influence. There is likely a close association (correlation) between health status and welfare or trends in standard of living, between trends in employment and income levels and trends in health status. This would be a rationale for close collaboration between HRDC and Health Canada in the development of joint composite indicators. Another justification might be to share the costs among federal departments of such composite-indicator-development work. Monitoring the Effects of Other Poverty-Alleviation Programs These programs would include the enriched federal Child Tax Benefit, Old Age Security, the Seniors' Benefit (which will come in to existence in 2001), Canada Pension Plan, youth employment promotion, all other tax and expenditure policies to support poverty groups (working poor, elderly, children34, Aboriginals) and complementary provincial initiatives. This proposal would be to develop composite-indicator measures based on improvements in the GPI (or similar indicators) to monitor the effects of federal and provincial expenditure and tax programs designed to improve the quality of life of households with poor children, working poor, seniors who are poor, and Aboriginals in each of the provinces. A joint approach to this work with Health Canada might also be contemplated. HRDC should develop provincial composite indicators of the GPI or HDI variety to track provincial socio-economic trends, including the effects of HRDC spending on CHST, similar or complementary provincial programs, and those of other government policies, as contextual evidence for the evaluation and monitoring of CHST. This work should be undertaken perhaps jointly with Health Canada to cover the CHST health component, since some key health determinants lie outside the health area (employment and income adequacy and education, which are of policy interest to HRDC). This is recommended even though these trends would be attributable to a multiplicity of factors — the state of the economy, personal decisions to pursue post-secondary training or to seek employment, etc., other government policies, and not just federal-provincial program interventions of the CHST variety. Partial indices which focused on trends in socio-economic variables directly linked to the characteristics of the beneficiaries affected by CHST social spending — social welfare, post-secondary education and perhaps also health — would also be useful. Such indicator trend analysis would constitute useful contextual information for monitoring and evaluation of a program like CHST and their provincial counterparts and might eventually support correlation-type analysis between such trends and potential explanatory variables. The lead role in the work on broad composite indicators within Strategic Policy, HRDC, resides with Applied Research Branch (ARB). It is reportedly examining the possibility of undertaking further work on GPI or ISH-type indicators with Statistics Canada. Since Applied Research Branch (ARB) is playing the lead role within HRDC in this area, it is essential that Evaluation and Data Development collaborate with ARB in the development of these indices. The composition of these indices would also interest provincial, territorial or municipal governments, and they should be kept informed of these activities. Also, their input in the construction of these indices should be invited. 3.2 Use of Input-Output Framework with a Social Accounting Dimension Statistics Canada has begun developmental work in the area of adding social accounting dimensions to its national and provincial Input-Output (I-O) framework tables. Statistics Canada has already carried out a certain amount of exploratory work on how environmental accounts might be integrated into the I-O models. But it has not done anything of this nature in terms of other nonmarket effects (e.g., crime, culture). It could incorporate details surrounding provision of post-secondary education and health services (some of the latter are already reflected as transactions in the I-O commodity vectors). Welfare would pose certain problems, but good data exist to track how welfare recipients and poverty groups spend their income in the form of the Family Expenditure Survey (FAMEX)35 and the Survey of Consumer Finances (SCF)36, which might be linked to provincial I-O tables. Such an instrument might also help to ascertain the effects of different combinations of CHST and associated provincial spending (welfare, PSE, health). Monitoring the Effects of CHST The use of Input-Output framework tables with social accounting dimensions for the nation and the provinces might be one way to derive national and provincial attribution effects from federal spending on the HRDC components of Canada Health and Social Transfer (CHST), namely welfare and post- secondary education (PSE). Any simulations with the improved I-O framework tables which incorporated some social dimensions, such as the environment, would be an improvement over the results currently generated with purely economic transaction-type I-O models. As well, the impacts of the CHST components administered by HRDC and the health component administered by Health Canada could be estimated jointly for Canada and the provinces for the reasons advanced for joint development work on composite social indicators. But the exercise of modelling the effects of CHST spending with an I-O framework that included social accounting dimensions for the CHST components administered by HRDC, namely of welfare and PSE, would pose significant challenges. One challenge would be to develop provincial I-O tables with social accounting dimensions with base years, not earlier than 1993 (and preferably even more recent years), a task which would have to be directed and carried out by Statistics Canada. A full accounting for all nonmarket societal effects, positive and negative, is likely out of the question because of the size and complexity of the task. What may be available are I-O provincial models that incorporated environmental effects, perhaps some other nonmarket effects for which data are readily available, and more details on the sectors primarily impacted by the CHST spending. The distribution of CHST expenditure (tax points and cash) by the provinces would not be known directly and would have to be inferred in terms of the overall distribution of provincial spending in these domains. This raises a second challenge, which is to obtain sufficient information on the distribution of CHST funding, in terms of the expenditures of various provincial governments, on (a) welfare benefits and the administration of welfare benefits funded through CHST; and (b) the distribution of provincial PSE expenditures also funded through CHST, which match in a reasonable way with the I-O commodity/services classification of the provincial I-O tables. This work would have to be done jointly by EDD and Statistics Canada. Initially, such aggregate anticipated expenditures would likely appear in budget allocation intentions of the various responsible provincial departments (education, social services) and in subsequent retrospective expenditure statements. Any I-O impact analysis based on anticipated outcomes would have to be compared with subsequent I-O impact assessments of their real attribution effects, once the actual distribution of CHST- welfare and CHST-PSE expenditures was known. The major categories of recipients would be welfare recipients and those working for post-secondary institutions, assuming a much smaller amount of such spending went into capital expenditures and commodity spending in terms of direct spending effects37. A related challenge would be to estimate the distribution of expenditures (money going back to the economy through personal expenditures) of those groups which benefit, namely, the allocation of the spending decisions associated with the benefits to welfare recipients, the salaries of welfare administrators and of those teaching at post-secondary education establishments. This would require substantial econometric analysis. The FAMEX and SCF databases, with consumption spending by different income classes by province, might be good sources of data for estimating the consumption functions for the CHST-affected income groups by province38. The Evaluation and Data Development Branch (EDD) of HRDC should collaborate with Statistics Canada to try to develop provincial I-O tables with social accounting dimensions for one year, for the purposes of obtaining the attribution effects from provincial spending by the federal government on CHST component activities administered by HRDC. EDD should invite Statistics Canada to make a proposal for such I-O work. It might be a first step in developing such annual provincial I-O tables on a continuous basis. The proposed I-O tables should include environmental effects, other nonmarket effects for which data is readily available, and expanded and detailed post-secondary education39 and welfare components. Any Statistics Canada proposal should explain (a) how the distribution of consumption-spending decisions by those households deemed to be likely recipients of CHST welfare and post-secondary education spending (from data sets available within Statistics Canada or elsewhere), or through the estimation of corresponding consumption functions, would link up to the I-O tables; and (b) how to assure a realistic match between provincial expenditures (from data sets available within Statistics Canada or elsewhere) on CHST-type activities financed by HRDC, with the I-O system of commodity/industry classifications. This work might include the modelling of the CHST health component of Health Canada, since some key health determinants lie outside the health area (employment, income adequacy and education, which are of interest to HRDC policy-making). This work to develop these attribution effects with provincial I-O tables for CHST spending will have to be a joint collaborative effort of EDD, HRDC and Statistics Canada. This effort might include the modelling of the CHST health component of Health Canada, for the same reasons mentioned in support of including the health dimension in any composite social indicator work. Monitoring the Effects of Other Poverty-Alleviation Programs This discussion has focused on what could be provided to estimate the attribution effects of CHST. If I-O models were successfully adapted to the needs of CHST, they might also provide a similar monitoring role for other social programs. These include the federal component of the federal-provincial enriched National Child Benefit System (or both federal and provincial components), or of joint federal and provincial tax and expenditure program effects, targeting particular groups (seniors, the disabled) by age, income level, regions/provinces. 3.3 Micro-Simulation Modelling This option is micro-simulation modelling built on plausible theoretical bases and founded upon a credible benchmarking of the objectives of major government social programs, like CHST. However, it has major limitations for new programs like CHST. Some consensus of federal and provincial governments for the set of indicators (economic, social) to monitor the outcomes (dependent variables — reduction in poverty of target groups) of the program interventions and their causes, would be useful. This is because these major interventions in the social policy area involve both levels of government. This would occur concurrently with the development of a theoretical `cause and effect' model of the determinants, intervening variables (exogenous, endogenous), outputs, and desired outcomes. Data sources for the CHST or child initiatives might initially include current Statistics Canada surveys (Census, FAMEX, SCF, the National Longitudinal Survey of Children and Youth40). Another source of information for micro-simulation analysis might be information from a small-sample panel survey, or from household or community case studies, which are discussed later. Econometric techniques might be used to specify and estimate various plausible functional forms, with the appropriate lag effects, to derive attribution effects from program spending for the target groups in various provinces, sub-regions, or communities. This is in order to determine the joint and separate effects contributing to certain observable and expected changes in the dependent variables (outputs, or proxies for same).
These would comprise empirically derived sets of structural equation coefficients, for which the determined values would be a relative importance of various potential causes. Trends in these indicators (coefficients) could be derived at the national, provincial, sub-regional or community levels. This would be a very long-term project because of the associated data availability and theoretical specification problems/challenges (described in the limitations of this approach). But the utility of this kind of monitoring instrument would improve over time as more and better data was obtained and with better specification of econometric techniques. Monitoring the Effects of CHST Some potential effects of joint or separate CHST/provincial expenditures on welfare and post-secondary education might include:
It might be more easy to develop the appropriate set of overall objectives regarding the eradication of child poverty. A broad consensus among different political viewpoints might be more easy to achieve in this instance, than for general welfare support, where broad agreement on what are appropriate accountability objectives (even soft ones) of CHST welfare support, for example, might be more difficult to achieve. This kind of work would have to take into account the lag effects between policy expenditure and its desired outcomes. There are lag effects in spending on welfare, post-secondary education or health, and there would be a need to separate out short-term, intermediate and long-term effects from any program spending. For CHST, sources of inspiration for the design of the underlying theoretical model of `cause and effect' might be interpretations of the broad values, principles and objectives of the Social Union which are to guide the joint federal-provincial stewardship of the CHST or the agenda of the Policy Research Committee (interdepartmental)41. Monitoring the Effects of Other Poverty-Alleviation Programs Potential lagged impacts of joint or separate federal and provincial programs aimed at reducing child poverty might be the changes in the proportions of children living below `poverty line' household incomes, or changes in number of daycare-centre spaces (assuming this was one the provincial initiatives to complement the federal enriched Child Tax Credit). This technique might also find applicability in estimating the relative importance of various factors contributing to poverty among other target groups, such as seniors, disabled persons and Aboriginals. Such micro-simulation approaches might also include useful social dimensions implicit in some economic variables, e.g., impact on classes of desired skills sets, by income level, etc. Limitations of Micro-simulation Analysis This approach has very serious drawbacks as a short- or medium-term solution insofar as it constitutes a recent intervention with consequently little by way of experiential data to serve the purposes of estimation of `cause and effect'. Also, there is no theoretical framework to determine and test the relative contribution of different causal factors, including CHST, for, say, standard of living changes among provincial welfare recipients over time 42 . The use of micro-simulation modelling approaches, built on a plausible theoretical basis and founded upon a credible benchmarking of the objectives of a major HRDC social program like CHST, is appealing but would constitute a long-term project. It is impractical in the short and medium term. There are serious data limitation problems because of the recency of the program. It lacks a well articulated theoretical framework to test the contribution of different causes, including CHST spending, similar or complementary provincial spending, accounting for any trends in socio-economic well-being (e.g., reduction in poverty). 3.4 Small-Sample Panel Survey A recurring small-sample survey should be undertaken of a panel of HRDC-CHST funding beneficiaries (provincial welfare recipients and PSE students), and perhaps for the CHST health component, and perhaps jointly with Health Canada and the provinces. This might supply continuous time series data on dependent variables affected by the program (extent to which the standard of living of welfare beneficiaries has improved, changes in the educational level of PSE beneficiaries). In time, it might provide the basis for econometric analysis of causal hypotheses related to the relative importance of CHST and similar provincial spending, relative to other potential causal factors, for determining trends in socio-economic well-being. This kind of longitudinal database might also yield composite and partial social indicators of well-being for population sub-sets. 3.5 Case Studies Recurring household and community case studies might supply evidence of the impacts of major federal and provincial social programs like CHST. These are relatively simple approaches and would rest on a combination of objectively observed household and community level effects (e.g., increasing number of lower income people with more than low-income cut-off revenues), and their subjective impressions of well-being attributable to such social program(s). Selected "most affected" community and household case studies should be carried out to provide contextual information regarding the impacts and effects of provincial welfare assistance and PSE spending, financed in part by CHST (and perhaps for the CHST health component, and perhaps jointly with Health Canada and the provinces). These might also provide composite and partial social indicators of well-being for population sub-sets, but with more detail than would be provided by a small sample survey. 3.6 Macro Models No macro (economy or province-wide) models, incorporating the CHST, exist. This type of development work is considered at this time of very low priority, because such macro effects could be derived through the proposed provincial Input-Output analysis. 3.7 Other Social Statistics The other social statistics methods discussed earlier in Part 2, namely, social statistics/living conditions, level-of-living research and quality-of-life research, would find application as contextual information in the evaluation of major social programs. However, they would not provide statements of combined program effects, nor program attribution effects. But the exploration of such questions as changes in relative poverty and income distribution would utilise information obtained through these other means, especially for comparative inter-temporal, inter-regional group analysis. This kind of contextual information is often used in program evaluation and monitoring. 3.8 Data Requirements The need for data comparable across provinces is a critical issue. Only comparable provincial data can make it possible to assess progress in meeting agreed-upon national program objectives. The identification of both suitable measures and data collection strategies may require federal-provincial partnerships.
| |||||||
![[To Top]](/web/20060127020027im_/http://www11.hrdc-drhc.gc.ca/edd-img/top.gif){kind=small}
![[Previous Page]](/web/20060127020027im_/http://www11.hrdc-drhc.gc.ca/edd-img/back.gif){kind=small}
![[Table of Contents]](/web/20060127020027im_/http://www11.hrdc-drhc.gc.ca/edd-img/content.gif)
![[Next Page]](/web/20060127020027im_/http://www11.hrdc-drhc.gc.ca/edd-img/next.gif){kind=small}