Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

Our analysis of the current use of social indicators approaches and their potential utility to the monitoring and evaluation mandates of HRDC's program evaluation group has concentrated on three broadly drawn options. Although there are many more nuances to be drawn between these three options, our analysis along several dimensions has led us to the following conclusions about each.

Before outlining each of these options, two points must be made. First, whatever option or options the Evaluation and Data Development group of HRDC adopts to evaluate the Canada Health and Social Transfer, we would suggest that EDD prepare detailed work plans which would specify objectives and goals, research issues, approaches and data collection methods. This would be carried with due regard for provincial interests under the new "Social Union".

Second, we should emphasize the need to evaluate the ramifications of the CHST in all areas covered by the transfer, not just the areas of responsibility under HRDC's purview. In other words, we are suggesting that health be included in this exercise in addition to post-secondary education and social welfare. Research has shown that health has a significant influence on educational achievement and living conditions, and thus must be treated along with the latter two components.

The first option, composite social indices, currently being sponsored by the Applied Research Branch, follows most closely along the path of the previous social indicators movement. Such measures would be used to provide social intelligence on changes in various input and output areas such as social expenditures, physical health, and child poverty. The main weakness of social composites relates to a lesson not learned from the previous social indicators movement: there is little or no connection to either an explanatory causal model of the social states in question, or to explicit goals or benchmarks against which progress can be measured. Moreover, and critical to the mandate of HRDC's program evaluation group, this approach does not permit causal attribution of changes in outputs to changes in inputs, from the federal or any other level of government.

The strengths of such an approach lie in its relative simplicity and public acceptability, and in the relatively low investment required to produce indicators from a wide array of existing data. Also, composite indicators are compatible with, indeed perhaps indistinguishable from, current efforts in various NGO and government policy research groups. For these reasons, an approach based on the construction of composite indicators and tracking them over time would be relatively easy to implement and provide contextual evidence on changes in social conditions, though with minimal capability for attribution. However, their utility as evaluation instruments could be significantly enhanced if the Evaluation and Data Development group were to be involved in their selection and construction.

The second option, "social benchmarks plus", involves using social indicators derived from conceptual models in relation to consensually derived social benchmarks or goals. Under this approach, a set of social benchmarks and the respective inputs are first selected on the basis of consultation with all relevant stakeholders, including representatives of the general public and potential clients of the programs in question, as well as policy makers, and analysts. For the indices to be useful for program evaluation and monitoring, the program evaluation perspective must be represented in the process of selecting and constructing indicators. For purposes of evaluating CHST, these indicators would be in the three areas of concern under the CHST, namely education, welfare, and, ideally, health. The selected indicators would then be used to measure conditions in each of the respective areas of concern and to monitor change therein. And, over a longer period, the conceptual model would be validated on the basis of expert opinion and microsimulation or econometric analysis and supported by data gathering through a longitudinal (panel) survey of beneficiaries and through case studies.

A not insignificant disadvantage of this approach is that, compared to the development of composite social indices, it would be relatively more costly in terms of the resources required to develop possible datasets, the models, the benchmarks, and the indicators. Moreover, this approach would be more complex to implement because of the intellectual challenge posed by the nature of the CHST program and the need to incorporate the views of a variety of persons from outside the department and outside government. Finally, it would take time to amass a long enough time series to complete the estimation process, especially if that involved establishing a full causal model.

Still, this approach, potentially, might be more likely than the first option to meet the needs of HRDC's program evaluation group. For one thing, this option at least partially fills the void created by the lack of stated objectives under the CHST. For another, it could ultimately permit attribution of changes in social states to changes in government expenditures and programs and enable empirical assessment of Treasury Board's evaluation questions of relevance, success and cost-effectiveness. However, while this option would be likely to satisfy public needs for accountability and report cards, by addressing directly the attribution problem, its implementation will be limited in the short and medium owing to the constraints outlined above.

The feasibility of the third option, the social accounting approach, for provincial attribution purposes should be shown through a special feasibility or demonstration project carried out jointly by Statistics Canada and the EDD group at HRDC. The main advantage of the approach is that it could draw on the considerable analytical strength and intelligence (data) existing within Statistics Canada and the international community. Work is currently under way at Statistics Canada to build a social dimension into the Input-Output component of the National and provincial accounts, which would go a long way to being able to attribute changes in social conditions to changes in provincial expenditures in the wake of the CHST.

But there is one cautionary note that must be sounded with regard to this approach. Over and above the wide acceptance of national accounting systems within government and the academic community, this approach may have less intuitive appeal for non-economists and may be difficult to present in the form of report cards to the public. Care must be taken, therefore, that this approach be packaged and "sold" to the public in a clearly transparent and user friendly fashion. If this is done and if the feasibility of incorporating social dimensions into the provincial and federal accounts and attribution can be clearly shown through an initial demonstration project, then this approach should be pursued.