The Use of Social Indicators as Evaluation Instruments - Final Report

Appendix B

Interview Guide 1
Respondents Previously Involved with the Social Indicators Movement

I. Demise of the Social Indicators Movement

1. Why, in your view, did the social indicators movement die out in the Canadian government policy community? (Discuss generally, then ask for respondent's assessment of the importance of each of the following if not already mentioned).

1. Failure to demonstrate utility in shaping social policy.
2. Economic downturn and pessimism about social conditions.
3. Return to power of conservative governments (Reagan era).
4. Lack of an overarching social theory and lack of consensus among social scientists as to which indicators could be used.
5. Failure to come up with a consensus on shared social values.
6. Too much emphasis on the econometrics and not enough on the practical.
7. Undue complexity of the measures derived.
8. Relative absence of social scientists in influential positions in government (compared to economists).
9. Difficulty in creating a system of measurements with common units of well-being.
10. Difficulty linking objective measures to subjective impacts.
11. Indicators as vindicators: conflicts between the value-neutral (social scientists) and value-driven (policy-makers, particularly those close to prevailing political ideology); framing of questions that lead to indicator construction.
12. Failure to recognize process as product, i.e., to develop institutional arrangements for development of understanding, consensus and commitment to indicators among policy-makers and the public

2. What do feel are the main lessons learned from the previous social indicators movement, particularly from evaluation and objective-achievement points of view?

II. Future Utility of Social Indicators

3. What could be the future utility of social indicators approaches, given:

1. evaluation and monitoring responsibilities in a department like HRDC that administers broad transfer programs like the CHST?
2. the new Parliamentary accountability frameworks?
3. public expectations of accountability? In particular, do you think that the idea of national social reports is becoming attractive again?

4. What do feel is likely to happen in the areas of social standards and social audit: more emphasis in the future or less? Why?

1. How do you think that social indicators fit into the notions of social standards and social audit?

5. In what areas of social programming (particularly employment, welfare, training, health, education), do you think meaningful social indicators could be applied to program evaluation, objective-achievement measurement, and monitoring problems?

1. What would be promising types of evaluation-oriented indicators in each of these areas?
2. How could these indicators be developed (process)?
3. Can social indicators be used to measure impacts on specific equity target groups?

Interview Guide 2
Respondents Currently Involved with Social Indicators

I. General Description of Work Being Carried Out

1. Could you please describe the work that you are doing in social indicators/social measurement/social accounting?

1. What are its objectives?
2. Could you describe it? How will it be used?
3. Is this work based on particular conceptual frameworks, and if so, could you describe them? (Obtain references and documentation wherever possible)
   • Conceptual frameworks to guide the choice, construction and validation of indicators?
   • Conceptual frameworks to guide how the indicators are to be communicated and used?

2. How, if at all, does this work build on lessons learned from the heyday of social indicators approaches? (Discuss generally, then ask for respondent's assessment of the importance of each of the following if not already mentioned)

1. More explicit goal to demonstrate utility in shaping social policy.
2. Allowing for economic downturn and change in political ideology.
3. Tying of indicators to an overarching social theory or conceptual framework.
4. Using indicators in a relation to the creation and maintenance of social standards.
5. Use of social scientists in influential positions (in or outside government).
6. Attention/use of social lobbies and critics.
7. Creating measures will a more practical focus and less econometric focus.
8. Creating simpler, more transparent measures.
9. Creating a system of measurements with common units (e.g., time use).
10. Other means of dealing with the objective-subjective problem.
11. More explicitly dealing with the question of values and coming up with a consensus on social values.
12. Making institutional arrangements for development of understanding, consensus and commitment to indicators among policy-makers and the public.

II. Future Utility of Social Indicators

3. What could be the future utility of social indicators approaches:

1. Given evaluation and monitoring responsibilities in a department like HRDC which administers broad transfer programs like the CHST?
2. The new Parliamentary accountability frameworks?
3. Public expectations of accountability? In particular, do you think that the idea of national social reports is becoming attractive again?

4. What do you feel is likely to happen in the areas of social standards and social audit: more emphasis in the future or less? Why?

1. How do you think that social indicators fit into the notions of social standards and social audit?

5. In what areas of social programming (particularly employment, welfare, training, health, education), do you think meaningful social indicators could be applied to program evaluation, objective-achievement measurement, and monitoring problems?

1. What would be promising types of evaluation-oriented indicators in each of these areas?
2. How could these indicators be developed (process)?
3. Can social indicators be used to measure impacts on specific equity target groups?

Previous

Contents

Next