Formative Evaluation of the Strategic Initiative Western
|
Introduction
This executive summary summarises the findings from the formative evaluation study of the Western Aboriginal Development Alliance (WADA) Strategic Initiative. The evaluation was managed by representatives from Human Resources Development Canada, with input from the Western Aboriginal staff.
The formative evaluation is the second component in a three-staged evaluation process. The first stage was the development of a detailed evaluation framework, and the final component will be a summative evaluation in the fall of 1998. The intent of the formative evaluation is to collect baseline data to be used in the summative analysis as well as to identify the types of models being used and some of the issues which have arisen to date, in order to make midterm modifications to the program.
Project Description
The Stronger Together Strategic Initiative was developed with full co-operation of First Nations and Metis communities, Human Resources Development Canada (HRDC) and the Western Aboriginal Development Alliance (WADA). The mandate of the program is to create employment for Aboriginal and First Nation people and decrease reliance on social assistance. The program aims to form a partnership between Aboriginal and First Nation people, businesses and the community, thereby opening the door to long-term employment opportunities for Aboriginal and First Nations people. The financial support which is available to program participants is intended to provide support services such as training, career and family counselling, child care and, in some cases, supplementary wages.
One of the most unique attributes of the strategic initiative is the use of consumer leverage as a form of moral suasion to encourage employers to provide employment opportunities to Aboriginal and First Nation people. In cases where Aboriginal and First Nation people purchase significant goods from a business, the hope is that the employer will consider Aboriginal people for available jobs. If this initiative is successful in the selected Alberta communities, this framework may serve as a guide to other Canadian provinces.
Evaluation Approach
The formative evaluation of the WADA Strategic Initiative utilises a triangulation approach to collecting the required data. That is, multiple data sources were accessed, more than one evaluator was involved in the study, and the results were analysed in more than one way (qualitative and quantitative approach). A wide range of data collection approaches were utilised in the study, including interviews with key administrators and project stakeholders and meetings with representatives from the communities included in the evaluation. Other tasks included the collection of baseline data from 35 program participants, interviews with ten employers; development of participant profiles and an attempt to collect comparison group data. A literature review of similar programs and the findings from their evaluations was also completed.
Key Findings
Program Delivery
The four communities included in the formative evaluation had all utilised slightly different models to carry out the Strategic Initiative, depending on their history, culture, local infrastructure and administration. There appeared to be flexibility in the way the program was introduced by WADA to each community, and the local administrators appreciated this flexibility and adapted the Strategic Initiative to the needs of their area. The overriding philosophy of the communities was the need to create long term employment, however each had slightly different approaches to doing this (e.g. the development of on-reserve businesses and the belief that wage subsidies are ineffective). In only a very few instances was the consumer leverage model considered appropriate and therefore utilised to generate employment opportunities. Participants and employers were identified in different ways, with some communities seeking out appropriate candidates and employment opportunities, while others relied on the participant themselves to identify jobs and then seek financial assistance. There were substantial differences in the infrastructure of the communities, both in terms of available staff, existing programs and services. The commitment of the program administrator appeared to play a significant role in the success of the Strategic Initiative.Baseline Questionnaire Findings
Participants in the Strategic Initiative were found to have fewer barriers, with a reasonable level of education and some work experience. While they did encounter barriers to employment, most notably child care issues and lack of transportation, they did not appear to have substantial barriers which would preclude their participation in the workforce. Lack of skills training, training on the job and work experience were considered to be the primary job skills they needed to obtain employment. An interesting finding was that the majority of respondents considered working with other Aboriginals, working in an Aboriginal firm, working close to home and working in the community were important factors to them. Almost all participants interviewed were currently working an average of 40 hours per week and earning an average of $8.33 per hour. More than three quarters of respondents were working in the processing, manufacturing and utilities industries, or sales and service or trades/transport/equipment areas. Respondents had a healthy perception of life, and felt that in the future it was likely they would be working in a company/industry they enjoy and applying new skills. They felt less certain that their employment would be a career job; however, the interpretation of a 'career job' was left up to the respondent. Almost none of the respondents felt they would be unemployed in the future.
Employer findings
The majority of the employers interviewed had never accessed government-funded employment programs, mainly because they had never heard of any that were applicable to their business. Employers were keen to access the wage subsidy and felt that they were particularly helpful in the training of new employees. Four out of ten employers had heard of WADA, however three of the four understood it to be a wage subsidy program, and only one mentioned the concept of consumer leverage.
Comparison Group Findings
The WADA Stronger Together Strategic Initiative had a comparison component as part of its overall national framework research design. After careful review of the comparison group approach, and following discussions with HRDC and the participating communities, the research design and tools proposed for the comparison study were not deemed suitable nor acceptable for use with the participants in the WADA Strategic Initiative.
Conclusions
Based on the qualitative and quantitative data collected in the formative evaluation, the following conclusions were drawn from the Western Aboriginal Development Alliance Strategic Initiative.
Recommendations
The report makes several recommendations which require consideration by the program manager. Key among these are:
Responses to Recommendations
The following actions are proposed as a set of mid-term corrections in the WADA ASI project.
First, is the enhancement of project staff skills and knowledge in the area of human resource planning. WADA is currently developing a plan that will see the project staff fully competent in this area by the end of February/98. Additional funds will be made available to the project to defray these costs. In addition, WADA is planning the use of a workshop (perhaps more than one depending on need) to transfer HRP skills/knowledge to community administrators.
Second, WADA staff are in the process of developing literature and fact sheets that can be provided to participating communities. The fact sheets, which will be available by the end of December, will cover a range of topics including expectations, reporting, administrative procedure, approval of expenditure categories, etc.
Third, the criteria outlined in the report were in fact those used to select the communities. Any additional selections (to replace those that are "dormant" by virtue of low activity), will also include the need to demonstrate commitment to organising resources around the individuals (program participants to get appropriate access to community transportation, child care, and ongoing skills development), as well as have one point of administrative contact for employers participating in joint human resource planning within the project.
Fourth, the addition of a second cohort of participants through additional funding from HRDC and INAC will allow for the implementation of the mid-term corrections noted above. In addition, the inclusion of a modest initiative in urban Edmonton may test the feasibility of such an approach in a larger centre where a significant number of head offices are located.
As has been noted, the initiative is producing long term employment, and the community-based model is working effectively given the commitment required at the political level. The summative will provide a picture of the program impacts in terms of the first and second participant cohorts.
The Stronger Together Strategic Initiative was developed with full co-operation of First Nations and Metis communities, the federal government Human Resource Development Canada (HRDC) and Western Aboriginal Development Alliance (WADA). The mandate of the program is to create employment for Aboriginal and First Nation people and decrease reliance on social assistance. The aim of the program is to form a partnership between Aboriginal and First Nation people, businesses and the community, thereby opening the door to long term employment opportunities for Aboriginal and First Nation people. The financial support which is available to program participants is to provide support services such as training, career and family counselling, child care and, in some cases, supplementary wages. One of the most unique attributes of the strategic initiative is the use of consumer leverage as a form of moral suasion to encourage employers to provide employment opportunities to Aboriginal and First Nation people.
This formative evaluation is the second component in a three staged evaluation process. The first stage was the development of a detailed evaluation framework, and the final component will be a summative evaluation in the fall of 1998. The intent of the formative evaluation is to collect baseline data to be used in the summative analysis as well as to identify the types of models being used and some of the issues which have arisen to date, in order to make midterm modifications to the program.
The Stronger Together Strategic Initiative has been developed as a new approach to increase labour market success by Aboriginal and First Nation people. The Strategic Initiative was developed with full co-operation of First Nations and Metis communities, the federal government (HRDC) and Western Aboriginal Development Alliance (WADA). The mandate of the program is to create employment for Aboriginal and First Nation people and decrease reliance on social assistance. The program aims to form a partnership between Aboriginal and First Nation people, businesses and the community, thereby opening the door to long term employment opportunities for Aboriginal and First Nation people. The financial support which is available to program participants is to provide support services such as training, career and family counselling, child care and, in some cases, supplementary wages.
One of the most unique attributes of the Strategic Initiative is the use of consumer leverage as a form of moral suasion to encourage employers to provide employment opportunities to Aboriginal and First Nation people. The potential benefits of such a model are unknown, but may be substantial in terms of creating employment opportunities. The WADA states that, "since Aboriginal and First Nation people are major consumers of goods and services but are not equal beneficiaries of the economic and employment opportunities generated by their consumption, positive changes in employers attitudes about Aboriginal and First Nation employment can be achieved." In cases where Aboriginal and First Nation people purchase significant goods from a business, the hope is that the employer will consider these individuals for available jobs. If this initiative is successful in the selected Alberta communities, this framework may serve as a guide to other Canadian provinces.
2.1 Expected Outcomes of the Strategic Initiative
2.1.1 Overall
The purpose of the Stronger Together Initiative, as set out in the initial documentation, is to encourage long-term employment for Aboriginal and First Nation people and thereby minimise the reliance on government transfer payments, primarily social assistance, to increase self-esteem of participants through the securing of employment, and to help improve the attitude of employers towards Aboriginal and First Nation people.
The goal of this program comes, in part, from the overall goals of the WADA which are as follows:
The major objective of the initiative is to develop a consumer leverage model that encourages the creation of employment for Aboriginal and First Nation people, by demonstrating the mutual benefits to business and Aboriginal and First Nation people, resulting from a co-operation of the two groups.
More specifically, the objectives of the Stronger Together Initiative are:
2.1.2 Additional Expected Outcomes
Through discussions with key stakeholders of the program, additional or clarified expected outcomes were identified as follows:
2.1.2.1 Human Resource Development Staff
A number of outcomes were identified in discussion with HRDC staff:
2.1.2.2 WADA Staff
In discussions with WADA staff, the following expected outcomes were identified:
The formative evaluation of the Western Aboriginal Development Alliance (WADA) Strategic Initiative utilises a triangulation approach to collecting the required data. That is, multiple data sources were accessed, more than one evaluator was involved in the study, and the results were analysed in more than one way (qualitative and quantitative approach).
The data sources which were accessed or collected during the course of the study were as follows:
3.1 Primary Data Sources
3.1.1 Meeting With HRDC Staff
A meeting was held with program staff from HRDC to review the expectations of the formative evaluation and to obtain additional insight on the program, its delivery and anticipated outcomes arising from the initiative. The use of consumer leverage was discussed and the necessity of using this model in the Strategic Initiative was reinforced. A copy of the HRDC staff questionnaire is in Appendix A of the Technical Report.
3.1.2 Meeting With WADA staff
A meeting was held with WADA program staff to review the delivery of the program, issues encountered to date and expected outcomes. Additional insight on the communities involved in the evaluation was collected at this time. A copy of the WADA staff questionnaire is in Appendix B of the Technical Report.
3.1.3 Joint Community Meeting
A meeting was held with representatives from the four communities involved in the evaluation to inform them of the evaluation, the steps which would take place to collect the necessary data, and their expected role in the process. Feedback was solicited from the group on their understanding and agreement to conduct the various aspects of data collection. One of the results of this meeting was the decision not to conduct focus groups in some of the communities.
3.1.4 Community Administrator Meetings
A meeting was held with administrative staff in each of the four communities to review various aspects of the program, including program records, methods used to identify participants and employers, and other areas of interest. Preliminary schedules for interviewing were established at this time. Employer contact names were collected at a later date. A copy of the questionnaire used in the community administrator meetings is in Appendix C of the Technical Report.
3.1.5 Program Participant Interviews (Baseline Questionnaire)
In-person interviews were conducted with program participants who were available during the course of the evaluation. A total of 35 interviews were conducted by a trained interviewer who was familiar with the sensitivities of the project. A senior member of the evaluation team arranged and attended the first few interviews in each community to ensure that all parties were comfortable with the process. The following interviews were conducted in each of the four communities:
A copy of the baseline participant questionnaire is found in Appendix D and the detailed findings from the participant survey are found in Appendix E of the Technical Report.
3.1.6 Comparison Group Focus Group
A focus group was held in Alexander to collect information from community members with similar characteristics to program participants. The questions asked in the focus group and the results from the session are provided in Appendix F of the Technical Report.
3.1.7 Employer Interviews
Telephone interviews were conducted with ten employers from various communities and in a range of businesses (lumber, convenience store, school, paint store, grocery store, hair stylist, industrial construction). The following is a breakdown of employer interviews by community:
The questions asked in the employer interviews and the findings are provided in Appendix G of the Technical Report.
3.1.8 Participant Profiles
Short write-ups were developed on all of the participants surveyed to provide a better understanding of the individual involved in WADA, including some history prior to starting the program, the assistance they received and their current employment status. This information is provided in Appendix H of the Technical Report.
3.2 Secondary Data Sources
3.2.1 Administrative Data and Operational Files
Administrative data and files were reviewed during the course of the community administrator meetings. This information was useful in the assessment of reporting practices and procedures.
3.2.2 Literature Review of Native Employment Programs
A literature review was conducted of Native Employment programs offered through the provincial and federal governments, to provide depth on the types of programs which had been tried in the past, as well as the findings and recommendations on areas which would be beneficial in the development of new programs. The literature review is provided in Appendix I of the Technical Report.
3.2.3 Additional Data
A review was made of the information available to complete a comparison group analysis. Information was collected through Aboriginal Affairs (provincial), Indian and Northern Affairs Canada, Statistics Canada and through the community.
3.3 Study Limitations
At the outset of this report, there are some study limitations which are worthy of note:
4.1 Introduction
The intent of this literature review is to provide some background on the employment needs of Aboriginal and First Nation people, and the approaches which have been tried to date. Some conclusions are drawn at the end of the section on which approaches appear to have the most success, and some concepts which may be useful to incorporate into the WADA Strategic Initiative.
4.2 Limitations
A number of limitations have been encountered in the preparation of this literature review, as follows:
4.3 The Issue
Judging from a variety of socio-economic indicators, the employment and training needs of Aboriginal and First Nation people are considerable when compared to non-Aboriginal and First Nation people. An analysis of 1996 Census data estimated that the unemployment rate for Aboriginal and First Nation people is double that of the national average, and in some areas of the county the rate is five to six times higher than that recorded for non-Aboriginal and First Nation people.1 In addition, the Aboriginal and First Nation population is younger, and while the Canadian population is moving towards retirement age in the next two decades, the Aboriginal and First Nation population will experience growth in the number of individuals who reach working age.2
Education will also play an important role in the participation of Aboriginal and First Nation people in the Canadian labour market. The current rate of Aboriginal and First Nation illiteracy is over twice the national average and the grade 12 completion rate for Aboriginal and First Nation people is half the rate of non-Aboriginal and First Nation people.3
The consequences of high unemployment and low educational attainment are profound. Low income levels (one half to one third of non-aboriginal incomes) combined with other factors (e.g. housing conditions, community infrastructure), create a situation of serious economic inequality. The social costs of high unemployment, dependency on social assistance and low educational attainment are high.4
Suicide rates among Aboriginal and First Nation people are three to four times the national average, and a similar trend is evident for rates of incarceration of Aboriginal and First Nation people in provincial and federal penitentiaries.5 Personal, community and systemic barriers contribute to a lack of self-esteem and the inability to break out of cycles of government assistance and unemployment.
4.4 Strategies
The lack of meaningful activity in the lives of Aboriginal and First Nation people and the barriers to achieving their potential has been typically addressed through individual and community-based resource development. By providing access to training and employment opportunities, it has been felt that Aboriginal and First Nation people and communities will become more responsible and accountable for their lives.6
Historically, employment programs were offered in Aboriginal and First Nation communities in a top-down fashion. This approach was not considered to be optimal, and in the past five years, the delivery and control of training and employment programs has been placed more in the hands of the Aboriginal and First Nation communities and organisations. This approach to Aboriginal and First Nation labour force development is consistent with other community development initiatives which have been successful in Aboriginal and First Nation communities, including Aboriginal-owned enterprises, educational programs, and training and educational institutions. This increasing development and control of community-based training, employment and educational programs supports the progressive goal of Aboriginal and First Nation self-government and self-determination.
The following section provides a brief overview of the types and scope of services and programs which have been implemented by the Federal and Provincial governments to assist Aboriginal and First Nation people in moving into the workforce. A brief overview of each approach is provided, and in some cases a discussion of the limitations of these programs in meeting the needs of Aboriginal and First Nation people.
4.4.1 Federal Government Programs
4.4.2 Provincial Government Aboriginal Programs
4.5 Conclusions
The significant resources which have been devoted to Aboriginal and First Nation employment and training have not necessarily met the human resource development plans and labour market objectives of Aboriginal and First Nation people.7 The following outlines some of the findings from the various Aboriginal initiatives which have been tried, and links these findings to the WADA Strategic Initiative.
The WADA Strategic Initiative does utilise a community-based approach to service delivery, however input from the communities on the design and limitations of the program would have been useful. For example, in isolated communities such as Goodfish it was not likely that the consumer leverage model, in its pure sense, would be appropriate.
The WADA Strategic Initiative does have a strong human resource planning component; however, this element of the program may not have been explained or encouraged sufficiently to the participating communities.
The WADA Strategic Initiative rely on existing individuals (staff members) in the community to deliver the program, and for the most part formal procedures are considered prohibitive.
The WADA Strategic Initiative has been used to fund existing and start-up on-reserve businesses, which, while creating employment opportunities, may not be within the guidelines of the program.
The WADA Strategic Initiative reporting requirements were considered to be onerous by the administrators in the participating communities.
5.1 Models/Approaches
The four communities included in the formative evaluation have all utilised slightly different models to carry out the Strategic Initiative, depending on their history, culture, local infrastructure and administration. There appeared to be flexibility in the way the program was introduced by WADA to each community, and the local administrators appreciated this flexibility and adapted the Strategic Initiative to the needs of their area. The overriding philosophy of the communities was the need to create long term employment, however each had slightly different approaches to doing this (e.g. the development of on-reserve businesses in Goodfish, the disbelief in the effectiveness of wage subsidy in Marlboro). In only a few instances was the consumer leverage model considered appropriate to generate employment opportunities.
Participants and employers were identified in different ways, with some communities seeking out appropriate candidates and employment opportunities, while others relied on the participant themselves to identify jobs and then seek financial assistance. There were substantial differences in the infrastructure of the community, both in terms of available staff, existing programs and services. The commitment of the program administrator appears to play a significant role in the success of the Strategic Initiative.
5.2 Participant Findings
5.2.1 Demographics
Sex
Overall, 69% of the respondents were male, 31% female.
Age
The average age of all respondents was 33.4 years.
Marital Status
Overall, 63% of the respondents were married or living common-law, 23% were single (never married) and 11% were separated.
Education
Overall, 31% of respondents had finished high school, 20% had attended some community college, and 11% had attended some trade school.
Dependents
Overall, one third of respondents (34%) reported having three dependents, with 20% reporting having two, and 17% reported having four dependents.
Other Members of the Household
When asked about other members of their household, 60% of respondents indicated there was another member who was employed and 11% indicated other members were receiving social assistance.
5.2.2 Employment Related Factors
Current Employment Barriers
Overall, 23% of respondents indicated their primary barrier to employment was a lack of transportation, 17% indicated an inability to arrange child care; 9% indicated other factors such as lack of work experience, training; 6% indicated a health/disability problem and 3% each cited personal problems and "people say I'm too young".
Important Job Factors
Overall, the most important job factors were considered to be working with other Aboriginals (77%), working close to home (74%) and working in the community (71%).
Job Skills Needed
Overall, the primary job skills respondents felt they needed was a occupational skills training (37%), training on the job (23%) and work experience (20%).
5.2.3 Employment History (In Last Two Years)
Employed
Overall, 97% of respondents had been employed in the last two years.
Employed in One Plus Job
Overall, 91% had only been employed in one job.
Hours Per Week Worked
Overall, respondents had worked an average of 45 hours per week, with a maximum of 84 hours, and a minimum of 20 hours.
Hourly Wage
Overall, respondents had earned an average of $8.50 per hour, with a maximum of $25.00 per hour and a minimum of $5.00 per hour.
Type of Work
Overall, 40% of respondents were employed as a labourer, 31% in the trades and 26% in sales/service.
Length of Time at Main Job
Overall, respondents stayed at their main job for an average of 2.85 years.
Reasons for Leaving Main Job
Overall, reasons for leaving their main job included the end of a contract (26%), shortage of work (23%), and moved (11%).
Employed Full Time
Overall, in the past two years, respondents indicated that they had been employed, on average, 14 months full time; 11 months part time; unemployed 14 months; attending school 11 months; and enrolled in a training course 5 months.
Received the Following Training
Overall in the last year, 11% of respondents had been in a training project, 9% in job placement, 9% in academic upgrading and 9% in other (e.g. apprenticeship, WHIMS, AIDS Awareness).
Services Currently Receiving
Overall, 11% of respondents had received in the last year, or were currently receiving job-specific training, while 3% were receiving job placement services.
5.2.4 Respondents Currently Working
Current Employment Status
Overall, 94% of respondents were currently working, with the remainder unemployed and looking for work.
Hours Per Week
Overall, respondents were working an average of 40 hours per week.
Hourly Wage
Overall, respondents were earning $8.33 per hour.
Employed Since
Overall, 18% of respondents had been employed since March of 1997, 15% since October of 1996, and 12 % each since January and February of 1997.
Type of Work
Overall, 58% of respondents were working in the processing, manufacturing and utilities industries, with 12% each in sales and service and trades/transport/equipment.
Problems Encountered
Overall, problems encountered in the current job included other (accommodation, money, weather) 21%, availability of childcare (15%) and transportation problems (15%).
Reasons for Not Working
Overall, half of the participants cited an inability to arrange child care as the reason they were not working, and funding cutbacks as the other half (these responses were based on two participants).
Receiving Other Assistance
Overall, only one respondent was receiving Social Assistance and one was receiving employment insurance.
5.2.5 Personal Perceptions of Life
On a scale of 1 (not likely) to 5 (very likely), respondents gave the following average ratings:
In the near future
It would be fairly likely (2.7) that they would be placed in a career job.
It was likely (3.9) they would be applying existing and new skills.
It was likely (4.0) that they would be in a company/industry that they enjoy.
It was very likely (4.7) that they would be employed.
It was not likely (1.4) that they would be unemployed.
In the longer term
It was very likely (4.7) they would maintain steady employment.
It was likely (3.0) they would be in a career-track position.
It was likely (3.8) they would be with the same company.
It was not likely (1.2) they would be on social assistance.
It was not likely (1.0) they would be unemployed.
5.2.6 General Statements About Work
On a scale of 1 (strongly disagree) to 7 (strongly agree), respondents gave the following average ratings:
They strongly agree (7) that they want to be working in the near future.
They strongly agree (6.8) that it is important to have work which makes good use of their skills.
They disagree (2.1) that there are no good jobs out there for them.
They strongly agree (6.8) that they want to be working in an interesting field.
They agree (5.6) that they want to be in a career track position.
They strongly agree (6.9) that being respected in their work is important to them.
They strongly agree (6.7) that they are ready for a full-time position.
5.2.7 Summary
Based on the findings, Marlboro has the lowest level of education, the highest income in the last two years as well as now, the highest confidence for being in a job that they enjoy. Kehewin has the a high level of education, an average level of income, less than two years of average job retention, and a reasonable belief that they will be in a job that they could enjoy in the future. Alexander has a high education level, average level of income, less than two year job retention and high belief that they will be in a job that they could enjoy in the future. Goodfish has a low education, low income (reduction in the current position), job retention less than two year, and a reasonable belief that they will be in a job that they enjoy in the future.
5.3 Employer Findings
The majority of the employers interviewed had never accessed government-funded employment programs, mainly because they had never heard of any that were applicable to their business. Employers were keen to access wage subsidy and felt that they were particularly helpful in the training of new employees. Four out of ten employers had heard of WADA, however three of these understood it to be a wage subsidy program, and only one mentioned the concept of consumer leverage.
All employers interviewed had Aboriginal and First Nation people on staff and planned to hire more Aboriginal and First Nation workers in the future. The majority of employers were situated on the reserve or in the community and were committed to supporting individuals living in the area. Others indicated that as long as the Aboriginal and First Nation person had the qualifications and motivation they were looking for, they would hire them.
When asked if they had any other comments, the following responses were given:
5.4 Comparison Group Analysis
5.4.1 Use of Comparison Groups
Human service evaluations are conducted not only in order to better understand participant outcomes but more often to assess the effectiveness and efficiency of a particular program. Two types of evaluation design are most commonly used to conduct this type of evaluation. It is done either by comparing two different programs with the same outcome goals or by tracking the outcome of two different populations, those that receive the service and those that do not.
The Stronger Together Strategic Initiative has a comparison component as part of its overall national framework research design. It is designed as a quasi-experimental evaluation design using comparison groups, rather than randomly assigned control groups, as the baseline against which to measure net program impacts. The primary tool was the Strategic Initiatives Baseline Survey for Participants.
After careful review the research design and tools proposed for the comparison study were not deemed suitable nor acceptable for use with the participants in the Western Aboriginal Development Alliance Strategic Initiative. The evaluation team identified this difficulty at the onset of the formative evaluation stage. The rationale for not using a comparison group were many and obvious:
Based on the above mentioned factors a combination of two methods were proposed, the use of census data combined with focus groups in each community.
5.4.2 Census Data
Three primary sources of data were identified: Statistics Canada census data from 1991, Indian and Northern Affairs Canada Census Data 1995, and local data available from the communities involved in the evaluation. One additional source, Alberta Government, Aboriginal Affairs, was identified as a source of information during the project. After analysing the data available from all of these sources it became evident that very little information is available at the community level outside population profiles.
Statistics Canada had detailed census data from Alexander community, but the completeness of this data is not guaranteed. The other three communities had not participated in the 1991 census data collection.
Local community census data proved to be the most available and reliable. These data were cross-referenced with statistical data, when available, to ensure the validity. The community data were provided by the human resource personnel in three of the four communities, Goodfish, Kehewin, and Alexander. The community of Marlboro is part of the county of Yellowhead, and the data can therefore not easily be disaggregated. Still, estimates were provided by the project co-ordinator in the community.
5.4.3 Focus Groups
The strength in conducting focus groups is the ability to interview more than one person at a time and for participants to provide their response in a social context where people can consider their own views in the context of the views of others.
The foundation for successful focus groups are three fold:
During the initial meetings in each of the four communities the prospect of holding focus groups with persons of similar profile as those currently involved in the project was presented and discussed. Careful consideration was given to ensure that local control was maintained. The logistics of the groups such as the composition of the groups, flexibility in timing, location, size of the group and the questions to be asked, were all to be developed collaboratively with the community co-ordinator.
Three of the four communities declined to participate in the focus group sessions since they felt strongly that that particular method of data gathering did not reflect their traditions and local protocol for interaction. Also, the concept of having people discuss some very delicate issues related to employment barriers was a great concern.
Findings From Census Data
Community profiles were established using data available from the above mentioned sources. The three types of data sources, Statistics Canada, Government of Alberta, Aboriginal Affairs and verbal interviews did not always concur. In particular, the census data available from the community of Alexander did not correspond in detail with data from Alberta Government. The main reason for this is believed to be the difference in the year the data were gathered (1991 and 1995).
Chart 1 below provides a brief profile of the four communities that participated in the evaluation.
CHART 1 | |||||||
COMMUNITY PROFILES | |||||||
ALEXANDER FIRST NATION |
GOODFISH (Whitefish Lake First Nation) |
KEHEWIN CREE NATION |
MARLBORO | ||||
Size of community (1995) |
1,255 |
1,515 |
1,359 |
150 | |||
Total # of working age 20 - 59 Male Female |
627 (49%) 303 (24%) 324 (26%) |
721 (48%) 370 (24%) 351(23%) |
585 (43%) 261 (19%) 324 (23%) |
Approx 50 | |||
Level of Education |
< grade 8 12% grade 8-12 50% post sec. 26% |
grade 9 (average) |
grade 9.7 (average) |
n/a | |||
Employment level |
44% |
30% |
25% |
n/a | |||
Average income |
<10,000 66% <20,000 22% |
n/a |
n/a |
n/a | |||
Proportion of population living on reserve |
743 (59%) |
713 (47%) |
844 (62%) |
not a reserve | |||
# of persons placed |
12 |
28 |
10 |
18 |
Community Size
As can be seen in the chart above three of the communities are very similar in size. The fourth community Marlboro is very small, a tenth of the size of the others. About half of the total population is of working age (20 - 59 years of age).
Level of Education
The average level of education is some secondary education grade 9 - 10. Alexander has the highest level with one quarter of their population having post-secondary education.Level of Employment
Unemployment levels are much higher in all of the communities as compared to provincial levels. In three of the communities the unemployment is around 70 - 75 %. The exception is Alexander where the unemployment level is reported to be around 55% (1991).5.4.4 Focus Groups Findings
As was discussed above the focus group methods was not an effective way of gathering data in these communities. The findings were predictable and not based on any in-depth discussions. The following are the findings from the focus group session:
Important Elements of Work
The following responses were given as the most important elements of work:
The following were not considered to be important:
Jobs in the Community
Most participants felt that the easiest place to find jobs was on the reserve.
Looking for a Job
All participants were currently looking for a job.
Barriers to Employment
The barriers to employment cited by the participants were as follows:
5.5 Impact of WADA Strategic Initiative
Participants were told about the scope and nature of the WADA SI, and were asked whether they felt an initiative of this kind would have an impact on their entry into the workforce. The following responses were given:
All participants indicated an interest in participating in the program.
5.6 Participant Profiles
As was described in the Research Design section (Section 3), the original intent of the study was to conduct 2-3 case studies in order to better understand what the clients experienced as participants in WADA and what the outcomes were. The strengths of a case-study are its ability to allow the same condition(s) or element(s) to be studied from several perspectives. The proposed model was a multi case, embedded strategy of inquiry.
After carefully reviewing each participant that had been interviewed it became apparent that there was no one case that encompassed all the variables under investigation. Also, it became apparent to the evaluation team that the strengths of the project were too diverse and broad in nature to be truly captured and understood through a few case studies. Therefore, it was decided that a brief profile of each participant be prepared and submitted. The participant profiles are provided in Appendix H of the Technical Report.
The profiles describe the following:
Based on the qualitative and quantitative data collected in the formative evaluation, the following conclusions can be drawn of the Western Aboriginal Development Alliance Strategic Initiative.
6.1 Overall
The following general conclusions can be reached regarding the WADA Strategic Initiative:
6.2 Success in Meeting Program Objectives
The following measures the success of the Strategic Initiative in meeting the program objectives as outlined in Section 2.
A total of 678 participants have been funded to date through the four communities, as follows:
Of this total of 67, the consulting team was able to interview 35 participants, and of these, 33 (94%) were working and 2 (6%) were unemployed. Thus, to date the Strategic Initiative has been successful in securing employment for its participants.
6.3 Success in Meeting Other Stated Objectives
6.4 The Strategic Initiative
The models used to implement the program differed among the communities based on local needs. The strength of the Strategic Initiative were:
6.5 The Appropriateness of the Strategic Initiative in Aboriginal and First Nation Communities
It does not appear that consumer leverage is always an appropriate approach to creating long term employment for Aboriginal and First Nation people, for the following reasons:
6.6 Administration of the Program
Several conclusions can be reached regarding the administration of the program, as follows: For the most part, the WADA staff were well received by the communities, but many administrators indicated they would have liked to have had more interaction with them.
6.7 Program Outcomes
7.1 The Conclusion of the Two Year Pilot Project
If HRDC and WADA want to assess the Strategic Initiative, in its purest sense, in the final year of the pilot project, participants should be funded only where consumer leverage is used to create the employment opportunity. In addition, long-term planning documents between the community and the employer should be submitted before funding is provided.
7.2 Administration of the Program
Some suggestions on administration of the program are as follows:
7.3 The Future of the Strategic Initiative
Last Modified: 1999-02-24 | Important Notices |