Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

The formative evaluation of the Western Aboriginal Development Alliance (WADA) Strategic Initiative utilises a triangulation approach to collecting the required data. That is, multiple data sources were accessed, more than one evaluator was involved in the study, and the results were analysed in more than one way (qualitative and quantitative approach).

The data sources which were accessed or collected during the course of the study were as follows:

3.1 Primary Data Sources

3.1.1 Meeting With HRDC Staff

A meeting was held with program staff from HRDC to review the expectations of the formative evaluation and to obtain additional insight on the program, its delivery and anticipated outcomes arising from the initiative. The use of consumer leverage was discussed and the necessity of using this model in the Strategic Initiative was reinforced. A copy of the HRDC staff questionnaire is in Appendix A of the Technical Report.

3.1.2 Meeting With WADA staff

A meeting was held with WADA program staff to review the delivery of the program, issues encountered to date and expected outcomes. Additional insight on the communities involved in the evaluation was collected at this time. A copy of the WADA staff questionnaire is in Appendix B of the Technical Report.

3.1.3 Joint Community Meeting

A meeting was held with representatives from the four communities involved in the evaluation to inform them of the evaluation, the steps which would take place to collect the necessary data, and their expected role in the process. Feedback was solicited from the group on their understanding and agreement to conduct the various aspects of data collection. One of the results of this meeting was the decision not to conduct focus groups in some of the communities.

3.1.4 Community Administrator Meetings

A meeting was held with administrative staff in each of the four communities to review various aspects of the program, including program records, methods used to identify participants and employers, and other areas of interest. Preliminary schedules for interviewing were established at this time. Employer contact names were collected at a later date. A copy of the questionnaire used in the community administrator meetings is in Appendix C of the Technical Report.

3.1.5 Program Participant Interviews (Baseline Questionnaire)

In-person interviews were conducted with program participants who were available during the course of the evaluation. A total of 35 interviews were conducted by a trained interviewer who was familiar with the sensitivities of the project. A senior member of the evaluation team arranged and attended the first few interviews in each community to ensure that all parties were comfortable with the process. The following interviews were conducted in each of the four communities:

• Alexander First Nation (Alexander): 6 (out of a possible 11 participants)
• Whitefish Lake First Nation (Goodfish): 16 (out of a possible 28- no participants from the garment factory were interviewed, as it was considered to be inappropriate to take them off the assembly line).
• Kehewin Cree Nation (Kehewin): 6 (out of a possible 10 participants)
• Marlboro: 7 (out of a possible 18 participants).

A copy of the baseline participant questionnaire is found in Appendix D and the detailed findings from the participant survey are found in Appendix E of the Technical Report.

3.1.6 Comparison Group Focus Group

A focus group was held in Alexander to collect information from community members with similar characteristics to program participants. The questions asked in the focus group and the results from the session are provided in Appendix F of the Technical Report.

3.1.7 Employer Interviews

Telephone interviews were conducted with ten employers from various communities and in a range of businesses (lumber, convenience store, school, paint store, grocery store, hair stylist, industrial construction). The following is a breakdown of employer interviews by community:

• Alexander: 2
• Goodfish: 1
• Kehewin: 3
• Marlboro: 4

The questions asked in the employer interviews and the findings are provided in Appendix G of the Technical Report.

3.1.8 Participant Profiles

Short write-ups were developed on all of the participants surveyed to provide a better understanding of the individual involved in WADA, including some history prior to starting the program, the assistance they received and their current employment status. This information is provided in Appendix H of the Technical Report.

3.2 Secondary Data Sources

3.2.1 Administrative Data and Operational Files

Administrative data and files were reviewed during the course of the community administrator meetings. This information was useful in the assessment of reporting practices and procedures.

3.2.2 Literature Review of Native Employment Programs

A literature review was conducted of Native Employment programs offered through the provincial and federal governments, to provide depth on the types of programs which had been tried in the past, as well as the findings and recommendations on areas which would be beneficial in the development of new programs. The literature review is provided in Appendix I of the Technical Report.

3.2.3 Additional Data

A review was made of the information available to complete a comparison group analysis. Information was collected through Aboriginal Affairs (provincial), Indian and Northern Affairs Canada, Statistics Canada and through the community.

3.3 Study Limitations

At the outset of this report, there are some study limitations which are worthy of note:

• The findings from this evaluation are based on four of the seventeen communities involved in WADA, and while different types of communities were chosen for study, it is important not to generalise the findings to all participating communities.

• There were roadblocks in the collection of data throughout the course of the study, and these prevented the collection of as much primary data as desired. The study team may have been able to collect additional insights and data if more time had been available to develop a relationship with the community.

• Comprehensive comparison group analyses were difficult to complete, since current detailed data on individual communities were not available. In addition, the lack of secondary data at the community or reserve level prohibited the study team from completing a detailed comparison group analysis. (See Section 5.4.1)

• The entire population of program participants was not available to be interviewed, given that many were working away from the community and were not returning during the study timeframe (and telephone contact was not possible), or the nature of the job (assembly line) discouraged the participants' supervisors from allowing them to be interviewed.

• Summertime presents some constraints in the Aboriginal and First Nation community, given the number of individuals off the community on holidays, as well as attending pow-wows, rodeo circuits and pilgrimages.

• The survey instrument has been modified from the original version to obtain more meaningful data from the Aboriginal and First Nation communities. It is important to keep this in mind when comparisons are made among the different Strategic Initiatives.