|
|
     |
||
 |
     |
 |
|
Until the late 1980s the Atlantic fishery provided some level of employment and income in five provinces for over 61,000 fishers -- 28,000 in Newfoundland alone — and over 60,000 plant workers. The most important source of jobs was the groundfish industry in which cod was the largest stock. Between 1991 and 1994 the Federal Department of Fisheries and Oceans (DFO) was forced to scale back the groundfish harvest on the Atlantic coast because of rapidly falling landings. A series of initiatives was mounted to help industry participants adjust to the resulting loss of employment and incomes. The Atlantic Fisheries Adjustment Program (AFAP) was put in place in 1990 to assist with industry restructuring. In 1992 the Northern Cod Adjustment and Recovery Program (NCARP) and in 1993 the Atlantic Adjustment Groundfish Program (AGAP) were set up to support industry restructuring and provide displaced fisheries workers with incomes and alternative employment opportunities. By 1994 the groundfish fishery was virtually shut down, resulting in what has been described as the largest industrial layoff in Canadian history. Following the recommendations of the Task Force on Incomes and Adjustment in the Atlantic Fishery in 1993, The Atlantic Groundfish Strategy (TAGS) was established in May of 1994 as a comprehensive, long-term response to this crisis. TAGS was originally designed to provide income and adjustment supports over a two to five-year period for some 30,000 displaced groundfish-dependent harvesters and processing workers in five provinces. The total TAGS budget was $1.9 billion, and delivery of the program was to be carried out by two Federal departments and two Federal regional development agencies, all working in collaboration with provincial governments, industry organizations, community organizations and private sector service providers. Treasury Board policy requires evaluation of all significant programs. The TAGS/HRDC Evaluation Study was designed as a formative evaluation to provide TAGS/HRDC program managers with useful data and analytical insights to support the ongoing development of the program throughout its life. The formative evaluation would in turn establish the groundwork for the summative evaluation of TAGS/HRDC after the program was over. Because the TAGS program is now scheduled to end in August 1998 — almost a year earlier than planned — and because of other changes in the objectives, structures and operations of TAGS/HRDC, the TAGS/HRDC evaluation was concluded in March 1997. This document will therefore serve as the final report of the TAGS/HRDC Evaluation Study. 1.1 Objectives and Structure of this Report The objectives of this report are twofold:
To accomplish these objectives, the report is comprised of the following sections: Section 1 — description of the TAGS/HRDC Evaluation Project, its principal activities, data sources and research methods; Section 2 — brief description of the TAGS/HRDC program and its discrete program elements; Section 3 — descriptive profile of TAGS clients and their adjustment activities; Section 4 — analysis of adjustment outcomes and assessment of HRDC contributions to adjustment; Section 5 — discussion of the influences of households and communities on the adjustment process. There are also four appendices. The first three appendices are technical in nature while the fourth one provides an overview of the Newfoundland economy. 1.2 Description of the Evaluation Study This report concerns the evaluation of those elements of the overall TAGS program for which Human Resources Development Canada (HRDC) was responsible. With a budget of approximately $1.7 billion (out of the total $1.9 billion TAGS budget), the TAGS/HRDC activities included income support, counselling, training, mobility supports, employment supports, early retirement programs and some job creation initiatives. Because of the scope and complexity of the TAGS program, the TAGS/HRDC Evaluation Study has employed a variety of research approaches including client and comparison group surveys, focus groups and key-informant interviews, community case studies, and examined program policies and delivery mechanisms. The study has also made extensive use of HRDC program and administrative data. Overall direction for the Evaluation Study and monitoring of ongoing work was the responsibility of the HRDC Evaluation and Data Development Branch with support from the TAGS Evaluation Advisory Committee (EAC). The ongoing Committee was comprised of representatives of the TAGS/HRDC program staff from HRDC National Headquarters (NHQ) and from the Regional Headquarters (RHQs), the Department of Fisheries and Oceans (DFO), Treasury Board Secretariat, and the Province of Newfoundland and Labrador. The Atlantic Canada Opportunities Agency (ACOA) and the Federal Office for Regional Development in Québec (FORD-Q) participated towards the beginning of the project. 1.2.1 Evaluation Issues In developing the methodology for this evaluation the consultant team interviewed TAGS/HRDC officials at the NHQ and RHQ levels. On the basis of these interviews, two evaluation issues were given priority:
Three other issues were identified as important, but due to the shortened time and reduced resources for the evaluation project they were not as fully addressed. These issues were:
1.2.2 Description of Research Activities The evaluation project began in October 1994 when the TAGS program was still in its early stages of implementation. The methodology design and workplan for the evaluation were approved by the EAC in January 1995. Research priorities and workplans were updated continually and approved at meetings of the EAC in October 1995, February 1996, and September 1996. The following table describes the major research activities undertaken as part of the TAGS/HRDC Evaluation Project. Table 1.1 Research Activities of the TAGS/HRDC Evaluation Project
1.2.3 Principal Methodologies and Data Sources 1.2.3.1 TAGS/HRDC Program Data The TAGS/HRDC Evaluation Study relied heavily on electronic file data for information about TAGS clients. There were three main sources of client file data: program data from the Client Adjustment Tracking System (CATS) database, TAGS pay data from the pay data file, and data from the administrative research files maintained in Ottawa by HRDC. The CATS database contains tombstone data on clients as well as essential information such as eligible pay rates and active program participation.3 CATS data used in this report was from the file as of February 1997 and therefore covers the time period from the beginning of TAGS to early February 1997. The TAGS pay data file contains information from the cards mailed in by TAGS clients and information on cheques issued to TAGS clients. This information was available for 1995 and 1996. The administrative research file contains historical data from the Unemployment Insurance (UI) data files, the Canada Job Strategy (CJS) files and Records of Employment as well as income taxation data.4 The UI files used covered the time period from 1988 to 1996 and the other files from 1988 to 1994. 1.2.3.2 Surveys A panel survey approach was used to track attitudes, activities and intentions of TAGS clients and to assess the actual and perceived impacts of the various TAGS program elements. The population for the study was TAGS clients who were eligible for pay from TAGS in January 1995. Initial data provided by the Newfoundland TAGS/HRDC RHQ indicated that 39,450 people met that criterion.5 To have results that would allow for analyses of subgroups and for drop-off of respondents in future waves of the panel study, a target sample size of 10% or 3,945 was set. This sample size would also allow for results that were accurate within plus or minus 3% at a 95% confidence level. The HRDC Administrative Research File contains detailed income and employment information for a 10% sample of Canadians with Social Insurance Numbers.6 This information is taken from Record of Employment files, UI Status Vector files, Canadian Job Strategy data and T1 and T4 Income tax data files. As a selection criterion, the TAGS client sample contains all TAGS clients in the HRDC Administrative Research File. This procedure generated a representative sample of TAGS clients for whom the Evaluation Team had access to detailed historical income and employment information (1988 onwards). For the population of TAGS clients this produced a sample of 3,856 clients.7 In general, response rates for the client survey waves were high. There was a very low refusal rate with the most common reason for non-completion being that the telephone number in CATS was invalid and the client could not be reached by telephone. There was a 70 percent completion rate in the first client survey wave, resulting in 2,709 clients being interviewed. The second survey wave was complicated by an announcement made while the survey was in the field in July 1996. The Minister of Fisheries at that time announced changes to the TAGS program that reduced the duration of the Program and made cuts to active programming. After this announcement, when respondents were contacted to be interviewed there was a higher refusal rate, the length of interviews that did take place dramatically increased and respondents were tending to give negative responses to all questions.8 After consulting with HRDC, the consultants decided to stop interviewing and rely on the third client survey wave for information on clients' income, activities and attitudes in 1996. For this reason the completion rate in the second survey wave is much lower than the first. 61 percent of the respondents interviewed in the first wave were interviewed a second time and, because interviewing of first time respondents had just begun when the survey was cut short, just 16 percent of the sample who had not been interviewed before were interviewed. The total completed sample was 1,750. In the third survey wave 2,078 respondents were interviewed. Because the focus was on looking for change over time, interviewing focused on repeat respondents. Over 90 percent of the completed sample were interviewed in either or both of the first or second survey wave. The completion rate for repeat respondents was 67 percent. In addition to this sample of fully eligible clients, a further sample was drawn of clients whose eligibility had expired on December 31, 1994. These clients had been on NCARP but did not fully meet the TAGS eligibility requirements. They were made eligible for TAGS to give them more time to adjust. This population group was studied separately to provide early indications of adjustment outcomes for TAGS clients after their eligibility expired. Because in total there were only 3,518 such clients, this sub-group of TAGS clients was over-sampled. The 10% of clients who were a part of HRDC's administrative data file were selected,9 but approximately 1,000 more names were selected randomly until 700 interviews in total were completed.10 In the analysis of the group, use was made of HRDC's 100% file of administrative data. To better understand the impact of TAGS, a sample of individuals who were not clients of the TAGS/HRDC program, but who were similar to TAGS clients, was selected as a comparison group. Given economic conditions in rural Newfoundland and in the areas within other provinces where TAGS clients are concentrated, it was difficult to find a comparison group that had similar demographic and social characteristics to TAGS clients. The sample frame chosen was that of individuals who were part of HRDC's administrative data file, had an Unemployment Insurance (UI) claim in 1994, lived in a rural area,11and, based on the National Occupation Code (NOC) classification, had jobs which did not require post-secondary education.12 A purposeful stratified sample was chosen for the non-participant comparison group. First, individuals were chosen to match the provincial composition of eligible TAGS clients. Then, within each province, members of the sample were chosen to match the age and gender composition of TAGS clients.13 In total 1,546 interviews were completed with the `non-participants'. The first panel survey of TAGS clients was carried out in October 1995. The second panel was begun in June 1996 but was terminated after approximately half the interviews were completed because of the impacts of the major program changes announced by the Minister of Fisheries and Oceans on July 22. The final client panel was conducted in November and December 1996. The NCARP Roll-over clients were surveyed in June-July 1995, January 1996 and again in January 1997. The non-participant group was surveyed in October 1995 and in November-December 1996. 1.2.3.3 Interviews HRDC staff at the national, regional and local levels and officials of other Federal and provincial departments were interviewed on a regular basis to track policy and program changes and perceptions of the effectiveness of TAGS/HRDC implementation. The findings from these interviews were presented in Program Update Reports prepared for each meeting of the Evaluation Advisory Committee. To understand the impacts of TAGS programs on adjustment and particularly on employment and job creation, interviews were carried out with `key-informants' in groundfish dependent communities. These included local government officials, officials in community economic development agencies or regional development commissions, leaders in fishing industry organizations (unions, processor associations, etc.) and local business groups (e.g., Boards of Trade), local journalists, and other community leaders. The interviews were carried out in conjunction with visits to the communities for focus group research. 1.2.3.4 Focus Groups Focus groups were used in the TAGS evaluation process to generate deeper understandings of the feelings, attitudes, motivations and aspirations of TAGS clients. As well, focus groups were used to study the attitudes and experiences of front line TAGS/HRDC staff members. Altogether three rounds of focus groups were conducted: regular rounds with TAGS clients and field staff in April-May 1995 and June 1996, and a special round in the fall of 1996 to examine household issues and the experiences of female TAGS clients. Each round consisted of between 12 and 16 actual groups with at least eight in Newfoundland each time and the rest distributed across the other four provinces. The groups were balanced between plant workers and harvesters, and between the smaller, more remote communities and the larger fishing centres. A special set of focus groups specifically with TAGS/HRDC staff in the Human Resources Canada Centres14 (HRCCs) was carried out in Newfoundland and Nova Scotia in early 1996 as part of a detailed examination of the community dynamics of adjustment. 1.2.4 Constraints and Limitations on the Evaluation Study The most significant challenge for the Evaluation Study was that frequent changes to the TAGS program required adjustments in goals, methods and workplans for the research. For example, the first survey of TAGS clients was scheduled for March 1995, but was postponed until October 1995 while the first series of TAGS program changes — most notably, the removal of the active requirement — was implemented. While the second survey was under way in July of 1996, the Minister of Fisheries and Oceans announced a number of additional changes including the end of TAGS adjustment programs and the reduction of the life of the program by as much as a year. Because of the direct impacts of these changes on attitudes and outlooks among respondents, conducting of the survey was terminated and a new panel was undertaken later in the year. With the termination of TAGS/HRDC adjustment programming in the fall of 1996, the more in-depth analyses of the impacts of TAGS income support, training, employment and mobility programs planned for 1996-97 were cut back. While these constraints did require adjustments in approach and scheduling, they did not in any way limit the quality or reliability of data collected by the various research activities, nor did they eliminate consideration of key evaluation issues altogether.
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![[To Top]](/web/20060127004314im_/http://www11.hrdc-drhc.gc.ca/edd-img/top.gif){kind=small}
![[Previous Page]](/web/20060127004314im_/http://www11.hrdc-drhc.gc.ca/edd-img/back.gif){kind=small}
![[Table of Contents]](/web/20060127004314im_/http://www11.hrdc-drhc.gc.ca/edd-img/content.gif)
![[Next Page]](/web/20060127004314im_/http://www11.hrdc-drhc.gc.ca/edd-img/next.gif){kind=small}