Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

Evaluation findings pertaining to the consistency of the Provincial Benefits and Provincial Measures (PBPMs) with the EI legislation, the LMDA and government priorities as well as the relevance of LMDA programs and services are presented in this chapter. The findings draw on qualitative evidence gathered from the key informant interviews and the focus groups.

3.1 Compatibility of PBPMs with EI Legislation, LMDA and Government Priorities

Federal and provincial officials agreed that the PBPMs reflect the guidelines, principles and intent of the EI Act and the LMDA. In addition, the officials agreed that given the PBPMs are virtually identical to the former HRDC programs, they continue to be relevant and consistent with HRDC's priorities. Some provincial key informants also indicated that the PBPMs reflect HRDC's priorities in terms of the returns to work and the savings to the EI account.

Provincial officials believe that the PBPMs do not respond to all of Manitoba's priorities given that the province is facing skills shortages and needs to provide training to a broader range of individuals than simply EI clients, including in particular, the marginally-employed (working in low skill jobs without other skills)²¹. Provincial officials indicated they also have the responsibility for providing training and employment support to people on income assistance and they are now trying to integrate the delivery of services and programs targeted to this group with the delivery of some PBPMs. The process is not completed but this integration has been made possible by the LMDA.

Federal officials observed that the province is putting a stronger emphasis than HRDC did on serving individual clients as opposed to employer or project-based interventions, particularly through Skills Loans and Grants/Enhanced Fee Payer, which is consistent with the LMDA.

The content of the PBPMs was determined within the context of those parameters set out by the LMDA and the EI Act. In consultation with other senior managers in the federal government, provincial officials divided the dollars between the various PBPMs based on historical data and current trends in the labour market.

3.2 Relevance to the Needs of Individuals, Employers and Communities

Most key informants and focus group participants agreed that the PBPMs are relevant to the needs of individuals, employers and communities, though there are some limitations and service gaps. There was, however, some difference in opinion among Employment Centre staff. Some staff stated they can meet clients' needs better now than before the LMDA because they are aware of additional resources and provincial programs, whereas other Employment Centre staff disagreed and indicated they were better able to serve the needs of clients before the implementation of the LMDA because less local decision-making is possible now.

Community partners felt the Province is committed to increasing the relevance of programming as they noted that the Province's organizational structure to deliver PBPMs has improved since the LMDA was first implemented. The Province's willingness to evaluate service delivery (e.g., evaluating the accessibility of Employment Centres) also increases the confidence of LMDA partners. Finally, the approval process for PBPMs has become faster due to increased specialization of project officers. Evidence of the relevance of the PBPMs include: the availability of funding for regionally and provincially relevant projects (e.g., long-distance learning, health care training, accessibility assessments and sensitivity training for staff); improved focus on basic education; more partnering, including more involvement from employers; and the use of the individual training approach for certain areas where there is a high demand for training (e.g., computers and IT).

Key informants and focus group participants commented on some facets of the administration and delivery of the PBPMs that reduce the relevance of the programs. Some Employment Centre staff perceive that decision making (approval of expenditures) is more centralized than it had been prior to the LMDA and that this can cause delays in the implementation of programs. Many provincial officials, however, believe that there is probably more flexibility within the LMDA than is currently being utilized because LMDA partners are fairly new to the process.

Concerns were expressed regarding the eligibility criteria limiting accessibility to LMDA funded programming to EI clients, which the vast majority of respondents see as creating limitations in terms of serving the needs of the Manitobans. Individuals noted that the limited access to LMDA funding affects a significant cross-section of Manitobans, including:

• the marginally-employed;
• persons with disabilities who have been unemployed for a long time;
• mothers who have been unemployed for a long period (while raising children);
• Aboriginal peoples;
• training for youth "at risk";
• immigrants entering the province;
• income assistance recipients; and,
• people in rural areas where greater reliance on part-time and seasonal employment means lower levels of EI eligibility.

The issue of eligibility for the PBPMs is complex and confusing to many respondents (e.g., employers, staff, community partners, third-party delivery agents)²². Some respondents perceived there are important needs for employment and training programming that are specifically not addressed through the LMDA PBPMs. Other respondents did not feel that these needs were being addressed at all, whether through the LMDA or any other existing provincial or federal initiatives. As well, some respondents do not seem to be able to clearly identify and distinguish between LMDA dollars and provincial dollars and hence, which activities (i.e., training and employment programs) are being funded by the different sources of money. In fact, some programs offered through ETS can address the needs of both EI clients and individuals receiving/eligible for income assistance, so this confusion may not be unexpected. For example, Employment Partnerships aim to facilitate the employment of "eligible" unemployed participants. If the participant is an EI client, programming is delivered through third-party service providers who are under contract with ETS using LMDA funding. If the participant is receiving income assistance, programming is funded with provincial dollars. Another example is the Wage Subsidies program, an employment benefit under the LMDA, which is being integrated with the provincial wage subsidies programming.

Provincial officials indicated that Manitoba is in the process of developing a training strategy to address provincial skill shortages. These officials feel constrained in that they can not access LMDA funding to re-skill an existing worker who is in an entry-level position who is not an EI client.

Another issue with respect to the development of a provincial training strategy to address provincial skill shortages is how the Province will address skills shortages in the immediate future if it is not able to support longer-term community college programs or university programs that actually respond to skill shortage areas. As noted previously, Section 1.1, Background, the EI Act specifies that three years following the date of its enactment training could no longer be funded through payments made directly to a public or private training facility. Rather, training is to be funded, in whole or in part:

• through payments flowing directly to the client in the form of grants or contributions, and / or loans or loan guarantees, for the payment of approved services; and / or,
• with vouchers given to the client to be exchanged for approved services.

As a result of this change, Purchase of Training and Project-Based Training programs were phased out on June 30, 1999. In light of this policy change many third-party delivery agents stated that unless the longer-term needs for higher skill training are addressed, as well as the broader economical and social issues, the governments will not achieve the expected savings to the EI account. For example, they feel that quick skill programs to get people into low-level entry positions which they will likely no longer occupy six months later are not a solution. They explained that clients who lack solid and useful labour market skills will not be able to relocate into better employment and will thus be back applying again for EI support.

Some respondents (third-party delivery agents and provincial officials) noted that many persons with disabilities do not qualify as EI clients and as such their employment training is not covered under LMDA. While this client group has access to EAS, key informants do not feel that current EAS are sufficient to meet the long-term needs of these clients.

Another group of individuals that normally falls outside the "EI client" requirement is recent immigrants.

Relevance to Clients

Most survey participants agreed that the PBPMs met their needs and expectations and impacted positively on their lives. They were asked how interested they were in pursuing each of three different labour market activities (i.e., education or training program, starting a business and working on a full-time or part-time basis) in the week prior to starting their program. Overall, the results suggest that PBPM participants were well matched to their interventions. Specifically, Self-Employment participants were most likely to rate themselves as very interested in starting their own business; participants involved in education or training programs were most likely to rate themselves as very interested in pursuing education or training; and Employment Partnerships or Wage Subsidies participants were most likely to have been very interested in entering the workforce on a full-time or part-time basis. This evidence demonstrates a good match between participants' interests and the interventions they received. However, it should be noted that this question was asked of participants after they had completed their programs and may be subject to retrospective bias.

Overall, clients participating in the focus groups stated that the PBPMs they had access to provided hope, helped them gain self-confidence, gave them a sense of direction and increased their ability to get a job. In certain cases, clients stated that the services received even exceeded their initial expectations. However, many clients complained about not being adequately informed of available programs and services, or of available jobs and related details (e.g., employment prospects, salary range) by Employment Centre staff.

Most staff agreed that the PBPMs were doing a good job of meeting the needs of some clients but also identified programming gaps, administrative problems and insufficient communication as possible threats to the level of relevance to clients' needs. The PBPM guidelines are found by many staff to be too restrictive to respond to the needs of multi-barriered clients.

Another major issue is the lack of relevant training available in rural and northern communities. Key informants and focus group participants indicated that the availability of relevant training has been hindered by the replacement of the former Purchase of Training approach with the new Skills Loans and Grants/Enhanced Fee Payer program. There is a perceived need for more funding for distance learning approaches for clients in rural and remote areas (e.g., correspondence courses, courses over the Internet/e-mail). In addition, some key informants indicated that the LMDA will not provide funding for clients to attend an out-of-province training program, even if it is thought to make sense and is the closest available location (e.g., those living close to the Saskatchewan border). Key informants stated that they believe relevance could be improved with more local, decentralized decision-making authority regarding the delivery of PBPMs.

Community partners identified the following regarding the relevance of PBPMs to clients:

• The lack of financial resources has contributed to a focus on shorter-term programs when longer-term interventions are often thought to be more appropriate (e.g., for multi-barriered clientele).
• Skills Loans and Grants is regarded as relevant for assisting students with their tuition and educational expenses but there was a lack of funding for program materials and supports, such as textbooks or day care.
• A student loan or LMDA money is now considered income, which works against students who would like access to subsidized housing.
• The amount of the government contribution has decreased over the last few years and student needs are not being met to the same degree as they used to be. This lack of funds has compromised program effectiveness and relevance because clients are unable to access programming.

Relevance to Employers and Communities

Most employers felt that the PBPMs are relevant to their needs, those of individuals and of communities. Some employers indicated that the PBPMs could be more responsive to local needs and should better address the cultural and economic peculiarities of specific regions of Manitoba. They suggested that the lack of opportunities, the isolation and the cost of living in northern communities should be taken into consideration when developing programs. Several employers who took part in the focus group held in Winnipeg also indicated that the devolution of responsibility for labour market programs to the Province has increased the extent to which the needs of clients, communities and employers are being met.

Employers who participated in the Winnipeg focus group were involved in Apprenticeship (a subset of SLG), Wage Subsidies or Employment Partnerships. Results from the focus group indicate the following:

• Employer expectations for impact on their organizations tended to involve improvements to their human resources, either through training for employees or a long-term job placement of a suitable candidate.
• Although the programs currently delivered were deemed to be useful to employers (e.g., Wage Subsidies), the lack of programming to train and upgrade current employees represented a programming gap (e.g., the employees need to be laid off before they can access training under LMDA PBPMs²³).
• Employers are generally pleased with the degree to which programs meet their organization's goals and with the employees working for them through the Wage Subsidies program. There was some dissatisfaction with regard to the lack of information they were given on a candidate's skills as well as on the length of programs (which they generally perceive to be too short).
• Some employers stated that there is sometimes a failure to match the interests and skills of program participants and employers, and that there is a lack of follow-up to ensure the success of a program. In particular, employers in specialized fields for which no training is available in the province of Manitoba stated they take a financial risk in investing upwards of two years providing hands-on training to employees and therefore cannot afford a high turnover rate. Thus, the quality of the candidates they accept under the program becomes of primary importance because they will need to continue training these employees beyond the end of the wage subsidy and must be able to rely on these employees to stay with the company following the end of their training.
• Feedback from employers involved in Wage Subsidies programs indicates that these programs enabled the employers to provide a "decent" wage (i.e., above minimum wage) for needed employees during the subsidy period.
• Employment Partnerships are regarded by Employment Centre staff as relevant to employers' needs so long as a comprehensive assessment of the needs of the individual and the employer is completed.
• One particular issue raised during the evaluation relates to potential problems of specialized training for an industry such as aerospace. Specifically, the demand for workers and/or skills can change quite abruptly in this field, so sometimes highly skilled workers who have just completed their training (initiated when demand was high) can find themselves with no work if the demand decreases.
• It was noted that the approval process for PBPMs has become faster due to increased specialization for project officers (one deals with Wage Subsidies, one with training vouchers, etc.), thus there has been greater efficiency and more willingness by employers to participate.
• Finally, it should also be noted that all employers in Winnipeg felt they would like to be better informed about the available PBPMs.

Employment Partnerships are regarded by staff as relevant to employers' needs so long as "all the homework" is done about these needs. Specifically, a needs assessment has to be completed to ensure an appropriate match between employers and employees.

The evaluation found that the issue of eliminating Project-Based Training (PBT) and Purchase of Training (POT), was viewed as hindering the relevance to employers and communities. Specifically, the idea that training can only be done under the LMDA through the Skills Loans and Grants or Enhanced Fee Payer (EFP) program was viewed by many provincial officials and community partners as a limitation because this approach doesn't take into account the needs of industry. For example, when new companies are considering relocating to the province, there can no longer be an industry-based approach using LMDA PBPMs to develop programs at the local community college to meet the skills needs of such new companies. The phase-out of PBT and POT was felt by the majority of training deliverers (colleges, universities, institutions, etc.) to have seriously compromised the relevance of the PBPMs. Some respondents perceived that there is limited recourse to address the training needs of a particular sector, industry or employer, and that there is less incentive for training institutions to partner with local industry in addressing labour market issues.

Suggestions for Improvement

Key informants and focus group participants had numerous suggestions to improve the relevance of the PBPMs to the needs of individuals, employers and communities. Some changes and adjustments have already been made to enhance the relevance of the PBPMs. Prior to the transfer of dollars for the programs from the federal government, the province of Manitoba had a number of programs for income assistance recipients. Provincial officials stated that they have tried to take a blended approach where they use some of their provincial resources and some federal resources to bring together a package that will help meet the needs of a range of people (e.g., income assistance recipients) other than only EI clients²⁴. However, some provincial officials specified that they are limited in terms of the number of circumstances where they can use such an approach. It was suggested that greater relevance could be achieved through partnerships with external stakeholders to provide assistance to multi-barriered client groups when those barriers are beyond the Province's mandate and ability to deal with them under LMDA PBPMs.

Other respondents suggested there is a need for more dollars to be invested in skills development and training and for a lot more flexibility in the interpretation of the guidelines (e.g., third-party delivery agents cannot provide work experience under EAS). Other respondents felt that the issue was less one of the lack of relevant programming but more one of insufficient information and communication between governments concerning what programs can be accessed.

Also identified was the need for programs to be more responsive to the regional and local needs of communities. Some respondents felt that this situation would be improved with the recent hiring by HRDC of labour market information analysts in the north. Better labour market information and communication would also improve the relevance of programming.

Footnotes

21	At issue is the ability to access the EI Account to cover the provision of PBPMs to Manitobans who are not EI clients. The use of funds from the EI Account for the provision of training and/or employment programs under the LMDA is clear — pursuant to the EI Act, funds from the EI Account are to be used to reimburse the Province for PBPMs accessed by "EI clients". Under the EI Act, an EI client is defined as an unemployed person requesting assistance who; (i) is an active EI claimant; or (ii) had a benefit period that ended within the previous three years; or (iii) had a benefit period within the last five years and was paid maternity/parental benefits, subsequently withdrew from the labour force and would like now to re-enter the labour force. However, it is important to note that the EI Act does not prevent the Province of Manitoba from using its own resources to fund PBPMs accessed by Manitobans in need of training and/or employment programs, but who are not EI clients.
22	See footnote # 21.
23	See footnote # 21.
24	This is acceptable as long as the funds from the EI Account are only used to reimburse the Province for PBPMs accessed by "EI clients".