Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

I. Introduction

This report summarises evidence gathered during an evaluation of the Labour-Management Partnerships Program (LMPP) conducted in 1997. The evaluation examined three core groups of evaluation issues: program rationale, delivery and effects. To address these issues, the evaluation made use of two main lines of evidence, in addition to a literature and documentation review, as follows:

• Interviews with 11 key informants with varying degrees of association with the LMPP, including those directly involved in the program, those who have been involved in the review process, academics with professional interest in labour-management relations and national representatives of labour and business.
• Mini case studies of 18 LMPP projects. The distribution of case studies, selected from among completed projects, was: three involving research; three conferences; six involving workplace change in a single workplace; and six in multiple workplaces. The case study methodology included document reviews, interviews by telephone and in person, and site visits and discussion groups.

II. Findings: Rationale Issues

The majority of respondents believed the government was justified in encouraging partnership between labour and management. Just as the literature review indicated, respondents noted that, despite evidence that firms involved in worker participation tended to perform better, a majority has still not adopted a "co-operative workplace" model. In light of this apparent "market failure", there was an expressed need for a third party like government to broker with labour and management to experiment with new ways of doing things together. There was also moderate support for a federal presence for reasons of equity, efficiency and international competitiveness.

With respect to the types of projects funded, opinion was mixed: some felt that research was more cost-effective and reached more people, while others held that workplace changes provided practical examples that other workplaces can readily relate to and emulate. Evidence indicated that funding was mainly complementary with other Human Resources Development Canada (HRDC) programs and services and that few programs exist at the provincial level that overlap with the federal LMPP.

III. Findings: Delivery Issues

An important delivery issue concerned the appropriateness of the assessment or selection criteria used by LMPP officers to judge candidate projects. Most of those consulted for this evaluation thought the criteria overall were important, especially the dissemination of information, incrementality, the joint labour-management partnership, and the 50/50 cost sharing of the costs by labour and management.

For one criterion, however, some concerns were raised. The requirement for innovativeness might seem appropriate in theory but may be problematic to enforce because it is difficult to come up with something entirely new in the industrial relations field.

To what extent were these selection criteria met? First, it was generally agreed that, although most projects were not strictly innovative in the sense of making a unique contribution to the overall body of knowledge on labour-management relations, all were likely unique to the particular sector in question. Second, there were some questions raised with regard to the partnership criterion and the motives of sponsors. There was speculation that labour was often involved just to obtain LMPP funding and not for the sake of forming true partnership. Third, the cost sharing criterion served to redress resource imbalances between labour and management, with the latter tending to contribute more financially while the former tended to make up their share using in-kind contributions.

Two separate funding issues were raised regarding program delivery. First, most thought that the overall level of funding for the program was insufficient to effect change in the economy. On the other hand, annual program budgets were rarely used up, possibly indicative of excess funds and deficient demand. Second, the smaller projects, which perhaps were in the most need of change and assistance, were the least likely to obtain funding from the LMPP because they posed the greatest risk of failure, even though the amount risked in these projects was smaller than that contributed in larger projects.

Concerning administrative processes, evidence gathered from the participants in earlier projects uncovered some dissatisfaction with the length of time it had taken to get applications for LMPP funding approved. However, program officers have since reacted positively to these complaints by streamlining the review process and shortening delays down to two months. As for reporting requirements, most respondents felt that they were somewhat, but not overly, bureaucratic and demanding; most agreed, however, that they were necessary in order to maintain checks and balances.

IV. Findings: Project Level Impacts

All project participants and program staff consulted for this evaluation indicated that outputs were generated by projects as set out in the respective LMPP contribution agreement. In all cases, the immediate, first-order, intended outcomes (e.g., the implementation of International Standards Organisation (ISO) certification, the development of a work-reorganisation training framework, the carrying out of research) were achieved and disseminated. In many cases, workplace innovations led to other tangible impacts, such as productivity increases, better and cheaper service delivery, higher morale, and lower scrap rates. Second-order impacts were also realised, as greater labour-management co-operation within the workplace, in the form of information sharing and joint committees. Such impacts were an important outcome of many projects, at least for the project's duration, and, in some cases, beyond its life.

Our research identified a number of factors, both inherent and external to individual projects, that can contribute to or inhibit the success of the projects or be indicators of success. It is suggested that both potential project partners and LMPP staff should take these factors into consideration when considering potential projects and evaluating completed ones.

With respect to inherent outcome factors, the following were identified as either affecting the final outcome or inhibiting its measurement:

• Genuine partnership between labour and management: The more successful projects are the ones where both parties are actively involved in the activities throughout a project's life; having some prior history together also helps.
• Other innovations to complement the main innovation being funded: As past research has shown, successful organisational changes are the ones introduced in complementary groups, not individually.
• Quality: Recognition was wider when the product was of high quality, particularly with respect to research projects, conferences and video productions.
• Self-selection: Tied in with the notion of incrementality, the fact that project partners already have an interest in improving relations between labour and management predisposes them to reaching a successful conclusion, which makes attribution to LMPP's contribution difficult.
• Relative size of contribution: As the LMPP contribution often represents a small proportion of total project funding, it is difficult to measure the specific role of the LMPP in the project's impact.
• Windfall gains: Most respondents believed that the incrementality criterion was being only partially met in projects, i.e., that windfall gains were occurring, in that projects would have proceeded without LMPP funding although in altered form (say with less labour involvement); moreover, the original LMPP contribution served as seed funding to leverage money from other partners.

A separate set of factors external to the immediate project can also condition a project's impacts. First, there is instability in leadership. Projects were more likely to succeed and the innovation to be sustained if the original sponsors and participants maintained their presence throughout the duration of the project and beyond its completion; however, it is a fact of life that management and union leaders change frequently. Second, there is the primacy of collective bargaining as LMPP activities often must be set aside during contract negotiations, thus possibly affecting the momentum of the projects. The third external factor affecting outcomes is the vagaries of industrial relations and business conditions. Underlying relations between labour and management could turn sour for external economic reasons unrelated to the project, which could colour participants' interpretations of its value. Fourth is faddism: management is often quick to drop an innovation when something new comes along, which demonstrates the difficulty in ensuring true commitment.

V. Factors Affecting Program Level Impacts: Strengths and Weaknesses

Respondents generally concluded that it would be difficult to assess program level (third-order) effects - enhanced relations between labour and management more generally. The main reason is that the program's impact can really be felt only over the long term and cumulatively over several projects. Moreover, evaluation respondents noted a number of potential program level shortcomings, which could limit the appeal and ultimate success of the program. First, a number of participants thought that the level of program funding was too low to effect real change in the overall industrial relations climate of the country. Second, many respondents thought there was inadequate outcome measurement: they found it unacceptable that corroboration of outcome attainment and dissemination was based mainly on the word of the project partners. Third, many identified the program's low profile: annual program budgets were rarely all expended, implying insufficient marketing.

Respondents also identified a number of program strengths, which have increased its appeal and should lead to positive outcomes down the road. First, there was the variety of projects, which has likely expanded the program's appeal, although it may also contribute to a lack of understanding of the program and an ostensible lack of focus. A second strength of the program was its adaptability, as demonstrated by the streamlining of its assessment process in response to concerns expressed over the length of the review and approval process, which has also expanded the program's appeal. Third, several thought that the program's third-party neutrality helped to defuse adversarial relationships and to encourage parties to persevere to a successful solution. This could also be said for the partnership and cost sharing criteria, which give parties a stake in the project and thus a reason for "sticking it out".

Two other strengths were identified. One was the participation of labour. Labour would not have been given a voice in the organisational change in some cases without the LMPP's insistence on partnership, which implies some incrementality. The other strength was the synergy that exists between the LMPP and the rest of the Mediation Services section of HRDC, which has resulted in uncovering potential cases, identifying potential weaknesses in projects, and providing expertise and advice.

VI. Suggestions for the Future

These observations led to a number of suggestions as to the future for the LMPP. These are as follows:

• Consider a market study, including a survey, to gauge demand for and awareness of the LMPP.
• Consider a market-test funding strategy whereby funds would be raised from a number of project partners (thus demonstrating substantive interest in the initiative) before the LMPP would contribute any funds.
• Draw up a list of standardised outcome indicators, specific to the type of project, that the sponsors of all projects should apply to their projects to quantitatively measure outcomes. Some of the instruments developed in individual projects could serve as models. The results would be used both by LMPP officers to market the program and by project sponsors to disseminate project outcomes as required by the contribution agreement.
• Consider other marketing strategies such as more aggressively using the Internet, their own field officers, and job fairs to publicise the program.
• Stay with its decision to place greater emphasis on workplace change projects than it has done in the past, thereby equalising the distribution between these types of projects and research and conference projects. The reason is that, whereas the latter are cheaper and tend to reach a wider audience, workplace change projects provide practical demonstrations that other workplaces and sectors can readily learn from and emulate.
• Give greater consideration to funding on the basis of need, i.e., to funding projects involving smaller firms, or to encouraging smaller firms to form consortia to take advantage of economies of scale. Smaller firms are most in need of expertise and assistance, although they are also the most risky investments, while larger firms are more likely to be able to afford the innovation.
• Consider limiting funding per project to $50,000 based on assumed consultants' time and thereby spreading funding over a larger number of smaller projects as opposed to a few large ones. This would widen the program's appeal, although at the possible risk of diluting its focus and increasing the chances of funding failures.
• Ensure that there will be genuine partnership between labour and management throughout the project and continue to allow labour to contribute in-kind.
• Consider projects that have national or regional significance, even before ensuring they fall under federal jurisdiction.