Directorates
·	Director General's Office
·	Data Development
·	Strategic Evaluations
·	Partnership and Continuous Evaluations
Publications
·	Evaluation Reports
·	Evaluation Briefs
·	Labour Market Development Agreement Series
·	Lessons Learned Series
·	Evaluation Tool Kit Series
·	Evaluation Forum Newsletter
·	From Theory To Practice

This section examines a number of evaluation issues regarding program results and impacts: To what extent did projects accomplish the "expected results" outlined in the contribution agreements? Did the projects influence labour-management relations at the workplace/industry or sectoral level? Did the LMPP enhance the productivity and performance of participating organizations, their associated clients or members? Did the LMPP have a significant influence on the settlement of labour disputes at the participating organizations or unions? This section also examines how positive results and impacts were achieved.

7.1 Overall Accomplishment of Expected Results

Evidence from the participant survey indicates that 92 percent of the projects accomplished goals outlined in their project proposal and contribution agreement.

The vast majority of projects selected for case studies achieved the expected results as outlined in their original proposals to HRDC, with 18 of the 21 case study projects fully meeting their objectives.

The participant survey corroborated this finding, with 87 percent indicating a high level of achievement of project objectives (as shown in Figure 8).

The participation survey also indicated that most project participants considered their LMPP project to be either very successful (87 percent) or somewhat successful (6 percent). Only 7 percent indicated that their project had little or no success (as shown in Figure 9). These results were very positive for a demonstration project program.

Although success levels were generally high, some shortfalls occurred. In spite of the fact that the vast majority of case study projects achieved the results outlined in their original proposal to HRDC, project participants were still somewhat mixed in their perception of this success or felt they "could have done more." For example, in four of the case study projects, labour and management differed in their view of the project's success. In these cases, some participants rated their projects as extremely successful and as having achieved all expected results, while others felt only some of the expected results occurred and were only somewhat successful.

In one such project, a labour participant who participated in a joint project found the effort extremely beneficial, while a management participant claimed that it was only "somewhat successful," explaining: "Results were achieved on a personal basis with my union partner, and the relationship/communication/mutual trust are certainly better than before, but the expected results — labour's trust of the company as a whole — were not achieved." In another project on methods to retain women's employment in male-dominated trades, coordinators felt that the project was extremely successful, although some labour and management participants noted that the project's long-term benefits had yet to be realized.

Other instances involved project adjustments during implementation. In the case of one less-than-successful project, labour-management tensions stood in the way of some of project objectives being achieved. Therefore, labour and management were not able to reach an agreement on the content of published materials, and the lack of agreement meant that a key report and informational pamphlets were not being published.

7.2 Impacts on Labour-Management Relationships

Nearly all LMPP projects had a positive influence on labour-management relations at the workplace level and more generally.

The survey of project participants indicated that nearly all of the projects were considered to have produced substantial benefits (60 percent) or some benefits (34 percent) on labour-management relationships. Overall, 94 percent of the management and labour officials included in the survey reported that labour-management relations benefited from the projects.¹⁵

Almost all of the case study projects also indicated positive effects on building trust, opening lines of communication and ameliorating longstanding difficult or tumultuous relationships.¹⁶ Only two of the case study projects were found not to have positive impacts on labour-management relationships as a significant benefit of the project.

The following viewpoints, drawn from the case study analysis, illustrate the benefits of LMPP projects on labour-management relationships: "[The project] created more trust so we can enter into other initiatives jointly and collaborate now; benefit plans are more effective now; joint activity [...] has helped us identify common goals.... [We are] more willing to agree on things as there is more trust [and a] much greater awareness of the work systems and the benefits of the work systems. [There is also] a much greater willingness to do business and people on our side are willing to take training in problem-solving and skill-training. Some notions of partnership are in place today as a result of that."

A number of underlying mechanisms were identified as facilitating the LMPP´s positive impact on labour-management relationships.

Five recurrent LMPP activities emerged from the case studies and point towards the underlying mechanisms that facilitated the LMPP's positive impact on labour-management relationships.¹⁷ Although this list of LMPP activities is not conclusive or exhaustive,¹⁸ it helps to identify a number of key aspects that make LMPP successful.

Sharing Information and Understanding Other Parties´ Positions and Interests:

Information sharing was the single greatest factor seen as aiding LMPP-related workplace changes. The case study analysis indicated the importance of sharing information, and comments in this area included:

• "Research was a collaborative disciplinary team [effort] with labour and management [working] on it. The conference allowed management to see how employees were impacted by their decisions [...] Helped create an understanding from both perspectives — management and employees."
• "By working together we were able to have labour understand what we were embarking on operating in a competitive environment. We were able to do our downsizing, but at the same time put together financial packages for older employees and find jobs within the company for a lot of employees not at retirement age."
• "[The project] broke down the walls by being able to talk to labour on a first-name basis [...] It let us carry on as a transit company and we were more knowledgeable about the 1999 contract talks."

Information sharing was reported as a factor in workplace improvements by 27 percent of the surveyed project participants.¹⁹ As a result of the LMPP projects, many labour and management participants reported they were in a position to share information more freely and openly. This information sharing had important effects on labour-management relationships, such as opening lines of communication and allowing each party to understand the position of the other. As a result, labour and management developed a better understanding of one another's positions. This increased understanding, facilitated better give and take, and aided the development of positive labour-management relationships.

Working Together/Meeting Regularly as a Joint Committee: Another LMPP feature that emerged as facilitating positive impacts on labour-management relationships was working together, particularly in the regular meeting of a joint steering committee that included senior representatives of management and labour. The positive impact of a joint committee was most clearly illustrated by a project designed to introduce work teams into automobile dealerships. In this project, two automobile dealerships were chosen as pilot sites, with somewhat different approaches taken for each. At one dealership, a consultant was hired to implement the pilot study, while a joint committee led the project at the second site. Although both dealerships had successes, the dealership with the joint committee had the added benefit of achieving a renewed relationship between union and management. The renewed relationship was attributed to working together as part of a joint committee and the information sharing that ensued.

This type of process emerged time and again as an explanation of changes observed in the case studies and was reported as a major factor by 13 percent of the surveyed participants. By meeting on a regular basis, and by involving senior personnel, labour and management were able to slowly build more positive and collaborative working relationships. These opportunities for labour and management to work together were often credited with the success of the project.

Working Towards Common Goals: Another key way in which the LMPP projects had a positive influence on labour-management relationships was through working towards specific, common goals. By working towards shared objectives in LMPP projects, both labour and management were able to put aside their differences and focus on common end goals. For example, a project on retaining women's employment in male-dominated trades described the positive benefits of labour and management working collaboratively. Project co-ordinators indicated that both parties were able to overcome differences to achieve the objectives of the project, leading to an improved relationship. This effort is illustrated by the comments of the principal investigator for one research project: "[The project] allowed parties to come together, to speak, to understand each other [and] to work together towards a common goal. [This] changed the fundamental way they communicated."

Meeting in a Neutral Setting : Another factor seen as aiding labour-management relations was the ability of labour and management participants to meet in a neutral setting — often outside of the workplace. Such meetings often occurred in the conference projects, which created neutral, non-adversarial arenas for management and labour to come together. The factor of neutral meeting ground was closely linked with the importance of common goals. Both aspects allowed labour and management to put aside differences to work together.

In demonstrating the importance of a neutral setting, one conference participant stated: "[It] brought labour and management from various countries together to share experiences and reach consensus on disability management programs, practices and policies." Meeting in a neutral environment often led to the opening of communication lines and the opportunity for positive dialogue. As one participant noted: "I talked to and spent time with these guys and was able to discuss conflict issues in a non-threatening way." Similarly, another participant in a series of roundtable discussions illustrated the importance of a neutral environment as follows: "[It was] an excellent opportunity for positive labour/management dialogue on the issues and a solid basis for continuing exploration in a non-adversarial environment, leading to a greater appreciation of the concerns of people on the other side of the table."

Implementing Practical Workplace Policies, Programs and Guidelines : A fifth factor that emerged (mainly from the case study projects) was the effect of implementing specific practical workplace policies, programs and guidelines. Examples of this occurred in the areas of interest-based negotiation, conflict resolution training, effective communication training and using clear language when writing collective agreements. Similarly, developing concrete policies in areas such as sexual harassment in the workplace and disability management also served to improve labour-management relationships. Another illustration was a project on ameliorating employee labour relations, which resulted in the "Working Relationship Agreement" signed between the City of Edmonton and the municipal unions. This agreement resulted in significant improvements in what had previously been seen as highly adversarial relations. These types of projects appear to underline the importance of LMPP projects focusing on practical, achievable and meaningful goals — to provide a positive "laboratory" for developing better labour-management relations.

7.3 Impacts on Productivity and Performance

The evidence shows that LMPP projects improved productivity and performance of the organization in most workplace projects to the benefit of both labour and management.

The survey of LMPP project participants indicated that most projects had a positive impact on productivity,²⁰ with 29 percent of respondents indicating substantial productivity benefits and another 56 percent indicating some productivity benefits.²¹ The survey showed that positive impacts on labour-management relations and productivity were intertwined, along with many other workplace factors. This relation was seen in improved labour-management relations, which boosted workforce morale, decreased absenteeism, reduced the number of workplace accidents, improved communication, and, in turn, aided productivity.

The case study analysis also provided evidence that LMPP projects had a positive impact on productivity and performance. Four of the case study projects provided detailed data on the effect of LMPP on enhancing productivity and workplace performance. In two projects, these types of benefits resulted from an improved business plan and included a forecasted increase in employment. Other projects that found benefits in these areas attributed them to the introduction of work teams. As was noted in one case study: "The project enabled better quality of services [to be] offered. Procedures were also enlarged." Although enhanced productivity and performance did not surface in all case studies, productivity benefits appeared to develop as a result of other effects of the LMPP, such as increased information sharing, reduced absenteeism, and increased worker participation in decision-making.

7.4 Impacts on the Prevention or Resolution of Labour Disputes

The evidence indicates that LMPP projects aided the prevention or resolution of labour disputes.

In several case studies, the LMPP project was credited with aiding the prevention or resolution of ongoing or imminent labour disputes. In two of the case study projects, LMPP projects were seen as a "last ditch chance" to renew labour relations and avoid impending disputes. The LMPP was also credited with reducing the number of grievances.

An illustration of these impacts can be seen in one project that implemented a new dispute resolution process. Another demonstrated that the presence of an HRDC mediator greatly enhanced their ability to reach a settlement and made it possible to "resolve grievances at an early stage before having to go through a long process." A third project discussed the high reduction in the number of grievances since implementing the project. "The grievance rate went from 150 grievances per year on average to 3 grievances per year on average. Misunderstandings and mistrust were removed [and] friendly relationships were created. Both parties tried to settle grievances instead of going to arbitration out of respect for their relationship."²²

The survey of management, labour and other project participants also indicated that LMPP projects aided the prevention or resolution of labour disputes, with 77 percent of the respondents reporting benefits in this area.

7.5 Other Impacts

LMPP projects increased worker participation in decision-making.

The survey of project participants indicated that increased worker participation was a result in 82 percent of LMPP projects (as shown in Figure 13).

The case study analysis indicated that LMPP projects had impacts on increasing worker participation in decision-making. This is illustrated by the comment: "Management actively and patiently encouraged every participant to express their concerns and they were not judgmental. ...Staff reported more involvement and knowledge about decisions involving staff and the City."

LMPP projects added to the existing body of knowledge.

Positive impacts of LMPP projects on the body of knowledge were noted by nearly all (96 percent) labour, management and other project participants surveyed.

Case studies also demonstrated specific impacts on an existing body of knowledge with many important other benefits. Some of these benefits were seen as including an increased awareness of workplace needs and improved labour-management relationships. For example, one case study participant noted: "[The project resulted] in a substantial benefit to labour-management relations and it substantially increased information-sharing and the overall body of knowledge on new and innovative approaches to the workplace."

LMPP projects contributed to workplace and public policy improvements.

A significant number of case study projects indicated broader effects on workplace and higher-level policies. These effects ranged from changes in workplace policies, to new collective agreement policies, to potential impacts on government policy. Five projects reported impacts on policy such as:

• A conference on pay equity resulted in a survey of collective agreement language that could be used for collective bargaining negotiations.
• A forum on disability management led to further international forums addressing this issue and served as a catalyst to obtaining a commitment to disability management from European countries.
• A study tour on successful union and management relations led to the involvement of one participant in their "staff relations re-organization program by exposing participants to a wider range of best practices."
• An investigation into the effects of contingent work led to an impact on public policy. "What had been learned from this project helped us proceed in 2001 with statutory changes, in particular amendment to the Public Service Act, which introduced a new classification of contingent workers, enabling the government to recruit highly skilled workers on a project basis but with full-time benefits."
• A conference on workplace stress and family violence resulted in a resolution seeking amendments to existing human rights legislation.

7.6 Lessons Learned

Five types of lessons were identified by the case study analysis, and they highlight the importance of having effective mechanisms to capture and share lessons learned regarding project success.

The importance of ensuring that successes have permanent value : The most common lesson involved the question of sustainability — how to make project impacts last.²³ Although little was learned at this stage on how to ensure sustainability, the case study analysis highlighted the importance of taking steps to ensure that successes have lasting impacts. This issue is examined further in the next two sections of this report.

The importance of having a clearly defined structure at the outset of the project :

Another lesson learned involved the structure of the projects. In four of the case study projects, participants emphasized the need to have a clearly defined structure at the beginning of the project. Ideally, this structure involved:

• clear and defined roles and responsibilities for participants;
• the involvement of key, decision-making personnel;
• well-identified and realistic expectations; and
• a strong commitment from all parties upfront.

The following comments by participants illustrate the need for a clear structure at the outset of the project:

• "There must be a unified stand from regional leaders prior to the commencement of the project. Regional leaders must be politically neutral and not simply adopt traditional stances when dealing with progressive issues around industry and technological changes."
• "[There is] a tendency to put the joint committee together and they become the new management. It is important to put it down to the operations [level]. We did not do that soon enough [which resulted in] significant setbacks and [we had to] have patience."
• "All stakeholders should be involved from the beginning of the process [...] Each party needs to be shown the benefits to them in order for any joint initiative to proceed."
• "One lesson learned was the need for collaboration and getting the participation from various groups."

The presence of a neutral third-party facilitator can be an important factor in project success : Participants from two of the case studies emphasized the importance of a third party neutral facilitator as a success factor. In one case, the third party facilitated non-adversarial discussions between the two main parties and helped improve labour-management relationships. In both of these projects, a non-governmental organization served as the third-party facilitator. The following assessment by one participant illustrates the potential importance of a facilitator: "Start off with an impartial third party — with a qualified, knowledgeable and experienced facilitator with a real knowledge of what they want to accomplish and what the project is — it's impossible to start any other way."

Recognizing and addressing situational considerations can be important to project success : Case study project participants also discussed some lessons learned that resulted from the specific focus of their project. These often involved situational needs that had not been forecast at the beginning of the project. For example, a project on family violence learned that there is a "need to pay attention to the needs of survivors by way of security."²⁴ Additional considerations included the need for simultaneous translation,²⁵ bilingual co-ordinators, and an awareness of external regulations and changes that might have an impact on the project.

Program benefits can be more effectively disseminated to program participants and non-participants : Some participants raised the issue of missed opportunities for sharing project information with their workplace and other environments. Also, the diminution over time of a project's impacts was associated with the departure of participants. A positive "spill-over" factor that was unmeasured, however, is the extent to which lessons learned from LMPP projects were carried to new workplaces by participants who "moved on."

Footnotes

15	Impacts were substantial in other areas as well as for labour-management relations and productivity. For example, positive impacts were reported in the survey of managers and union officials and others for: increased body of knowledge (96 percent), increased information sharing (94 percent), increased worker participation in decision-making (83 percent), settlement of labour disputes (77 percent), reduced grievances (71 percent), reduced injury rates (52 percent), and reduced absenteeism (49 percent).
16	Numerous projects were identified as having positive impacts on labour-management relations; among them one project represents a particularly dramatic instance. The project was seen as strengthening labour-management bonds enabling the parties to weather its tragic workplace killings in 1999. Another project was seen as a last-ditch effort to avoid impending labour disputes and an adversarial and antagonistic relationship.
17	It is noted that these findings may be an instance calling for additional research in the future to detail more fully the causal mechanisms at work.
18	Some general causal factors that emerged repeatedly in the survey included: information; the experience of working together; modeling of cooperative behaviors; efforts to increase mutual understanding; efforts to improve morale; and improvements to the workplace environment. Improved labour relations were also noted as a major factor leveraging other changes, such as changes in productivity, absenteeism, etc.
19	These reports were tabulated from open-ended comments. The researchers note that a high percentage would most likely have been obtained from a structured question.
20	Exact cost-benefit indicators could only be provided by a small minority of projects. These indicators suggested, however, that workplace LMPP projects were capable of leveraging extremely good increases in productivity.
21	Very few of the workplace participants could estimate an "exact" percentage improvement in productivity. Nine did, however, estimating that LMPP had an impact of increasing productivity (on average) by 4 percent per year.
22	Although labour dispute/settlement was always an important aspect of all of the LMPP projects, positive impacts on labour dispute/settlement were not seen in all projects, partly because such disputes were not always occurring. Additionally, a minority of projects faced specific obstacles during their implementation because of labour disputes, strikes and deteriorating labour-management relationships.
23	As noted earlier, sustainability was mentioned by 64 percent of those completing the telephone interview survey for this evaluation.
24	Survivors of Family Violence.
25	A number of projects identified a need for improved access to translation services as a "facilitator" of projects.