SERVICE DIMENSIONS AND DRIVERS OF OVERALL SATISFACTION

This section presents the results of multivariate analyses undertaken with the data to better understand how clients perceive the various dimensions of service delivery and the factors that are most important in terms of driving overall satisfaction.

Overview

Factor analysis was performed to reduce the 14 service-dimension variables (Qs 10-24) into a smaller set of composite measures. Using normal cutoffs for the number of factors yielded only one factor, demonstrating that all of these variables are highly interrelated. The exceptions were the variables for protection of personal information and service in the official language of choice. These variables did not correlate well with overall client satisfaction, and were removed from the subsequent analysis.

When forcing a larger number of solutions on the factor analysis to see if this would reveal any grouping of variables that could be used to create distinct composite measures, the most sensible results were obtained using a three-factor solution. The three factors were information quality, staff quality and access-speed (including accessibility and timeliness).

In terms of driving overall satisfaction, the most important service dimensions were the quality of the staff and, to a slightly lesser degree, the quality of the information. If the average rating of the quality of the staff or the quality of the information increased by ‘1’, we would expect the overall client satisfaction rating to increase by .3. While changes in an average rating of .3 may seem small, this represents 30% of the clients increasing their overall satisfaction rating by one full rating point on the 5-point scale. The staff and information factors account for 54.5% of the variance in overall client satisfaction. Access- speed was also statistically significant, but much less so.

This is consistent with the bivariate measures. For example, over 60% of clients who provided a low rating on the information or staff factors also provided a low overall client satisfaction rating (1-2), compared to only 37% of clients who provided a low rating on the access-speed factor. The average rating for the latter was lower than for the other two factors, indicating more room for improvement, but potentially less impact on overall client satisfaction.

The impact of these three factors on overall satisfaction varied substantially across the different programs/services used by the clients. For instance,

• The quality of the staff was dominant for CPP Disability, Employment Programs/Assistance and SIN
• For EI and 1-800 O’Canada, staff quality was still important but other factors had lower but sometimes similar coefficients
• For OAS, the staff quality and information factors were similar, and access- speed was not statistically significant
• For Passport, the information quality and access-speed factors were the key drivers, while staff quality was not statistically significant.
• Information quality was the only significant factor for GIS and CPP clients.

The key drivers for the client types were as follows:

• The quality of the staff was a critical factor for the overall client satisfaction of some of the most vulnerable groups served - visible minorities, persons with disabilities, Aboriginal persons, and new Canadians.
  • Persons with disabilities seemed to be especially influenced by the staff ratings; information quality was also a driver, albeit less dominant
  • For Aboriginal persons, access-speed was the second strongest driver of overall client satisfaction
• For males and working-age Canadians, staff quality had the strongest impact on overall satisfaction; the other two factors were statistically significant but smaller
• For youth, females and seniors, all three factors were similar in magnitude.

Detailed Findings

Overall Service Dimensions

Initially regression models were tested to predict overall client satisfaction using all of the core service dimensions (Q10 to Q24). Due to the large number of highly interrelated variables, the model coefficients for each variable tended to be small and were sometimes unstable or yielded questionable results. To avoid the multicollinearity problem (high correlation among the predictor variables in the regression equation), we tested the possibility of using factor analysis to reduce the 14 variables into a smaller set of composite measures. Using normal cutoffs for the number of factors yielded only one factor, demonstrating that all of these variables are highly interrelated and could be used to create an overall client satisfaction index. The exceptions were the variables for protection of personal information and service in the official language of choice. These variables did not load highly on the single factor and did not correlate well with overall client satisfaction. These variables were removed from the subsequent analysis since they showed little value in understanding the pattern of client ratings and were not useful in predicting overall client satisfaction.

To see if forcing a larger number of solutions on the factor analysis would reveal any grouping of variables that could be used to create distinct composite measures, three- and four-factor solutions were initially tested using the variables. The most senesults were obtained using a three-factor solution. Based on the loadings of the variables on these factors, the three factors were information quality, staff quality and access-speed (including accessibility and timeliness). Table 1.1 shows the core service dimensions associated with each of these factors. Reliability of these factors was tested using Cronbach’s Alpha, which confirmed the inclusion of these variables in the three factors. For the subsequent regression analysis, the factors were created using a simple linear composite by adding the variables in each factor. This composite index was converted to a five-point scale similar to the original variables by dividing the total score by the number of variables used to create the index.

Overall Client Satisfaction

Table 1.2 provides a summary of the relationship between overall client satisfaction and the client satisfaction service factors described above. The third column in this table provides the average rating for overall client satisfaction and the service factors, and the fourth column presents the simple Pearson correlation coefficient*  that provides a bivariate measure of the direction and strength of the relationship between overall client satisfaction and the client satisfaction service dimensions (ranges from –1 to +1). The fifth column provides another bivariate measure of how well the service factors match the overall client satisfaction ratings. A cross-tabulation of each service factor with overall client satisfaction was prepared to examine how often a respondent with a low rating on the service factor (1.0 to 2.5) also provided a low overall client satisfaction rating (1-2).

Based on this analysis, the most important service dimensions were the quality of the staff and the quality of the information. Both these coefficients exceeded .3 (the staff quality factor was close to .4). In other words, if the average rating of the quality of the staff or the quality of the information increased by ‘1’, we would expect the overall client satisfaction rating would increase by .3. While changes in an average rating of .3 may seem small, it represents 30% of the clients increasing their overall satisfaction rating by one full rating point on the 5-point scale. The finding that the staff-related variables are a key predictor of overall client satisfaction is consistent with the bivariate correlation analysis that showed the six top correlates were:

• Q11 Staff were responsive to your needs (.622)
• Q17 Staff gave you what you needed or guided you to others who could help you (0.613)
• Q21 Staff went the extra mile to make sure you got what you needed (0.610)
• Q10 Staff were knowledgeable and competent ( 0.598)
• Q24   Decisions were clearly explained to you (0.589)
• Q22  You were satisfied with the amount of time it took to get the service (0.588)

The staff and information factors account for 54.5% of the variance in overall client satisfaction. Access-speed was also statistically significant; however, the coefficient was smaller – approximately 0.2. This is consistent with the bivariate measures that showed the information and staff variables had a slightly higher correlation coefficient than the access- speed factor, and were also more likely show agreement between a low rating on the factor and a low rating overall. For example, over 60% of the respondents who provided a low rating on the information or staff factors also provided a low overall client satisfaction rating (1-2), compared to only 37% of the respondents who provided a low rating on the timeliness factor. The average rating for the access-speed factor was lower than the other two factors, indicating more room for improvement, but potentially less impact on overall client satisfaction.

*  It should be noted that for all three factors, the correlation between the factor and overall client satisfaction was larger than the correlation between overall client satisfaction and the individual variables included in the factor. That is, the composite measure was a better predictor of overall client satisfaction than any one variable in the factor.

Programs/Services

The overall client satisfaction rating was asked relative to specific programs/services the clients had experienced (e.g. EI, CPP, OAS, SIN, etc.). Table 1.3 shows the results when the same model was estimated, segmented by the program/service the client was asked about. To assist in the interpretation of the results, Table 1.4 lists the six top variables with the largest simple correlation coefficients with overall client satisfaction.

The three factors driving overall satisfaction (Table 1.3) varied substantially across the programs/services used by clients. For instance,

• The quality of the staff was dominant for CPPD, Employment Programs/ Assistance and SIN
• For EI and 1-800 O’Canada, staff quality was still important but other factors had lower but sometimes similar coefficients
• For OAS, the staff quality and information factors were similar, and access- speed was not statistically significant
• For Passport, the information quality and access-speed factors were the key drivers, while staff quality was not statistically significant
• Information quality was the only significant factor for GIS and CPP clients.

Client Type

Table 1.5 shows the coefficients for the models segmented by the following nine client types:

• Youth (18 – 30);
• Working-age Canadians (18 – 64);
• Seniors (60 plus);
• Aboriginal persons;
• Persons with a disability;
• Visible minorities;
• New Canadians (in Canada three years or less);
• Males; and
• Females.

The key drivers of overall satisfaction for the client types (Table 1.5) were as follows:

• The quality of the staff was a critical factor for the overall client satisfaction of some of the most vulnerable groups served – visible minorities, persons with disabilities, Aboriginal persons, and new Canadians.
  • Persons with disabilities seemed especially influenced by the staff ratings, with the simple correlation coefficients exceeding 0.7; however, information quality was also a driver albeit less dominant
  • For Aboriginal persons, access-speed was the second strongest driver of overall client satisfaction
• For males and working-age Canadians, staff quality had the strongest impact on overall satisfaction; the other two factors were statistically significant but smaller
• For youth, females and seniors, all three factors were similar in magnitude.

To assist in the interpretation of the results, Table 1.6 on the next page lists the six top variables with the largest simple correlation coefficients with overall client satisfaction.

[ previous  | table of contents | next  ]