2001 National Client Satisfaction Survey

• Introduction
• Methodology
• Demographics of the Survey
• Client satisfaction
• Basic service
• The importance attached to values
• Specific aspects of the service transaction
• The three indicators of satisfaction
• Leverages of satisfaction
• Satisfaction with the application transaction
• Client expectations
• How does the Passport Office stack up?
• Our weaknesses
• Priorities for improvement
• A Passport Office action plan
Tables
• Demographics of the 2001 Survey
  
• Values Declared by Participants of the 2001 Survey
Charts
• The Application Process
• Service Responsiveness and Reliability
• Access to Service
• Communications with the Passport Office
• Cost of the Passport
• The Passport as a Travel Document
• 2001 Overall Client Satisfaction (%)
• Satisfaction with the Application Transaction (%): Top 10 Elements
• Satisfaction with the Application Transaction (%): Bottom 10 Elements
• How We Stack Up Against the Federal Government and the Private Sector

Introduction

The 2001 National Client Satisfaction Survey took place during the summer of 2001, just prior to the pivotal events of September 11. At the time, the Passport Office was in the midst of a momentous period of transition. The Office was rolling out its electronic passport issuing system known as IRIS® and, pending completion of the rollout, continued to use a dated, more labour-intensive issuing method known as CUPID. The 2001 Survey sample was stratified by data source to reflect the bimodal nature of passport issuance at that time.

Participants to the survey told us they value a competent and effective staff; they value a reliable passport that is recognized throughout the world; and, they value clear answers to questions in their preferred official language.

While some 96 percent of those interviewed indicated that overall they were either satisfied or very satisfied with the service they received, a sizeable majority, 65 percent, selected the lower rating. A complete synopsis of the national Survey results is provided below.

Methodology

The 2001 Client Satisfaction Survey was conducted by the firm Écho Sondage, a survey division of Circum Network Inc., from July 12 to August 8, 2001. Conducted through telephone interviews, the purpose of the survey was to determine the degree of satisfaction clients felt with Passport Office services. The Survey had a response rate of 62 percent.

Interviews were initiated with 1,074 individuals who had applied for a passport during a three-month period (February to April) just prior to the survey.

To ensure that each passport issuing office was represented in the final results, ex post facto weighting was used. The sample was also stratified for proportional weighting of the IRIS® and CUPID methods of processing applications, both in use when the Survey was conducted.

The questionnaire included core questions from the Common Measurement Tool (CMT) model, the de facto standard of measurement recommended by Treasury Board. The award-winning Citizen-Centered Service Network (CCSN) of the Canadian Centre for Management Development (CCMD), together with Treasury Board, produced the measurement standard. CMT is essentially a guidance tool to measure current levels of satisfaction, to find out what clients expect and to determine priorities for improvement. Questions asked in previous Passport Office client feedback surveys were also considered in the makeup of the survey.

The sampling error was estimated at plus or minus 3.2 percentage points.

Demographics of the Survey

Table 1: Demographics of the 2001 survey

• * Six percent of participants used express or urgent service
• ** 70 percent of participants were born in Canada

Client satisfaction

Basic service

While clients may highly value what they consider to be basic service, they also expect to be able to take that service for granted. In effect, clients will not experience an increase in satisfaction when we provide them with elements of service they perceive as basic. Elements of basic service cited as important by survey participants included:

• the availability of application forms
• the security features of the passport, and
• a travel document that is widely accepted by the authorities of other countries.

The importance attached to values

Approximately one in four Survey participants were asked to rate the importance they attach to 33 values. The importance for each value was rated on a scale of 10 with 0 being "very unimportant" and 10 being "very important." The values were then ranked in order of importance. Scores ranged from 8.3 for "competence of staff" to 6.2 for "the extra cost for express delivery."

The three highest on the list of declared values: competence of staff, effectiveness of staff, and acceptance of the document by other countries indicate that, first and foremost, clients value both the reliability of the application process and of the document itself. In effect, clients value both reliable service and a reliable product.

Clarity of written information, satisfactory answers to questions, and service provided in the client's preferred official language ranked next indicating that effective communications are also highly valued.

Generally, clients experience a high degree of satisfaction from sound processes that are reliable and an overall solid service relationship.

Table 2: Values Declared by Participants of the 2001 Survey

Specific aspects of the service transaction

Most of the 2001 Survey focussed on general aspects of the application process, specific aspects of the service transaction and the passport as a travel document.

Several Survey questions prompted participants to gauge the degree of satisfaction they experienced with the service transaction by selecting one of five possible categories, "very dissatisfied," "dissatisfied," "neutral," "satisfied" or "very satisfied."

Each question fell into one of six categories:

• the application process
• service responsiveness and reliability
• access to service
• communications with the Passport Office
• cost of the passport
• the passport as a travel document.

The results of all participants were then averaged on a scale of 0 to10 with 0 indicating "very dissatisfied" and 10 indicating "very satisfied." All of the elements in each of these charts are measured using the average satisfaction rating.

Chart No.1: The application process

Rank and Description	Average Score (0 to 10)
Simplicity of the application form	8.1
Ease of submitting your application	8.1
Fairness of the process	8.0
Overall requirements	7.9
Ease of picking up passport	7.9
List of qualified guarantors	7.8

Chart No.2: Service responsiveness and reliability

Rank and Description	Average Score (0 to 10)
Competence of the staff	8.4
Courtesy of the staff	8.4
Effectiveness of the staff	8.4
Availability of the application forms	8.2
Understanding of your particular needs	8.1
Time it took to produce the passport	8.0
Total time and effort required	7.6
Waiting time by mail	7.5
Waiting time at office	6.4

Chart No.3: Access to service

Rank and Description	Average Score (0 to 10)
Flexibility: Methods of access	7.9
Ease of access by telephone	7.3
Hours of operation	7.3
Signage to locate office	7.2
Convenience of office location	6.9
Availability of parking	5.9

Chart No.4: Access to service

Rank and Description	Average Score (0 to 10)
Communications in your preferred official language	8.1
Clear answeres to your questions	8.0
Clarity of written information	7.9
Ease of access information	7.8

Chart No.5: Cost of the passport

Rank and Description	Average Score (0 to 10)
Convenient methods of payment	7.9
Timing of the payment	7.7
Extra cost for express delivery	6.5
Base cost of the passport	6.1

Chart No.6: The passport as a travel document

Rank and Description	Average Score (0 to 10)
Passport as a travel document	8.1
Acceptance by other countries	8.0
Number of pages in passport	7.6
Security features of the passport	7.6
Design to confound forgery	7.2
Duration of validity period	6.8

The three indicators of satisfaction

Chart No.7: 2001 Overall Client Satisfaction Satisfaction Rating Score (%) Very satisfied 31 Satisfied 65 Neutral 4 Dissatisfied 1 Very dissatisfied 0

Participants were then asked to rate their general overall satisfaction. Three indicators of client satisfaction were used to measure overall satisfaction. The first, the average satisfaction rating was determined using a scale ranging from 0 "very dissatisfied" to 10 "very satisfied." The average score was 8.1.

The second indicator, the percentage of participants indicating satisfaction by choosing "satisfied" or "very satisfied," is a usual, but misleading, way to express satisfaction. Here, ninety-six percent of participants indicated they were "satisfied" or "very satisfied."

The third indicator, the percentage of participants selecting the top satisfaction category, is a truer measure of satisfaction in that it provides a solid measurement of performance excellence. Here, 31 percent of participants selected "very satisfied."

Overall, the three indicators paint a positive picture of client reaction to Passport Office services. However, the fact that 65 percent of those who claimed to be satisfied were not prepared to select "very satisfied" is significant. In Western society, selecting "very satisfied" suggests the participant has made a substantial commitment whereas selecting "satisfied" suggests considerably less of a commitment. Consequently, despite the satisfaction registered by such a high percentage of participants to the Survey, it may be concluded that something happened to mitigate the passport experience of an astonishing 68 percent of clients. If we consider exclusively those who expressed satisfaction, some 29.8 percent indicated "very satisfied" while 70.2 percent did not.

We have seen that the percentage of clients selecting the top satisfaction category, provides a fair reflection of our current state of performance excellence. [Chart No. 7] In this respect, the Passport Office appears to have fallen short of a desirable level of performance in 2001. The achievement of other excellence organizations suggests the Passport Office can reasonably expect to raise its current performance excellence level of 30 percent to 50 or even 60 percent. This translates to an increase of approximately 67 to 100 percent.

Leverages of satisfaction

The more importance clients attach to a given element of service, the stronger its influence on the satisfaction they experience. The stronger that influence, the greater the satisfaction they experience when that element of service is improved.

The seven elements with the strongest leverage of satisfaction in the 2001 Survey were the:

• effectiveness of staff
• waiting time to receive the passport by mail
• competence of staff
• time it took to produce a passport
• courtesy of staff
• understanding of particular needs
• total time and effort required.

By concentrating our improvement efforts on elements with strong satisfaction leverage, we can achieve more concrete results.

Satisfaction with the application transaction

Of the 27 application transaction elements measured, the three most satisfying dealt directly with the client-employee relationship:

• competence of staff
• courtesy of staff, and
• effectiveness of staff.

Chart No.8: Satisfaction with the application transaction - Top 10 Elements

Competence of staff	98%
	38%
Courtesy of staff	98%
	40%
Effectiveness of staff	98%
	38%
Availability of forms	96%
	34%
Service responsiveness and reliability	95%
	33%
Understanding of your needs	95%
	31%
Communications in the prefered official language	97%
	27%
General communications	96%
	27%
Time to produce the passport	92%
	31%
Adequate answers to question	96%
	22%
		0%	100%
% satisfied or very satisfied		% very satisfied

At the other end of the spectrum, the last five on the list [Chart No. 9] were less satisfying:

• convenience of office location
• extra cost for express delivery
• waiting time at the office
• base cost of the passport, and
• availability of parking close to office.

Here, we can apply the same performance excellence indicator used in the question on overall satisfaction [Chart No. 7 ] ? the percentage of participants who selected the top satisfaction category ? to each of the application transaction satisfaction elements. By monitoring and comparing the results of the "very satisfied" ratings in the current survey [Chart No. 8 and Chart No. 9 ] with those of future surveys, the Passport Office has a solid gauge for measuring increases (or decreases) in performance excellence. In effect, the "very satisfied" rating becomes a powerful barometer for measuring the effectiveness of our improvement initiatives.

Chart No.9: Satisfaction with the Application Transaction - Bottom 10 Elements

Waiting time to receive passport by mail	82%
	30%
Cost and payment options	88%
	15%
Ease of access by telephone	83%
	19%
Hours of operation	84%
	18%
Signage to locate office	82%
	18%
Convenience of office locations	58%
	14%
Extra cost for express delivery	71%
	19%
Waiting time at office	67%
	19%
Base cost of the passport	63%
	9%
Availability of parking close to the office	58%
	13%
		0%	100%
% satisfied or very satisfied		% very satisfied

Client expectations

The Survey probed participants for their expectations regarding the service offered by the Passport Office. Those who were critical of the wait they had experienced said they actually expected a shorter wait of 23 minutes on average. Interestingly, this compares with those who waited 22 minutes on average to submit the application in person and said they found the wait acceptable.

Similar comparisons for pick up times respectively yielded results of 13 minutes (the expected wait) and 12 minutes (the actual wait) and for production times 8 days (the expected wait) and 9 days (the actual wait). The differences between actual and expected are considered statistically insignificant. In other words, dissatisfied clients only expected what had been provided to satisfied clients.

How does the Passport Office stack up?

When compared with other federal departments and agencies, 39 percent of participants told us that the Passport Office provided better service whereas two percent said our service was not as good.

Twenty percent of participants rated Passport Office service better than that of the private sector while 11 percent rated it not as good.

Chart 10: How does the Passport Office stack up against the federal government and the private sector?

(%)	50
	0	39	2	20	11	We are better than... We are worse than...
		The Federal Government		The Private Sector

Our weaknesses

Participants expressed dissatisfaction with passport services provided by mail services and the service provided by the Central Region (JWC). Clients in the 18-35 years of age group as well as clients residing in British Columbia also registered a degree of dissatisfaction.

By comparing the results of the next national client satisfaction survey, to be conducted in 2003, with those of the current survey, we will have a barometer that determines whether or not we are going in the right direction. We will then be in a better position to start taking corrective measures as appropriate.

The 2001 survey also revealed that regional surveys have a roll to play in helping passport offices across the country better manage their own affairs. The first set of biennial cross-country regional surveys have just been conducted and results will be available in spring 2003. National and regional surveys, focus groups and other measurement tools all serve to guide us by showing more precisely where we need to take action.

Beginning in 2003, the National Client Satisfaction Survey will be conducted on an annual basis.

Priorities for improvement

Generally, two priorities for improvement emerged from the 2001 Survey: the waiting time at regional offices for applicants to submit application forms and, for clients who used the service, the extra cost for express delivery service.

A Passport Office action plan

The launch of the Passport Office client satisfaction measurement program in 2001 was guided by concrete Treasury Board objectives and characterized by a clearly-defined starting point. It also opened the door to a well-coordinated and solidly planned continuous improvement initiative.

In the 2001 Survey report, Écho Sondage indicated that a substantial percentage of participants had shifted their allegiance from "very satisfied" to "somewhat satisfied." Other than that, no other comparisons with previous surveys were attempted in the current report. The 2001 Survey has, however, provided the kick-off benchmark for comparing the results of future surveys. These, the first of a new series of client feedback and public service employee surveys, have provided the benchmarks needed for measuring improvement (or regression) in client satisfaction and the workplace conditions of the civil servants who are called upon to serve them.

Throughout the public service, analysis and comparison of the first two employee surveys is already under way. Here at the Passport Office, section teams are, or will soon be, gathering to discuss and compare the results of the 2002 Public Service Employee Survey with those of the 1999 survey. By listening to the workplace concerns of public service employees, the government is taking the first step to addressing a key element ? that of strong employee motivation ? an element that is indispensable for responding to the satisfaction needs and wishes of its clients.

In tandem with the analysis of the first two public servant surveys, the Passport Office will compare the results of the 2001 National Survey with those of the 2003 National Survey, which is about to get under way, and annually thereafter. Regularly scheduled client and employee surveys and other measurement tools will give us in-depth knowledge and understanding of our clients and the public employees who serve them. They will also provide guidance and suggest measures for increasing client satisfaction.