Customer Satisfaction Survey Grading

What grades do you use in customer satisfaction surveys? How are grades influenced by the culture of the country? What to do if you perform surveys in different countries with different background?

I have been discussing customer satisfaction here a few times. Related to that, let me share a brief anecdote with you:

We implemented a new Service Desk SW on a customer's site. They are a managed services company providing support all over the world, 24x7. After the resolution of every ticket, contact person on a ticket receives a mail notification with a link to a short web-based survey.

There were just a few questions regarding speed of resolution, communication, competence of support people and overall satisfaction with the ticket resolution.

Grades were 1-5, with the above explanation that 1=poor and 5=excellent.

We received quite a bunch of survey results at the beginning, which was the intention. Here and there, a low score was received, but we were not alarmed, you can't please everyone. Service Manager was in charge to treat all grades below 3 as a customer complaint and to follow up with customers to raise their satisfaction.

Then one day we received two very bad results, averaging below 2. Both from the same market. Alert! We are doing something wrong.

So Service Manager sent apologetic mail with a inquiry what went wrong and how can we improve, blah...
The answer came quickly, from both customers, saying they are sorry, but they thought 1 is better and 5 is poor.

Was something wrong with the survey code? Customers sent us their screenshots, everything is fine, the explanation at the form header clearly stated "1=poor and 5=excellent". Both customers were from Germany. On a request to explain how they misunderstood this clear instruction, they said: "German school grading system is 1 to 5, a one being the best grade (Sehr gut), and five the worst - insufficient (Nicht genügend)". So they didn't bother looking at explanations, they automatically presumed that this survey from another country complies to their long-term grading experience in German scholar system. Can't blame them.

Therefore we looked arround, what are grading standards in school systems around the world?

USA and influenced states use ABCDEF grades. Europe differs very much depending on history, somewhere 1 is bad, under the German skirt it is excellent (Chezh republic and Slovakia also).
Eastern European countries, Asia and Oceania use either Russian 1-5 system or percentage system 0-100%.

Interesting details:

• Venezuela uses rather exotic 0-20 grading system
• Ecuador and Serbia (opposite hemispheres) use similar 5-10 grading systems where 5=fail and 10=excellent.
• A lot of countries use different grading systems for primary, high school and university grades.

So what approach to take in grading customer satisfaction in order to make it intuitive regardless of their local education background?

 

We had only two solutions:

• Go to using 1, 2, 3, 4 grades, where customers won't be able to relate to their finer graded school system. 1-4 is also good because it eliminates the indifferent middle grade, and it forces a customer to decide for better or worse 2 or 3.
• Take the -2, -1, 0, 1, 2 grading approach. Good: it is self-explanatory, negative numbers can't be good. Bad: it leads the customer to a mediocre "0" grade, suggesting that it is OK.

For now, we are using the second proposed grading system, accepting the downside that some customers see nothing bad in selecting a "0", which is a neutral grade.

An additional tweak would be to change grades to -1, 0, 1, 2, which would "push" a customer towards positive grades some more.

What metod do you think is most apropriate for you?


Related articles:

Customer Satisfaction 
How bad do we need it in IT Service Management? Where is it mentioned and where is it dealt with in ITIL V3? How do we manage it in real life?

Customer Satisfaction Survey: What Methods To Use?
How to gather customer satisfaction data? What methods are there? What ITIL says? What methods will work for you?

Incident Priority - What Everyone Should Know

As ITIL defines it, Incident priority is primarily formed out of it's Impact and Urgency. There are also additional elements, like size, scope, complexity and resources required for resolution.

So, most consultants recommend the simple matrix which will automatically calculate incident priority out of the simple value of Impact x Urgency.

Recommended granulation of Priority is 4 to 5 different values.
Usually the lower the value - the higher the Priority, thus Priority=1 is the highest one, and Priority=5 the lowest.
There are also some sophisticated methods for defining Priority, but eventually it all boils down to something like:


Picture: Standard Priority Matrix

Impact of the incident is the measure of how business critical it is. Since this is difficult to determine in shoes of overworked and underpaid 1st level operator, some simplifications are necessary here. So impact is usually directly proportional to a number of users influenced by the incident. If an up-to date CMDB is available, then it's easy to determine affected users from the Business Service which suffers from specific Configuration Item malfunction.

Urgency is a necessary speed of resolving an incident. Some incident management tools perform automatic calculations for Urgency based on Impact, SLA and OLA involved etc. Nice feature if you have it. If not, a simple workaround would be to educate Service Desk operators on a regular basis and to inform them on parameters of signed contracts. Incident Urgency for certain Services may vary in time (example: HR application during payroll calculation) and this additional complexity is easier to resolve by raising staff awareness level then to implement it in software tools.

Major Incident is an incident with extreme impact to business, or an excessive disruption of service. It will have Priority=1, and additionally, depending on your SLA and support process definition, it usually has additional attribute (checkbox is fine) that says this is a Major or Hot incident. All key Support Staff members attend to resolution of major incidents, and the project is strongly supported by Problem Management.

Also, should be said here that Incidents do not age gracefully. Data on that reside in the escalation schemes, based on Service Level Agreement (SLA) targets. If an incident owner can't deal with that, his manager has to be notified in time.



Update: Incident Escalation

Just a few words on Incident escalations. Escalations are mechanisms that help us to resolve incidents on time. There are two major types:

Functional Escalation - reassigning incidents to a higher tier support group due to lack of expertise. Also, this can happen after a predefined time interval passes, in accordance with SLA.


Picture: Functional Escalation


Hierarchical Escalation - when a support employee can't deal with an incident, either due to lack of knowledge or insufficient time, his manager has to be informed in order to preserve SLA targets and customer satisfaction. In practice, this escalation type usually boils down to a simple notification to the manager.


Picture: Hierarchical Escalation

All About Incident Classification

Incident classification is among the main tasks of Service Desk 1st tier people. It adds structured data to a basically hectic unstructured series of info we get from a nervous Customer.

Aside from prioritisation, which is another important issue, there are three main reasons for classification of incidents:

• Assignment and escalation
• Problem analysis
• Reporting

First reason gives us a hint why a Customer shouldn't categorize an incident. He is not trained for that, he reports his problem (over the phone, web, mail...) and it is an Operators task to assign the incident to a proper category.

Reporting is important because it provides inputs to other ITSM processes (Problem, Change...) and helps us stay in good relations with the Customer and the Management.

Therefore, the Incident Management tool should allow assigned people to re-classify the incident (change it's category) along the escalation path. Because sometimes the problem shows to be somewhere else. I.e. it's not the printer but the network. Some of the top Incident Management tools I know are very rigid regarding this and I know some Service Desks which used to close existing incident and open another one with a proper classification in this case. Sad.

Activities

Now, what are the best practices telling us, how do we assign categories to incidents? ITIL does not prescribe a hard coded form of classification. Usually the chosen IM tool will guide us, no matter how much is it advertised as fully customizable to adapt to your business process. So one should have this on the list of important factors during a tool selection process.

Here are a few things we should do:

• First, everyone has a legacy incident database. Being new to ITIL IM doesn't mean that you didn't resolve tickets, incidents or issues before. Take a look at the structure of legacy data, and discuss what is useful for your business process, what hierarchy should you prefer, and what's missing.
• Consider the management and customer reporting needs, this should give you some ideas.
• Business should understand your categories, although they are usually technical.
• Key Performance Indicator list - should define what is important.
• Review your process definitions and your main competency matrix. Are you technology or business oriented? Try to define your category tree based on that. See your legacy incidents, try to assign defined categories to them and analyse. Is it any good?
• Your Services Catalog and your SLAs should tell you a lot. Even if you don't have them, you probably know what are the major business critical services: mail, printing, document management, accounting. The longer the list, the more probable that you'll want to put these in the root of the category tree.
• Talk to your Problem Management people, they should know a lot.
• Brainstorm a lot.
• Start an Incident Management Pilot project for a month, and use your classification system. Analyze. See what's right and what can be better. Redefine if necessary. It's better to try and err then not to try at all until major production.

Possible Problems

• 1st line Operator education
• Customer portals sometimes offer a classification tool to a customer during Incident reporting. If this is done, this should be on a very basic level, and always checked and redefined by a 1st level operator.
• Depending on business process definition and classification scheme, it will be difficult for a 1st level operator to differ between Incidents, RFCs and Service Requests.

Examples

Now, how does this look in practice? As I mentioned, a tool can influence your decision a lot. Peregrine Service Center gives you a structure:
Category:Subcategory:ProductType:ProblemType
It's a nice four-tiered schema.

Remedy gives you
Category:Type:Item
A three-tiered schema, very good technical schema.

HP OpenView Service Desk provides two categories in combination with type and service.
It's fair to mention that the above tools and most of the others allow some amount of customization, adding or removing categories, and dynamically requesting different depth of classification based on type or main category. So whatever tool you choose, there is a chance that you will be able to adapt it fairly to your process. It's just that some of them will require more time/money and customized solutions can create problems on upgrades to following versions.

Now, examples? Some standard three tiered schema examples:

• Software:Server:Exchange (Technical classification according to your org. chart - Department:Team:Assignment)
• Server:Software:Exchange (Technical classification By CI categorization)
• Printing:Printer:Cannot Print (Service oriented - Service:Component:Problem)
• E-mail:Outlook:Access (Service oriented)

Depending on business requests and technology, some creativity is allowed here. For one of my customers we created the following: they had in-house developed IM tool, and some specific needs, so we created a category tree of unrestricted depth, with technology-oriented categories. Every node (category) has its own assigned Queue and escalation schema. Their expertise level varied with technology so some tickets had category field like:

Network:Switch

and some were:

Software:Server:OperatingSystem:Microsoft:Windows2k3

This system works for two years now, and they are a happy customer. They have a bit more complex reporting then usual, but also the classification that fits their business process.

Remember that along with categorization, Incidents carry a lot more of structured data (Priority, CI Info, SLA...), so in most cases a nice three-tiered categorization combined with these data will cover all your needs.

Of course, if there is a need for more, then a four-tiered possible classification would look like:

Software:Server:IBM:Domino
or:
Email:Server:Domino:JunkMail Filter (Service:Component:ProductType:ProblemType)

After all mentioned, one does understand that there is no easy formula. Sometimes the best way to understand things is to try and make mistakes. I hope that the above concerns will at least help you make less of them.

Service Management

Have been thinking about service management a lot lately. Since the core bussines of my company is SM, we are educated in most of the known methodologies and implementing most of the best practices from ITIL, MOF, COBIT, ISO...
Blogging about it could be a good way of setting some basic knowledge points for the new people in the business. I will take the liberty of reshaping and adding to previous posts according to comments and sugestions of the People.