There is a beautiful hypothesis that of all the touch-points a business has with its customers there are just a small handful that are Moment’s of Truth, essentially interactions that make or break the relationship (and in turn any future profits, customer loyalty, etc).
For example in case of your credit card company the moment of truth is when you call them to report your card lost. You are desperate and probably freaked out about charges someone could be making, and when you press 4 (or whatever) and the first question out of the operators mouth is: What is your credit card number? Or something equally silly. You just pressed the option to report a lost credit card, they should know better.
They probably just lost a customer, they definitely lost any good will and positive word of mouth that they had with you thus far.
For most online businesses a critical moment of truth with their customers is when the customer has a problem and they come to the company website for help. What you do from then on determines any future relationship you might have with the customer.
This post focuses on something all of us in “web analytics” struggle with: how to measure success of a Support website in delivering an optimal experience to customers in that moment of truth.
Most of the tools we have, books that have been published and a the material we have access to focuses on ecommerce / selling. Support is a unique challenge and we struggle with it because “success” is harder to measure, success is not a page view or submit order button.
Here is my little contribution, a point of view if you will, to the discussion:
# 5 Don’t measure Unique or Total Visitors.
This is a bit extreme but on a support site we are not in the “demand generation” business, why should visitor counts be any part of success measurement? Your company makes crappy software / hardware and more people will come. Many corporate support websites are hard to find, so fewer people will come.
Not measuring Visitors is not just an advice to save time but it is more about changing mindsets. You don’t care how many visitors come to your website. What you care is how many of those your helped, for how many of them you delivered a positive moment of truth experience. We have to get this in our gut and ensure that everyone around us gets it as well.
We throw an expensive web analytics tool on a support website and we run the standard reports. Most standard analytics reports are quite sub-optimal for a support website and need to be thrown away, below I’ll talk about some that IMHO are helpful.
# 4 Identify Top Methods to Find Information.
Most support website owners toil away in vain spending time with path analysis (full disclosure, I am not a fan of traditional path analysis, for reasons click here). Most support sites sit on top of complex knowledge base systems and if you have bad navigation, or “ultra cool” navigation, and path analysis is instantly sub optimal because you influence the path so much.
My post on Path Analysis shows one way of how we can identify how people find relevant information on your website. The picture on the right is from that post.
We really need to understand what options our customers are using: Internal Search Engine, Top FAQ Pages List, Product Downloads list, links from forums, etc etc. This is important because, as is usual on support websites, if most of your support content can only be found using your internal search engine and only 10% of the site traffic is using that search engine, it is guaranteed that your customers will be deeply unhappy.
Your company’s analytics tool might have other strategies that you can use to identify top methods that customers are using to find critical content on your website. If you are not satisfied with the methods your customers are using it is time to start testing and experimentation to try other methods.
# 3 Click Density Analysis of Top 10 – 20 FAQ’s (solutions / answers)
I use the term Click Density rather than Site Overlay because to me it communicates more effectively the purpose: what links have the highest density of clicks.
You probably know what the top problems are that your company is having. Either from your internal sources or simply from the top solutions to your product problems being viewed on your website (from your analytics package). Perform click density analysis on just the top 10 or 20 of those pages and identify if people are taking the action on that solutions page that you expect them to take.
[I am recommending top 10 or 20 both to avoid paralysis by analysis and also because on most support websites given the nature of the business, bugs tend to be concentrated, just a few pages will get 90% of the traffic. YMMV.]
If to fix the problem you want them to download a patch, are they doing that. If there are links on that page that take them to different solutions, are the customers able to find those links and are they clicking on them. What is the % of clicks on links below the fold (and are any of those links important).
If we find that customers are not doing what we expect them to do on the solutions page it is time to rethink the content, layout etc to make it more useful.
# 2 What percent of Site Visitors Call the Support Phone Number.
{start bitching} Most support websites do their best to hide the tech phone number deep in the site, clearly these companies don’t get the concept of Customer Satisfaction and Moment of Truth. In my book the only exception to this can be a company like Amazon who has excellent email support, they reply to support emails in less than 24 hours, very few fortune 500 companies have that record. {/start bitching}
Put a unique 800 phone number on your support website and measure the number of phone calls to that distinct phone number. My recommendation is to measure that call volume over time and if you are driving improvements to the website experience the phone calls should go down (as a % of traffic, ok so you should measure traffic for this one purpose : )).
Also measure successful resolutions in the phone channel for the callers who came from the website. The hypothesis is that even if the website was not able to eliminate the call perhaps it was good enough to provide the customers with the basic information that would make for a shorter phone call (and in turn some cost savings to your company).
# 1 Measure Problem Resolution, Timeliness & Likelihood to Recommend.
Most web analytics packages will be quite limited in their ability to provide insights on a support websites. That’s because the What is of such little importance, it is the Why that is of extreme importance (for background on these two concepts click here).
Implement a some kind of continuous measurement system for your website. You could do constant lab usability studies or follow me homes or surveys. The last one is something I recommend (ForeSee or iPerceptions). Using any of those qualitative methodologies I recommend measuring three metrics:
- Problem Resolution Rate: What % of respondents were successfully able to resolve the problem they had?
- Timeliness: What % were satisfied with the time it took them to resolve their problem?
- Likelihood to Recommend: What % are so satisfied that they are likely to recommend your support website to others.
These three metrics measure the core essence of if you are delivering against the customer expectations for a moment of truth. You will be humbled by the first results but of all the recommendations above this one will drive the deepest insights and the work that will actually have a positive impact on customer experience.
(There might be a temptation that you could use clickstream data to get the first two bullets, there might be a detailed post in the future but I assure you that will be fundamentally flawed and only a actual customer qualitative response will suffice.)
There is a tendency not to trust surveys. In our experience survey response rate is on average seven percent (vs internet standard survey response rate of one percent), more than enough to get statistically significant results. That is both a reflection of the quality of the survey and of when and to whom the survey is served up (using behavior targeting). You can complement surveys with other methodologies described above, Follow Me Homes or Lab Usability etc.
Of course one final bonus recommendation for you…..
# 0 Correlate Your Top Solutions Viewed with Top Problems Identified “Off-site”
Ask for the top ten product problems being reported in your phone call center and check if solutions to those problems are the top ten pages on your site. It is possible that customers call and come to the website for different problems, but if the solutions to the top ten call center problems are not top on your website it is highly likely that your site is failing your customers and you need to take immediate action.
Do the same for top problems people are talking about in User Forums (on your site or others) and in the great blogosphere.
Doing analytics on a support website is extremely difficult because we are dealing with irrationality. People are stressed, our websites are not optimal and there is no patience in getting answers. Standard Analytics tools can only go so far. I deeply recommend that 70% of your analysis should be qualitative (as the question: “If you were not able to solve your problem on our site, what was your problem?”) and 30% clickstream (quantitative).
Agree with the recommendations above? Disagree? Are there alternative strategies that have worked for you better? Please share your feedback via comments.
[Like this post? For more posts like this please click here.]
Via 
I had never heard of the "moment of truth" concept, but as I read your post it sounds like a very interesting way of thinking about customer interactions.
Thanks for this great post, you provide specific tips on how to measure a support site and each recommendation is awesome.
I totally agree with you. When it comes to support, attitudinal analysis should have the last word (disclosure: we are partners of iPerceptions). This is one area where making sure the total "experience" was satisfying is of the upmost importance. Just relying on the behavioral dimensions (disclosure: we are partners of WebTrends), where you would, for example, analyze the "problem resolution process" and calculate the percentage of visits getting to the "problem solved" page, would definitely not tell you if
a) that was easy for the customer to go through,
b )satisfying at all (you can accomplish any process while hoping you could grab a company representative by the throat), and
c) make it a retention element of your relationship with the customers.
As you said, visitors who have to use support are in a critical state of mind, and we need to understand how that state of mind was and how it evolved through the support process/experience. You definitely need attitudinal analysis here.
Hi Avinash,
Great post, very complete, I really enjoyed reading that!
I would however just want to add 2 things and would appreciate your opinion.
The first is some kind of lexicon or translation table for the searches entered on the website. This is from my own experience as a non-technical woman: I tend to call things differently and often get lost in support sites because I can't remember to call what I'm looking for by their right names.
So, Chief Web Analytics Officers (just read this funny post on Eric P.'s blog) should encourage their analysts not only to look at the search words entered but also make up some kind of "matching list". And when someone types in one of these words, that is not an official word but a matched word, it should show up in the results as "the information is not found but our experience shows us that you might be looking for so and so".
And this brings me to my next suggestion: add a "has this solved your problem" box. We all know that it will only be used when people are, as you say, stressed but it could turn out to be usefull. Add this to your top 10 problems: the top 10 non solvers and dig further into that.
Anyhow, let me know what you think, I always greatly appreciate your insight.
Kind regards,
Aurélie
Aurélie: Absolutely right on both counts. The first one is commonly referred to as "synonyms" in our world. Though the concept goes way beyond just suggesting alternative words.
Two examples of how this is one on our sites. This is on a ecommerce site and I searched for totally random something and it tells me "Customers who searched for xxx yyy found results at:" so it tries to guess what I am looking:
And this from a support site where I entered something completely wrong error code 123 (which does not exist) and the search tries to guess and help:
Couple of small examples of what you are saying. The best thing for support sites is to simply watch customers and ask and get out of the "engineering mindset" in creating them.
Your second suggestion is also on the money. I am not a fan of "page level surveys" in most cases (they do a terrible job of representing site experience). But for a support site I think it is a great way to learn (and I am very happy if only stressed people reply becuase our job is to solve problems of our stressed customers in their "moment of truth").
Here are two small examples of that:
Thanks for your comments, it is always nice to hear from my friends in Europe.
I rest my case, I'm impressed.
Keep up the good work!
Have a nice evening,
Aurélie
Pretty interesting post! I found I needed to be in the right state to read it all the way though and focus – and when I did that, everything your saying feels correct.
IBM has a support site (eSupport) and the way your approaching metrics could be helpful (it's not a project I'm working on today, however).
If you think about it – how many have the opportunity to really work on a support site like the type mentioned?
I love the ideas you've presented for measuring how we are living up to meeting customer expectations during "moments of truth" in the support experience. Particularly since I'm in 100% agreement with your bitching — I have yet to meet a support site that I actually would recommend to anyone. And (then I'll move on) I think that support on the web is the greatest area of opportunity for developing net promoters. If I have a great support experience, that is something that has a MUCH higher impact on my willingness to recommend the product or service to someone than anything else within the product lifecycle.
The only point I'd make is that moments of truth can happen anywhere in a person's web experience and at any time throughout the cycle we use to think about how the customer progresses through the product lifecycle (Research, Shop, Buy, Use). I definitely agree that the majority of the moments of truth are related to that "use" piece of the equation — I lost my credit card, I tried to pay for something and my credit card was declined — and those are probably the ones that often make the deepest impression on us, but they're not the only ones.
I just want to make sure that the whole concept of "moments of truth" isn't equated solely with support.
Avinash,
I have really enjoyed reading and listening to you.
In regard to measuring visitors, you may want to measure the ratio of support site to support calls — hopefully your site is handling the bulk of the work, even if your latest release was "buggy." This is especially helpful if you are trying to move customers online — you can measure adoption.
I like the measuring problem resolution ideas — if a customer wont recommend your site, they will probably not buy from you next time. Another factor of timeliness is the amount of solutions viewed. In an ideal support situation, the first solution solves your problem. i.e. Enter site, search, pick one result, the result solution fixes your problem, and you are done.
Which leads me to another approach to consider… I beleive the last page viewed is very indicative to how your customer did on your support site. If the last page viewed was a search result page, they likely failed. Likewise, if the last page they viewed was a solution, it is likely they solved their problem. If you compare this to your survey data, you will likely find that your most successful and satisfied customers viewed a solution just before leaving your site.
Of course this will break down for those users who looked at many solutions. (Again, compare to your survey data before drawing your own conclusion.) Those who last viewed a solution and reported success on your site likely did not look at more than 2 or 3 solutions. For viewers who viewed 4 or more, they likely just gave up, just like the person on the search result page.
So your successful visitors looked at no more than 3 solutions and the last page viewed was a solution page.
Hi Avinash,
Very nice insights you sharing.For myself,I'm working on the support section for my company,and I'm thinking about the possiblities of doing a segmentation testing through creation of 3 buttons namely for the first time user, repeat user and the non-IT savy users, in-comparison with the existing call to action buttons and search found the support section, in determining whether the customers are getting their questions solved.
What's your view or take on the test ?
cheers,
deric Loh
Thanks Avinash. super blog post as always.
One area I would like to add here is the richness of studying individual customers' (support) web visits and their call center interactions together ( ie..after the support site failed them and they had to call in) .
There are no analytical tools for this..you have to go case by case and customer by customer, but it tells you a lot about "why" the site failed them, and how to thus re-purpose your content, structure, search engine metatags, etc etc. I dont know how many people do this
I have also noticed that most "eSupport" orgs don't use a fraction of the tools used by their "eCommerce" brethren ( SEO,interactive content/MVT/ advanced analytics etc) and I'm wondering why that is the case. I would be obliged if you can point me, in your opinion, to a leading or best of breed ( in tech preferably) support web site? thanks again.
I agree with your higlighting off offline metrics. It's easy to not think outside your channel (especially if you business has a phone centre, retail locations, self-service machines, etc…).
If a customer can't solve their problem online, they'll either persevere by e-mailing, calling, coming to see you, etc… or just give up and leave. Either scenario is bad for you in both the short and long term!
The perception that I get is that despite a proliferation of new low cost channels becoming available to the customer (web, click-to-call, click-to-chat, etc…), companies seem to be providing worse and worse customer service. The new technology has made customers more and more anonymous (think back to the level of service most of us can still remember from those mom and pop store).
I was speaking to the owner of a web consultancy we work with who commented on a support issue he had with an internet provider. He had recently received such poor customer service from other companies, that he was ecstatic that the company provided what I would call an expected level of service (customer visits website, calls support line and issue resolved that day vs. multiple calls, messages, letters, visits, etc…).
Being successful during these moments of truth is not only good in the long run, it can also boost your revenues in the short run. We started to provide an online chat functionality towards the end of 2006 to our online banking customers. The focus was on support, however, the possibility of selling basic products was included. In 2007 we were able to monetise this support function to yield USD 9 per customer interaction on average (this added 7% to our sales revenue, paying for the team and turning a profit vs. just being a cost-centre).
Note: You must be cautious with trying to sell of course – there is nothing that will infuriate the customer more than their problem not being resolved and you trying to ram another product down their throats (hence the important of tracking your effectiveness at problem resolution first and foremost)! However, if you resolve the customer's issue, consider an opportunity to monetise on the goodwill that's been generated.
Hi Avinash,
Great Share… You have cleared all my doubts related to Google Analytics.
I think the first point is very clearly describing. I am always a little bit confused about unique and total visitors.
Thank you so much for sharing such an informative post with us…
The power of SEO and good analytics.
This is my experience. I work like webmaster of a international consumer electronics brand. When I arrived at company, all the help contents articles were into a self developed system. It displayed articles related to customer searches in the website search engine. This engine has lots of problems due to a bad logic and other troubles. As you are thinking, customers were not finding those help articles.
I changed totally the focus, using a marketing tool and creating help articles with a well designed SEO. Adding them a survey, to receive customer insights. Talking with our Call Center to discover the most frequent customer troubles.
It worked like magic. Suddenly we start to receive vistis, and visits, and visits, lot of them. The brand reputation in google help us a lot to get a good SEO. Customers began to find what they were looking for. We did a continuous work, receiving customer surveys and improving the articles and adding more and more.
With a small number of well designed articles, we receive 20 times more visits than the old company system. And from that moment until now, visits has been always growing.
Your article has remembered me this project. Your conclusions are very similar to ours. I'm happy to coincide with you.
Regards!
I think that a good web survey is critical for measuring success in a support website and decisive if your customer uses the online services and do not call to your Call Center.
It offers valuable customer insights and process satisfaction/success.
Think about:
– Time to display: do not offer the survey too soon or too late. The best moment is when the customer ends his process journey: submits a repair, submits an e-mail, ends a chat session, register an account, register a product… At this point customer is able to send you valuable insights. Before, he/she can't do it.
– Implementation: there are lots of good survey vendors with great solutions. Use them. They offer visual and friendly surveys, with stars, dynamic options, etc. Do not ask too many cuestions to your customers. Maybe two. One to get the satisfaction and other to write comments.
– Source: you need to know which process has created the survey. Also, it is a good point to know what product/service is related.
– Join the survey results with a good process analysis (sources of entrance, achievement…) and you will get the point for all your customer areas.
Best regards colleagues.