You got questions, we've got answers! – Radio Shack's great marketing slogan.
Is it enough to move from totally faith based banner ads buys to using conversion rate for identifying the best performing ones?
A reader asked that wonderful question the other day and I thought I'll share the answer with you all. Here's more context to the question….
This company was purchasing banner ads, and some other media, to create "brand awareness" (quotes from them! :)). On his own fruition our Hero was trying to optimize the campaigns.
His question was:
I have started measuring Conversion Rate for all the ad formats and traffic sources to identify which send the most qualified traffic. My formula is clicks on ads divided by leads. (AK: This was a non-ecommerce website so a lead would be 
download of pdf or requesting test drive or submitting a form or …)
My conversion analysis proves that sites in our vertical (AK: let's say it was a Travel site or Automotive or Expensive Mushrooms). My boss is ignoring me and wants me to run campaigns on all kinds of "general" sites (portals, financial sites, news websites, and other unrelated sites) because these are "branding" campaigns!
What do you think? Am I wrong? Is my boss wrong?
Nice!
You can see why I wanted to share the answer.
My first act was to congratulate our Hero. If you all have learned one thing it is that I am not a big fan of unmeasurable things like "brand awareness"! Fuzzy, undefined, not tied to outcomes things give me ulcers. For more context and rationale here's a post you might like:
Now I am sucker for brands just like the next guy (Sony, Google, Coke, Lenovo each have a piece of my heart!). But based on performance and value, i.e. Outcomes – think Web Analytics Trinity.
Brand awareness is a great quest, but even it has outcomes you desire (positive impression of the brand, unaided recall when asked, likelihood to buy in the future, likelihood to recommend etc etc). Solve for brand awareness but please do the hard work of identifying what quantitative or qualitative outcomes you desire.
In the case of our reader (Hero from this point onwards! :)), I was absolutely thrilled that he was trying to measure based on outcomes, in his case Conversion Rate as defined by actions taken on the site.
This guaranteed that the source websites that sent most bottom-line impacting traffic would win, and our Hero's company would make money.
Much better than faith based analytics.
But since our Hero's website did not directly sell anything, and was for a expensive product, my thought went to this picture that is in my book (Web Analytics: An Hour A Day) . . . .
It is always great to know what is happening in the conversion bucket (and remember every site has conversion events, even those that are non-ecommerce). You get control of your destiny. You can prove to your company that your website is an asset. You can impact the bottomline.
But no matter who you are and how hard you try your overall website conversion rate will be around 2% (and a bit higher for your Direct Marketing campaigns).
This means for approximately 98% of the traffic to your site something happened and you don't have an account of that. Perhaps they hated the site. Perhaps they all (unlikely) wanted to buy but your site was broken. Perhaps they are going to rush to the retail store and buy. Perhaps they were all there looking for jobs. Perhaps they were so impressed with your site they are going to tell everyone else. Perhaps….. perhaps…… perhaps…..
The biggest Hero amongst you will be the one who understands
1) why the 98% were there and 2) what was the outcome of their experience on your site.
With that in mind my advice to our Hero was:
You don't sell your expensive products and services on your site. Just measuring conversion rate for secondary actions is good but you might be missing a big piece of success. People could buy from your store / factory / dealership purely influenced by your site, but your conversion rate will not measure it.
So . . .
Consider doing a on-exit survey on your website survey that asks people for their impression of the site or if they found what they were looking for or if their likelihood to buy a boat / submarine / plane is increased because of visiting the site etc. Consider these:
Now qualitatively you can measure two more success metrics: Likelihood that a Visitor will visit your submarine / plane showroom (without requesting anything online) and likelihood that someone will recommend your site / products to others because of a visit to your site.
Combine those two metrics with your direct online conversion and you are set to more from Hero to Superhero!
Taking this recommendation a full circle . . . . .
With a small bit of sophistication you can tie the respondents to the survey (anonymously) with their source website, in this case your affiliates.
Now you are all set to analyze if some websites are sending you traffic that is more satisfied or has a higher likelihood to buy / visit the dealership. In this simple way you can add more sophistication to measurement of conversion and get a complete picture of success.
In the end where our Hero, or you, advertise should not depend on a gut feel or a vague "metric" or simply a deep desire. Figure out how to measure success as completely as possible and let that tell you. Maybe it is vertical sites or maybe it is generic sites or maybe it is portals.
Maybe, just maybe, your boss is right. [Though given you are reading the blog there is a higher chance that you are right! Sorry, could not resist!! :)]
Go for more than conversion. Good luck!!
Ok now its your turn.
Please share your perspectives, critique, bouquets and brickbats via comments. Please share your success stories.
[Like this post? For more posts like this please click here, if it might be of interest please check out my book: Web Analytics: An Hour A Day.]
PS:
Couple other related posts:
Via 
A quick comment on the 98% and that would be to make sure you're not spending too much time on that group.
I've seen people go crazy worrying about the 98% when, in fact, there's probably 20% of that 98% (so 19.6% of total traffic) that's worth spending real time with to eek out a few more responses.
I'll date myself by going back to the "good old days" of direct mail. We used to spend a lot of time moving response rate from 2% to 2.1%, or getting just 5% more responses. That translated to reducing cost per acquisition by 4.76% and gave you 5% more revenue and a compounding impact on profit due to the lower CPA on the customers.
We used to write off the remainder of non-responders. Yeah, we'd survey them periodically and test follow-up mailings, different offers and the like, but we tended to find that the reason for non-response was "wrong time". They'd open the mail (or in this case, click on the ad) and just decide "maybe later."
To wrap–don't worry about the WHOLE 98%. Get the 20% on the edges and use your "hour a day" on that group! Otherwise, that hour will turn to five and you'll be posting blog posts at 3:51 am!
I completely agree. We are often asked about conversion rates, and how these should be tracked back.
Typically we provide overall site conversions as well as specific channel conversions, and indeed product conversion rates.
However, this type of reporting also need to be shown in context, as a single visitor may respond to multiple marketing channels over time. If you treate each channel in isolation, then you will typically find that your conversions do not add up – think of someone responding to an email, but later using a search engine to find the site and make a booking. Most site analytics will use either the first or, more typically, the last event that drove the conversion. However if you to look at these in isolation (as a email or SEO agency), then both channels would stake a claim to the conversion (assuming they were both within the usual 30 days).
It is therefore an important part of the mix to ensure that you understand how you are analysing your marketing!
Increasingly, we are being asked to report on the contribution of various marketing channels to the overall conversions, and frequently find that over a 30 day period the majority of visitors have responded to at least two different channels – typically PPC and SEO.
This then come back to tying to build a full picture of your site visitors over time, rather than on a visit by visit basis. You can then report on effectiveness of marketing channels when compared to a visitors overall lifetime spend.
Site conversion rates of 2% is something that i believe people should be more passionate about – imagine a 2% conversion rate at a supermarket! That said, you're absolutely correct, that the focus must be on better understanding of your sites visitors and thus your customers. Integration of qualitative data, and true visitor segmentation can provide valuable insight, which although it can be time consuming, will provide true ROI, and at the end of the day a better experience for the customer.
Sean.
Great post, as it really hones in on the crux of the matter. Time and again, we see the same deep-rooted, endemic belief in our clients that 9/10ths of their attention needs to be devoted to the share of visitors clicking on "buy now."
They lose sight of the fact that for every 1 visitors who converts, there may be 6 visitors onsite for product documentation, who hated the fact that they found nothing sufficiently detailed to answer their questions, and who now intend to tell everyone they know to avoid that site like the plague.
The detrimental impact of the latter cohort should be of far greater significance to decision makers than the one visitor completing a $30 transaction.
Great post.
I've only been in web analytics for 6 months, but it seems pretty obvious to me that you would at least want to make sure that you were raising awareness and establishing branding more efficiently, no matter what the reason for the campaign.
I find it impossible to believe that anyone would would willing ignore that data.
Perhaps the real issue isn't the boss's argument , but the boss feels upset that his lackey (the hero of this post) thought of it first and he doesn't want to admit that his underling was right? If this were the case, my guess would be the hero was super excited that he found this information. And, in his excitement, didn't deliver it in a way that was sensitive to his boss's ego (assuming there is a way to communicate where he would listen) which lead to his seemingly illogical viewpoint.
I know I've seen this sort of thing in the technical support field a time or three.
Hey Avinash, thanks for posting this issue. I believe that measuring those faith based banner campaigns and having a proper metric is unbelievably important to have. Besides what good is it if it can't be measured and quantified ;)?
So what if your client *gasp* doesn't have the budget for a follow-up survey from the site. Maybe this is a mute point and long been discussed in the annuals of analytics – but are there other ways you can best measure the almighty branding campaign?
Hey Avinash –
I've thought about this issue a lot, and am not sure I've come up with a good solution as of yet.
My problem arises from the fact I work on movie websites. As you can imagine, the overall conversion is getting people in the theater. But since we don't actually sell tickets on the site (and don't direct people to ticketing sites until the week of release), defining on-site goals is more difficult.
I've defined goals on our sites in the past focusing on a user watching trailers – essentially it revolves around the consumption of our content. However, I do not have qualitative data from our customers saying this is what they came to the site for (or that it increases ticket buying).
I know the best option is to implement qualitative analysis (currently looking to do that with either iPerceptions or OpinionLab) and just ask our customers. However, that's not going to happen right away (time/money); so in the mean time I'm wondering what to use as a conversion metric.
For the time being, I'm likely going to continue use content consumption as a metric (until I can track ticketing), but would love any feedback on other options, general thoughts on the situation.
Very insightful post Avinash. Even I work for the direct ecommerce site, I learned that there's always something more than "Numeric Conversion Rate" to reflect real customer conversion factors. In this regard, recently we tried usability tests with diverse customer segments and learned a lot of stuff we didn't know. I couldn't agree more with you. Great job Avinash!
Jonghee
As usual, I 98% agree with you Avinash, and also balk at grasping after vague notions, but I think there's a scenario where the less targeted market is worth going after.
Let's say that you have great performance on sites in your vertical. Just for kicks, we'll say 10% conversion. What if the opportunities to buy in that vertical are very small, though, and only encompass a portion of the available budget? You might still want to advertise on larger, lower-performing sites (let's say 1% CR), to get a bigger audience. Even if those sites have lower conversion, as long as the ROI was still there, the ad spend would be justifiable.
I've seen this a lot in long-tail targeting. Some of my clients get great conversion on specific niches, but even with a max buy in that niche, the total traffic is fairly low. Put simply, there just aren't enough people in that niche to target.
Mark : Excellent point! My recommendation, and I think it gels with yours, is to become informed about the 98% (and hopefully not obsessed!)
!!
Addition of the qualitative "likelihood" metrics to the conversion rate will give you a better complete picture of success. Then looking at the first two questions (Primary Purpose and Task Completion) will help you judge the value and quality of the 98%.
A while back I had talked about determining the true opportunity pie. That helps ensure you make intelligent choices about which segment to focus on! Post:
Measure the Real Conversion Rate & “Opportunity Pie”.
P S : Here's to late nighters, we are changing the world. Cheers!!! :)
Sean : You are absolutely right about multiple "touch points". Most Purchasers on websites exhibit multi session experiences. This is why reporting of "assists" is so important – the last or the first touch point (campaign) does not deserve all the "credit" (attribution). Unfortunately computing "assists" is much more complex than it might seem on the surface.
Our good friend Ian Thomas has hinted/promised that Microsoft Live Analytics (Gatineau) will report on "assists" and I for one can't wait!
[You are too polite to say this but I'll say that I was so impressed with the Web Abacus Product Brochure – short to making me coffee it can do anything I would ever want! Should we give Ian some credit here as well (!!)? :) Here's a link for everyone else: http://www.webabacus.com/resources/WebAbacus_Brochure.pdf]
Jeremy : The interesting thing is that doing a survey does not have to be very expensive and it is only going to get cheaper – especially if you just want to do the type of survey that I am recommending. So stay tuned!
The other ways I have seen used. . . . .
+ Many large companies will do primary market research (good old traditional market research!) and so say twice a year they get a great understanding of their overall campaign effectiveness.
+ Circuit City does this Bizrate point of sale survey thing on their store receipts, and has questions about influences from time to time and that is another way.
+ I have also gotten order confirmation emails from companies that have a quick survey *in the email itself*, two or three questions, that ask about influences ("brand campaign effectiveness") that is a great way to know atleast for purchasers.
Those are top of mind. I am sure readers will add some to the blog.
Latham : You are on the right track with measuring content consumption! See some of the suggestions for Jeremy above, perhaps you'll find inspiration there.
Both iPerceptions and OpinionLab use surveys but each solve for completely different things. Site Level surveys solve for different things than Page Level surveys. For more on that please see point #3 in Part One in this post, I think you might find it helpful:
Eight Tips For Choosing An Online Survey Provider.
Dr. Pete : Your comment has pointed out a excellent nuance to be aware of.
My hope, at the end of the post, was that by using the ideas expressed readers will be able to make better decisions about sources of valuable traffic. That compared to faith based or simply conversion based.
Once the framework is in place I am sure the next step will be exactly what you recommend: How many Visits we can get from each of our optimal sites, then where are the next best, then what are our bottom of our barrel options. :) With each step Holistic Outcomes will be smaller, but based on data!
Thanks for all the delightful comments everyone!!
-Avinash.
Hi Avinash,
I'm really into posts about conversion rate and must admit I'm not sure if Ive completely understood your point why it's not worth obsessing over, yet. In the SEO world everybody thinks conversion rate/ROI are the two most important metrics.
Your example in this case is very clear to me: A conversion rate of secondary actions (what % of people filled out a form or something similar) could be a rather bad predictor for the actual conversion rate in stores.
However, if our hero was selling e-books or another product and doing so purely online – would conversion rate not be *the metric* to measure?
I just realized I might have given myself the reply to my question: You usually say 'stop obsessing over conversion rate' and I said in SEO everybody thinks conversion rate is the one metric you have to measure..without measuring anything else.
Is your point that it is a very critical metric, but that simply measuring it (even segmented)is not enough, because looking at other metrics (how satisfied people are with your site, etc.) and improving them will lead to a higher conversion rate? And that obsessing over conversion rate, conversion rate, conversion rate is a bad idea because then other things will be neglected (that are needed to improve 'conversion rate')?
I hope that made any sense!
Patrick : Let me clarify.
Measuring Conversion Rate is good, much better than trying to make decisions faith.
In addition to that focusing on the red part (non converted), in the graph above, using instruments such as surveys is even better. You'll understand much better why people don't convert, why they were there, and, for our Hero, get a feel for offline conversions.
It is tough work to analyze beyond just on-site conversions, but it can be a fun, rewarding and a near orgasmic experience! :)
-Avinash.
Hi Avinash,
Given the obsession of most organizations with Conversion Rate and the Homepage, what advice or comments would you have for or against linking overall site CR or Sales with the performance of the site Homepage?
So your point isn't that measuring and increasing conversion rate isn't important.
But your point is that marketers shouldn't just focus on those people who do convert (which is measured by conversion rate), but that they should look into why the other 98% or so do NOT convert (by finding out through surveys, etc.)..which will ultimately help them increase conversion rate?
in other words: Analysis > reporting?:-)
Hi Avinash
Very good post, remarkable as usual, but it is very difficult to measure what activity did the customers do who did not convert.
Avinash –
First off, great article. I have been a long time lurker and felt it was time to speak up.
In your book, website (and I am sure some t-shirts you wear) you always talk about implementing on-exit surveys. We fully understand the importance of these survey results and the actionable items they generate. However, the looooong standing question is …
"How do you implement exit surveys?"
If you could share specific solutions or methods it would be great!
Thanks again, keep up the good work!
BryGuy : Thanks for speaking up!
I am afraid the method that I have used in the past is property of a company and I can't copy paste the code here on how to do onexit surveys. But let me try to share something of value. . . .
If you want to just capture people who exit on a particular page (say just your cart page) then I have seen this classic method work successfully….
[Sorry for the image, I have not figured out how to paste html without wordpress rendering it!]
Another method that seemed cool but probably has "something scary that someone technical should review carefully first" was on objectionhandler . com.
The page is heavy on sales and trying to sell the script that does the survey, you can ignore the sales – try to move your mouse as if you are trying to click the back button etc and you'll see the survey. Perhaps there is something worth learning there.
Important: I am not endorsing the script author or the user experience of the onexit you'll see the site.
Finally the most decent on-exit survey implementation methodology is the one used by companies such as iPerceptions.
It is permission based – hence has a very high survey participation and completion rates.
Go to http://www.iperceptions.com/ and you will likely see a dhtml overlay asking you for your permission to participate in a survey. If you say yes then at the end of the visit there is a survey.
I can't disclose the code that does that, but feel free to reach out to iPerceptions! :)
ForeSee, another survey provider, normally does a pop-up with a survey to a randomly selected % of website browsers. But they also mention that they can do onexit if asked – you can call them as well.
There are some survey companies that say they do onexit but in reality they pop up a "minimized to your PC status bar" window. This "minimized" pop up window is monitoring the Visitor session and will pop a survey on exit (since it is essentially "monitoring" the visitor).
I am not a big fan of this method because you are "secretly" watching the customer who can click on your little minimized window at any time (and hopefully it says we are watching you). If you do this type of oneixt it is better to ask for permission.
I hope this helps a little bit.
-Avinash.
I used to think the same way about conversion metrics. But I've seen the brand searches convert far better. Brand isn't a fuzzy metric – you can value it by comparing 'generic keyword search' to 'generic keyword search +bran' and seeing the difference in ROI.
Another point is that brand awareness campaigns lead to brand oriented searches. Just because the gratification isn't instant conversion (as with few people clicking banners, and fewer converting), doesn't mean it won't happen later. And when it happens later, the bang for the buck is good.
I like what you have proposed here.
"This means for approximately 98% of the traffic to your site something happened and you don’t have an account of that."
In addressing this point, another interesting part of the equation worth mentioning is just how many of those clients who do visit, eventually do purchase or convert vs. the initial inquiry. The old television adage that a consumer must see an ad 7 times to make one impression.
Not everyone acts on the searches they make the same way, some bookmark, some ponder, others compare, etc. This is definitely where brand value exceeds impulse which is a bit more difficult to measure.
Good stuff, I will definitely be reading your book.