Try this.
Ask a famous blogger, a published author, a random twitterer or your mom how to succeed in web analytics, or how not to be a Reporting Squirrel. The answer will invariably be:
Before you provide the data, ask the requestor what is the business question they are trying to answer. Then fulfill that need.
It is a good answer.
Most of the time they, Marketers /bosses /HiPPO's, ask and we puke data out.
The result is also almost always the same.
After a while of doing this you, poor Squirrel, want to jump off a building. As if that was not painful enough, during the course of your employment the company made no actual decisions based on web analytics data. Ouch.
I call it a lose-lose.
If the outcome is so obvious. . . why do we still have this problem? Why is it that we don't ask for the magical thing you were told to ask since the day you were born into this world? Business questions.
Part of it is that we might not be in the position to ask for that question (I don't buy this 80% of the time, sorry).
Part of it is the case that we don't often understand the difference between a business question and a report request. Even for experienced Analysts / Consultants.
Let' us solve this problem today.
I really want you to become an amazing Analysis Ninja and help ensure that this issue never comes back to bite you in the butt.
Come with me.
<
Why Ask For Business Questions?
One word: Context.
I am sure this will not surprise you, but it turns out you are a very unique person. You are distinct from all your other siblings, even your twin.
And it turns out every business is unique, every website is unique. There is one and only one of its kind in the world.
Even if you completely copy and paste someone else's website and start executing, your business is probably unique in terms of the individuals you have collected around you and how they work together.
Or perhaps while you sell via the retail channel like everyone else, your strategy is different in its focus on driving purchases through the web, or maybe you are obsessed only about offline store sales. Or perhaps while you have 100% copy pasted GroupOn's business and are executing it in Russia. It is 100% likely that you are solving the problem completely differently than the US GroupOn site you are copying.
Or maybe you are HP and are obsessed and 100% focused on solving for Customer Satisfaction on your eCommerce site and your closest competitor Dell is obsessed 100% on getting more and more conversions.
This is a long preamble to tell you that the strategy we normally execute in measurement is wrong. We read books / blogs on Metrics & KPIs and we think we know what to do for an ecommerce site or a blog or support site etc and we start reporting what the blog / book recommended.
Then we are shocked no one cares one bit about the data.
If you are unique, why should you crack open a standard analytics tool with its standard reports and metrics and get going?
Or why simply respond to a "report request" and start data puking? The person at the other end is probably uninformed about Analytics and Segmentation and what is possible (even as they are supremely qualified to do their job in Marketing / Sales / HR).
You need business questions because
1. Rather than being told what metrics or dimensions to deliver you want business context: What's driving the request for that data? What is the answer the requestor looking for? Then you apply smarts because you have context.
2. Best practices are highly overrated. If this is your first day on the job, sure go ahead and puke out what "industry experts" recommend. But know that it won't impress anyone because you don't actually know what the business is doing / cares about / is prioritizing.
Convinced you need to only accept business questions?
I am glad.
Three Attributes of Business Questions.
We have done a pinky swear that you are going to start your daily web analytics journey by asking the business what questions they want answered.
I have to warn my budding Analysis Ninja that what you will typically hear is:
+ I want to know how much traffic is coming to our website
+ I want a conversion rate
+ I want a path analysis for our visitors (oy vey!)
+ I want to the list of top exit pages on our website
+ I want to know how many leads we got on website this month
+ Or give me a report that shows click-through rates of our home page promotions
All of these are requests I am imploring you to rebel against. They are not business questions.
Take a deep breath. Simple. With your eyes, not your mouth, say: "I really want to help you."
At this point what we really want to do is refocus the discussion and increase the likelihood that you can be something more than a reporting squirrel. With a twinkle in your eyes politely say:
We are executing a true Web Analytics 2.0 and Multiplicity in our company. In as much as we have access to many sources of data that can help answer What, Why and What Else questions quickly and efficiently.
Based on my expertise I can help you pick the right tool and metric if you could share the question you are trying to answer. What problem are you are trying to solve?
That will probably take all of 20 seconds to say (don't forget to twinkle your eyes).
They'll be struck by your sincerity, and shocked that you want to help this much.
The trick now is to make sure that you are able to recognize if what comes out of their mouth (or in words over email) is actually a business question and not a rephrased report request.
Business questions have these three simple characteristics:
#1. They are usually open-ended and at a much higher level, leaving you room to think and add value.
#2. They likely require you to go outside your current systems and sources to look for data and guidance in order to measure success.
#3. They rarely include columns and rows into which you can plunk data you already have.
If what you hear fails any of the above tests then you have to go back and try again.
Try not to be impatient or show off how smart you are or pick a fight. Keep the twinkle in your eyes, highlight what was different about what they said compared to the first time around, and then gently ask them a specific follow up question.
If you read the three characteristics carefully you'll notice that they are encouraging the best from your requestor (context, business priorities, problem framing), while at the same time encouraging the best from you (your knowledge of data, systems, analytical strategies).
That is the basis for the magic that converts you and me from data puking Reporting Squirrels to Analysis Ninjas who leverage custom reporting, advanced segmentation, statistics, surveys, competitive intelligence tools and to deliver reports with specific insights.
Guess what's the result?
People who just wanted data are now running around taking action based on your insights. Why? Because you did not provide data. You answered questions that were important to the person, tied to business priorities.
In rare cases your requestors might not even know how to reply when you ask them for business questions. Let me send you off with a little gift for the times when that will happen. . . .
Sample Business Questions for Ninjas & HiPPO's.
Look the person in the eye, still with the twinkle, don't forget the twinkle, and say:
My dear friend Avinash Kaushik asked me to share these sample questions with you. He said it would help us identify what's most important for our business.
If you hear the questions below you've hit the jackpot, because these are questions, have the three characteristics outlined above. . . .
- How can I improve revenue by 15 percent in the next three months from our website?
- What are the most productive inbound traffic streams and which sources are we missing?
- Have we become better at allowing our customers to solve their problems via self- help on the website rather than our customers feeling like they have to call us?
- What is the impact of our website on our phone channel?
- How can I increase the number of customer evangelists by leveraging our website?
- What are the most influential buckets of content on our website?
- If we could only do one thing to increase revenue on our website what would it be?
- What is the incremental impact of our display ad campaigns?
- Are we building brand value via activity on our website?
- Do fully featured trials or Flash demos work better on the website?
- What are the top five problems our customers face on our website?
- What is the cost for us to earn $1.00 on our website?
- What is the effect of our website on our offline sales?
Open ended. Force you to go to new sources. Don't contain columns and rows.
That's when you know you are on the right track.
It is quite likely that you will run into other business questions that might be more pertinent in your case.
But the theme that you are looking for is tough, highest-level business problems that you can help solve by analyzing the data you have (or data you don't have but will figure out how to get).
These are the questions that give you goose bumps. These are questions that give you a drugless high. These are questions that validate your decision to come into this field. These are questions that get you out of bed in the morning and feel excited to be alive. (Ok so maybe that is just me. But you'll see what I mean!)
Questions that rock your world, and coincidentally make for a truly data-driven org.
Parting Words of Wisdom from a Practitioner.
For the longest time in the web analytics world we have been content to do one of two things:
1. Provide the data we have in our applications in the hope that in the deluge of visitors, page views, referring URLs, time on site, and exit pages, there is something that marketers and business stakeholders will find of interest and take action on.
2. (Eager to please.) Take requests for reports, create them, and figure out how to email them or publish them on the intranet.
Reality is rather messier as a result of this.
Business leaders feels frustrated that they are not getting insights that they can act on. On the other hand this can't be fun for you. It can't be easy for you to hold the title of Senior Web Analyst and you are reduced to running reports.
Hence the most important foundational element of any effective web analytics program is to ask real business questions, understand those business questions, and have the freedom to do what it takes to find answers to those questions by using Web Analytics 2.0 strategies.
So. . . .
If you are the Business Honcho, bare your soul and share the questions (samples above) that keep you up at night or the priorities that you think are required to go out and win against your competitors (again these are not reports you want).
If you are the underling, seek to get a peek into the said soul and understand the strategic questions that the business wants answered. When you learn what the questions are, go get answers, one at a time. You will now be on your way to truly adding value to your company.
If you are a powerless underling, provide the reports, puke the data being asked, play the useless non value added game that you are being asked to play. But all the while know that real glory lies some place else. Be on a question quest at every given opportunity, with a twinkle in your eye.
Identifying business questions is a journey.
As you solve one set, the next will come up. Or you may be in the middle of solving one set, and suddenly that set will become irrelevant and there will be a new set.
This evolution and change is a sign that you are actually answering business questions and not just doing reporting, because business is always evolving and changing and you have to simply learn to change with it.
From the bottom of my heart I wish you all the very best.
Carpe diem!
Ok now its your turn.
How do you spend most of your time at your job or with your client? Providing data rich columns and rows and reports with pretty font, or answering business questions? If you have tried this strategy already what are the biggest barriers to you being asked questions rather than data? If a strategy has worked very well for you in this context what is it? Twinkle in your eyes? [I knew it! :)]
Please share your experience / feedback / pain / joy.
Thanks.
Via 
Spot on! Me, standing up from my desk chair and applauding! :-)
I regularly receive requests like the ones you listed "I want to know how much traffic is coming to our website during…", "how many visits did we got for following product sections…", "I want a path analysis for our visitors" (oh, soooo common).
While, I don't answer what you are suggesting (should try once for the fun :-)), I always ask "why? What for?" trying to understand the motivation and reason behind a request. Why?
#1. I am curious :-)
#2. To understand the context as I want to make sure I will provide the right data that will answer the question
#3. To provide an answer, not just raw data or a line graph.
If the person is not trying to solve a problem but just wants figures for the sake of getting figures, I politely explain them where they can find these – (and get them an account).
I am a Web analyst – not a squirrel. Not anymore, done that for too long.
It is up to "us" the analyst to adopt the right behavior and your post described it very well.
Great work!
Hi Avinash,
Your post is really eye opener for me.
I am getting the lots of request from the customers, in which they are requesting like "how many hits their website recieves"
I have to then ask customer that there is no hits data available, & again ask whether they want visits or pageviews.
Some customers ask me that how many peoples visited home page of the website.
Let's go ahead and add +1 to people standing up from their office chairs and applauding!
I think everyone who does Web Analytics gets requests from their clients such as the examples you already listed, things like "How many visits did I get" and "What's my bounce rate?". And honestly, it's not the client's fault – they know what they know, but that doesn't mean that they're not willing to at least listen and consider what you have to say (in my experience almost everyone is willing to at least listen).
It's really the WA's responsibility to ask the business questions and help solve the business problems by asking said questions. Every business man and woman wants to make more $$, so if there's a possibility that this might happen by listening to you, they'll start doing some listening.
But yeah, it's our responsibility to do so, as WA practitioners. Imagine going to a doctor and asking the doctor "What's my blood pressure?". Do you think that the doctor will just tell you "120 over 80" and collect your $15 co-pay and that's that? You came to the doctor for a reason (to solve your health problem), so the doctor is going to probe and ask questions that are going to help solve your health problem – and not just "puke" out your blood work statistics and send you on your way. And if your doctor does do this, it's time for a new doctor :)
Thinking like a doctor, asking poignant business questions and solving issues and problems is the best way that we in the WA industry can help businesses everywhere.
P.S. I wouldn't flat out "refuse" a report request from a client, though, as the title of this post would suggest. That's a fast way of losing a customer right there :). I suggest to give your client exactly what they are asking for, but before they see the data or open the attachment, do make sure that you are asking business-oriented questions in return. Chances are that despite your best intentions, your client (who may be a point of contact and not the CEO / owner) is just doing their job and doing their job means asking you for reports because their boss said so.
Excellent post. I am a big believer in delivering actionable insights and simple data puking does not paint a clear enough picture for HIPPOs to take action.
Delivering actionable insights connected to real business outcomes is like painting a picture, if I may. Artists use different colors, light, and objects in their paintings that, by themselves, can be beautiful, but when the artist creates relationships between these colors, the use of light, and the objects on the canvas, we receive the gift of a more full experience and insight.
Much like Web Analytics, it's the relationships between objects, or data points, that bring about greater insight.
Asking business questions helps us understand what picture a HIPPO is expecting to see. We web analysts then get to go on a great adventure to figure out how to paint that picture. What colors do we use, what COMBINATION of data points address the business question, how much light or a lack of light should we employ, what competitive intelligence can I incorporate?
Great job Avinash! Keep 'em coming!
I was doing some squirrel reporting thinking "there is no way in the world my client can consume this data, there has to be a better way… I wonder what Avinash has to say about crappy reporting" and here you are like a ray of sunshine. What would I do without you reminding me I am not alone, shining some light on a better way?
I love the ideas you have posted here and I am soooo going to spend tomorrow's Squirrel status meeting asking for business questions.
+2 for standing on desk & cheering!
Hi Avinash,
Great 'rally cry' of a post!
I strongly feel it is the analyst's duty to sahpe the questions they are asked – answering bad questions/requests is a road to nowhere.
In pursuit of actionable insight we have been running an iniative called 'Performance Acceleration' with our clients. We request their business objectives and then focus on tuning, starting or terminating marketing tactics in pursuit of them.
We try and sit objectively inbetween them and their agencies/providers – ensuring that every tactic, paid or natural, contribute efficiently to achieving the overall objectives.
By doing this we are super lucky, we only get asked the cool questions – but i guess that is the point of our process, to focus on the important stuff!
Great post and thanks again,
Rob.
Well said Avinash.
I love questions that dig deeper into the WHY than the standard web analytics.
Since the employer/client/friend is paying you to answer these questions, they should ask them! I can get anyone a log in to the Google Analytics account. That data is all there for them.
So take advantage of your "web guy"today!. Ask him or her a question that has relevance to the context of where your business sits today.
Are we better off than we were a year ago? A month ago?
WHY are potential customers coming to us from Facebook… Not just how many.
Is it a good thing that our page views are up? What does it mean?
Those are the questions that a log in can't answer. And those are the ones that need to be asked.
I also have had luck with the questions:
What decision are you trying to make?
What happened to make you ask for that data? (Another way to say "why do you want it?")
And,
Be humble: "If I understand what you're going for here, you never know, I might notice something interesting when I'm putting together the report."
Hey there,
It's impossible to disagree with this post, or really anything you ever write. The problem I'm having is finding advice that is more directly relevant to me. I'm posting this not b/c I"m a narcissist and want a site that creates content solely for me, but b/c I'm sure there are others out there that feel the same way I do. There are 2 things I find myself constantly saying after reading any blog post:
1) "I'm not really empowered to do that kind of stuff"
I only this year began my career in web analytics at a major internet media company. I'm young, so I recognize that I'm still in a learning stage and have to prove myself before people start empowering me to make strategic changes to the company. In no way do I feel anxious or entitled to anything more than what I'm currently tasked with. But being a major company, there are things that have been discovered by people long before I even considered an analytics career and yet still have not been addressed. I constantly will find reasons why bounce rate is high or have suggestions to increase traffic. But what do you do when you don't work at a smaller internet shop? When you feel that your company is too big to make any massive changes in any any sort of realstic time frame? Do you accept it and try to focus on the smaller yet more attainable challenges? Do you have to part ways with the company merely b/c the ship is too large to make any major changes in direction?
2) "What if the business goal is not to make a sale?"
I find that a lot of discussion about bounce rates and conversions become more difficult to apply when you are dealing in content driven sites. How do you bring revenue to the company other than the suggestion of "more traffic". In line with the first issue, when a company is very large, how does an analyst provide better reporting when things like product changes, SEO, and marketing strategies are not exactly in your purview. Or are they?
Maybe the answer is to just shut up and push through, but I feel sometimes a bit restricted on what I can offer even with theh great advice given on this site and in your books as well.
I have had exact same issues you mentioned, our company is content based, most of the web analytics blogs, books, articles are marketing oriented.
Also since is a large company is not easy to convince so many high level position staff to take decisions based on the results obtained from our analytics tools.
Analyst 2: Maybe there is a way to frame back to your bosses what value the digital existence delivers that will get them to pay attention to you.
Start with this post: The Biggest Mistake Web Analysts Make… And How To Avoid It! If you are able to distill down what the business is solving for, your analysis will be focused on what the business really cares about. No boss can ignore that.
If you would like tips about what metrics will be most relevant for your business, see this post: Best Web Metrics / KPIs for a Small, Medium or Large Sized Business. I recommend a mix of ecommerce and non-ecommerce metrics.
Finally here is a collection of tips to help your analysis be a little sharper: Data Analysis 101: Seven Simple Mistakes That Limit Your Salary.
All the best!
Avinash.
Avinash,
What a great post with which to start the week! Here's an exercise that served a previous employer, myself, and my team very well:
1. Ask as many of your constituents as possible to print every report they get/need. If it shows up in your inbox regularly, you clearly need it, right? And don't forget all of the emailed reports that are routed into invisibility by inbox rules.
2. Meet with each constituent and really talk to them about their part of the business. No leading questions about data, just talk. Listen. Learn.
3. After speaking with them for a while, ask them to go through the stack of reports they printed and highlight each specific data point that she or he uses to make real business decisions. Sit there with them. It takes a long time, yes. The light bulbs go off as people realize how much useless data they're being asked to consume and manage.
4. After you have numerous reams of paper, sort by report title.
5. Flip through all of the copies of each report and note which data points were most commonly highlighted.
6. Compile a dictionary of these metrics and index by business users and purpose.
7. My bet is that you'll have a handful of metrics that you can use as a foundation for an informational infrastructure.
8. Get stakeholders (Finance, IT, Product Management, Marketing, etc…) to agree on definitions and sourcing.
9. The process of agreed-upon definitions will likely organically lead into deeper questions such as those mentioned in today's post. If not, WA practioners have a prime opportunity, and responsibility, to prompt with business questions.
Cheers,
John
Less good cop, more bad cop.
As a consultant the gold at the end of the rainbow is tempting, but if you show some patience that gold will turn into diamonds.
Resist the fallen fruit that you can pick without reaching higher–it's often rotten! Low hanging fruit is better, and fruit closer to the sun may be even more ripe.
Coincidentally, I blogged that "your reports suck!" a few days ago. On my Swedish blog webbanalys.nu though, which sucks too for some, I guess. Google Translate may think it's all bork bork bork..
Beware of both HiPPOs and hogs. Hogs hog reports and want to bring you wuth them when rolling around in dirt.
Great post. Lots of these questions are so basic and eye opening that people really forget or take it for granted. I am big fan of the fact that asking questions is always educating both for the person asking it and also the person whom we are asking
Outstanding post, Avinash!
These questions are perfect for getting to the root of answering true business questions, however..
Our biggest client (for the agency I work for) is extremely traditional when it comes to advertising which yields many roadblocks. The main one from my prospective is the insane amount of data we are able to collect and gather insight from via web analytics VS yesteryear and the methods of determining ROI. The client cannot step outside of wanting us to give benchmarks and only report on things like: visits, pageviews, bounce rate etc.. (your standard and by default GA dashboard).
My question to you is, besides giving them the twinkle in my eye, how can I get them to step outside of this metric quicksand just because they think these are the only "important" metrics? I've even tried your method of asking the simple question, why (3 times)? I understand your logic and see great value in obtaining this information from a client, but do you have any recommendations on how to really convince people/clients to define clear business objectives and questions in order to provide them with the appropriate KPIs and metrics that should be reported on?
@ Web Analyst Newbie –
I'm not Avinash, but having started my own web analyst career in a large difficult-to-change content-based media company, I feel your pain and maybe this might help you.
1 – Accept the truth. Not all companies are difficult to change and include layers and layers of bureaucracy that prevent anything real from getting done. And if your company is like that (as some companies are) then your focus might not be changing the company (there are too many layers getting in the way) and more focusing on yourself. So hone your skills, listen, and don't get frustrated if you don't see change. Eventually, you will part ways with the company. Be sure that when you do, you have a long list of recommendations, analysis papers you've written, tools that you know about, and business communication skills that you can easily pick up even on the lowest level of the biggest company.
2 – Do fabulous analysis anyway. For example, still ask all the questions to your immediate higher-ups. They might not know, of course, or care, but it gives you a place to start. If they don't know the answers, make some up and provide analysis that pushes towards those. Then use the answers to do side projects. Chances are that you will get good at rote reporting and busy work after a while– create for yourself some time at work or at home to work on unasked-for analysis. You can push these to your immediate higher-ups and if they like them, they will move up the ladder. Chances are good that even if the VPs all like them, they themselves might not really have the ability to make changes based on them, but if you play your cards right, they will have excellent things to say about you and you will hone your analytics skills.
3 – Research, research, research. Let's say that your company uses Solution X, which is really lousy for what they want to do. You can make your own blog site to implement and learn Solution Y or Z. You might not be able to implement it on your work media site, but if you have a good argument, do the research, put it in a paper, compare another solution, you can push the professional report with your expert opinion to the Immediate Higher-Up. Even if they throw it away, you now have the skills in both solutions and the experience comparing them.
4 – Make one-on-one connections. Chances are good, in a big company, there are a lot of disparate people using the data. They might not even be using it properly. But they are all individual chances of showing how useful your analysis can be, and what an expert you are on their data system. These people and their support will be more important to you in the long run as business connections who believe in your power to change their lives.
I'm sure Avinash has better suggestions, but as a fellow web-analyst, I thought I'd weigh in. Hopefully this at least gives you a place to start.
@Kalani
Thank you. Really great advice. I agree completely, and I also am trying my hardest on Making my 1:1 connections. There's a lot of smart people here and I want to work with them and learn from them as much as I can.
Great article.
However, I wouldn't always assume that a request for a data point is useless.
I find that there are two different types of requests that come to us, and we often focus on only one type. The first is best served by answering a business question, which is best served by the method Avinash so eloquently writes about here.
The second type of request is for a data point that is simply needed for a "sell" presentation. E.g. I'm putting together a presentation and need some impressive numbers to use in order to make my point. Don't underestimate the importance of some good data points for "sell" presentations. The intended audience might not know that your website gets a tremendous amount of traffic so is something that should be invested in vs. say a field marketing program. Ideally the analyst would be brought in to help do a detailed analysis to answer that business question, but sometimes the answer is so painfully obvious your HIPPO just needs some simple data points to present to his HIPPO to make his case (it's HIPPO's all the way up…).
Michael: I like your idea of simply giving people access to the data if all they want is data for the sake of having data. They can then go an bathe in data as much as they want without wasting your time. Great recommendation!
Joe: Both you and Michael mentioned this and I did not. I should have mentioned that to a great extent I do lay the blame for our sorry state on us, Analysts and Practitioners. We keep doing data puking and then we wonder how come no one is data driven. Good point.
I am glad you see that my call to "rebel!" was more a call to a new mental model and less a job limiting recommendation. It is certainly a gradual process to evolve away from data puking. But if one does not embrace the rebel mental model nothing will change. So… Rebel! :)
Rob: I love th idea of doing a deliberate and specific "performance acceleration" program with your clients. It provides a structured path out of this. Thanks so much for sharing this idea!
Web Analyst: Regarding #1: I have never believed that being proactive needs to wait until you are an expert or empowered to be proactive. I realize that you are new and young but you can start to add small amounts of value with proactive questions and proactive providing of 5% extra stuff or right stuff. And then over time you get better. If it company ignores you or won't listen to your insight then it might be that 1. Your (or my) insights are not powerful enough (say not quantifying economic value, just reporting goals) or 2. It is time to move to another company that will appreciate your work (this is not easy in a recession but I would certainly put my resume out there).
Regarding #2: A vast majority of business goals will have nothing to do with eCommerce or making a sale. This is were lead gen and downloads and voter registrations and rss sign ups and videos watched and…. tons of micro conversions come in. You measure all of those and quantify value. Please see my post on macro and micro conversions for many more specific examples.
Oh and never shut up. Be the change. If not here then someplace else. Life is too short to shut up.
[Also please see John Stansbury's nine step process, I think it will work in your case.]
-Avinash.
John: Awesome nine step process to go from data puking to focusing on things that actually add value. Thank you for sharing this much detail and your experience. Thank you very much.
I can't think of a more worthy cause for killing a small number of trees, they would have died for a good cause!! :)
Lars: Joe had reflected some reality, that he can't just say no to his clients when they ask for data puking. And that worried me, because he is right and so how do we deal with it.
I think your wonderful analogy provides a great path forward. Don't just go for the $500 that you can make from the client via data puking (which by the way we all have to do to some extent) but rather internally in our mind we have to extract the business questions because that will turn gold into diamonds.
The client wins, better insights, and we as consultants win, diamonds!
Kalani: Your feedback for Web analyst Newbie are much better than anything I would have said, thank you for taking the time to add such detailed and specific recommendations. Much appreciated!
Mark: If you have tried the "so what" method to show them how bad their metrics are then I do have one more recommendation.
Please try using my Line of Sight model. I have found that taking all the metrics out of the equation and forcing your clients to think about objectives, and not just some vaporware objectives but Price – Cost – Market Size – Market Share, and flowing that out to specific sales and marketing levers being used at the moment and then bringing in the metric (which will now be directly tied to a desirable outcome) will normally help accomplish.
Pick the right metrics by taking metrics out of the first three conversations! :)
Simon: Thanks for the distinguishing between the two main types of data puking requests. You are right, they are both important.
For the second one though I would recommend the strategy Michael Notté highlighted in his comment: Give them access to the data. Train them if you have to, the investment is well worth it. Then they can go help themselves when they just "want the data" and you have just reduced your personal burden (to do more value added work) AND empowered people to help themselves for request type number two!
-Avinash.
Spot on here Avinash – again – the ultimate CTA!
It's like the #Measure "12 step" process:
* Admit your data addiction has become unmanageable, constantly feeding data without positive results or insight
* Believe in a greater future
* Take inventory of the real contribution you can offer
* Confess your sins to the #measure folks around you (or blogs like this)
* You may find yourself asking HiPPO's for forgiveness (in other 12 steps this would be relapse but not here!)
* Seek out those prior requests/requestors & amend the results with better questions
* Constantly challenge yourself to ask & provide more
* Never stop learning, and finally
* Pass on this awareness to those you work with
There will be relapses, it must constantly be practiced, you will be tempted to fall back in the data bottle, those quick hits, but don't
And perhaps Avinash the only thing missing here is the emergency list of tips for those awkward moments when it's apparent those asking for data have no clue the root question.
It can be rough, and it may not be a 100% success with all, some will resist (see coping list immediately above). There are times where data is completely valid.
The job of a professional here is knowing when that is, and the other 95% of the time having the conversations that your clients deserve and all good consultants know how to draw out.
Great post, Avinash!
Data vs. Information – it's the story I find myself repeating to our clients and internal stakeholders. I like to start my response with "Yes, I **could** provide you that data, but what if I could give you xxxx instead?" Add a chat about how x becomes y and then y becomes $$. (now I'm talking their language).
As with so many other things, making the other person feel like they're creating their solution goes a long way in gaining their buy-in and earning their conversion to the data-driven lifestyle.
Short-term positive feedback – I love the point in the conversation where you can truly see the light bulb turn on and they become an ally in the data-driven crusade.
Long-term positive feedback – I know when a former report requestor drank the Kool-Aid when they qctually book time to sit down and talk through their needs.
RE: Your line on twins and businesses and how no two are alike – Love it! And it's so true.
Thank you!! ;)
As other people have said spot on.
This is what I find the hardest thing on my job. It is about educating people to ask the right questions but it takes time. I wonder if there is an organisation out there that excells in this?
What if the analytics access is not given to the marketing person and kept with the technical lead?
By technical lead I mean a person who knows to implement the code and get that analytics blurt out those numbers and nothing beyond that.
A lot of companies are making this mistake and expecting those numbers to make some difference to their revenues. Any points on how to deal that?
While all the suggestions here are good, the reality is that unless you have done your homework, nothing much will come of your efforts. You need a champion who can stand the heat with you as you do the right thing.
As a consultant I (sort of) have a champion in the person who hired me. Of course I've also had that same person turn on me and proclaim that I am the source of the problem. Goes with the territory.
You can not force a person to eat especially if they are higher up in the chain than you are. You need to make them hungry. 80% of your work has to be done before the shoot out (or protracted fire-fight).
Solving problems by eliciting the proper measurable questions first is Business 101. As my favorite commercial says "I gots it."
Were it not for my dear friend Avinash, this squirrel would've jumped long ago. :)
Thanks for keeping hope alive.
Great post Avinash!
I'd like to add that I've noticed that clients who tend to ask open-ended business questions to me have more respect for this "reporting squirrels" opinions than the clients who simply expect data points from me.
Also, the same clients who ask these open-ended business questions generally adapt quicker than the other clients.
Personally, I love the clients who attempt to teach me more about their business and what they expect out of their website.
Great post as usual.
I find that asking an additional question is often required and I'm surprised how many analysts don't ask it:
"What are you going to do with my answer once I give it to you?"
Sometime the person asking the question isn't quite sure, so I help them out by giving them make up answers and then asking them again – what are you going to do with this data?
For example, I might be asked "what's the bounce rate for campaign X over the past 14 days?"
I tell them (on the spot) "It's 45%. Now what?"
You need to hammer into their heads the "now what" :)
Ophir
Mark: I love it!
Thanks for sharing this specific guidance and inspiring others with your words. I really appreciate this very much.
PS: #7 & 8 I think are the hardest. I am constantly surprised at how rare they are.
Chris: You are absolutely on the money that taking the conversations from "you want this, but I could give you this + 2, and here is how this + 2 ties to your personal $$$" is a magnificent strategy. I will keep beating the drum of Outcomes, Outcomes, Outcomes!
Thanks for sharing the two signs to look to judge if things are going in the right direction. They are on the money.
Theodor: I wish there were more, the ones I see are mostly small or medium sized companies.
What I do see in my engagements with large companies is that there are individuals who excel in this and hence their small group / division in a large behemoth does really well. But at a big company level it is exceedingly rare to see this.
But that is why we need to fight and that is why we need to focus on this. :)
Abhinav: Two recommendations.
1. Try to use a framework like the Clear Line of Sight Model to work directly with the business leaders to execute the recommended steps. It will get them to appreciate what you bring to the table, it will get them (if they are smart) to help you understand what the priorities are, and it will help you focus on what matters to measurement. Technical stuff will then be secondary.
2. If the above does not work, or you have tried other structured processes to address the issue, then it might be time to look for another job. Don't quit tomorrow, but do get your resume out there.
Dan: I am a passionate advocate for
1. Squirrels investing in themselves to elevate themselves to Ninjas (i.e. continuous self improvement)
2. Using frameworks like the Web Analytics Measurement Model and Clear Line of Sight to approach web analytics right.
So you are 100% correct about the homework part.
Consider this specific blog post as step three. After you do the above.
You have your personal ducks in order. How to do you ensure your company/client gets value from you? Answer business questions. And if the person, as you kindly put it, won't eat or does not feel hungry then there are other options for you personally. :)
Ophir: I have this process of asking "so what" but I like your more street smart "now what!". [See: Kill Useless Web Metrics: Apply The "Three Layers Of So What" Test]
:)
-Avinash.
I must have to hone my "eye twinkle" a bit more because I just wrapped up a nice big column/row report. I am definitely prone to your 1-2 list of the web analytics world.
Experienced Analysis Ninjas such as yourself seem to have a type of "reporting aikido" whereby you redirect their request for report puke to Web Analytics and Multiplicity.
I may have to offer up some ad hoc analysis based on your list of business questions for extra credit.
Wonderful post as always!
Love reading this blog even though this is my first time posting.
I am trying to figure out how I can apply this ideal to my own situation.
We are currently building out our own internal analytics system so these questions really hit home for me.
The problem is that we are performance based marketers and really these questions for us turn into 'How can we improve our EPC?' and 'How can we balance our CPC costs in order to leverage the most traffic?'
I think my point is that KPIs such as conversion rate, click through rate, and visits are the critical questions that we have to ask in order to gain insight into what our response should be so it is difficult for me to just throw out that way of thinking.
The real value for me, and is the reason why we are building this system, is when we are able to automate these insights and actions to the point where we can optimize things based on comparisons of historical data from working campaigns.
So what happens when your business questions are a directly affected by these metrics that you shouldn't be thinking about?
Though it is a very real possibility that I may have completely misunderstood your point also.
Stephan: I think, and this is just my hypothesis, that you are conflating two different roles: the business's role in asking good questions and the analyst's job to answer the questions using analytical techniques and metrics available at her/his disposal.
The two questions you mention in your comment are not business questions. They are important, they are specific, they should be answered. But strictly speaking they are not business questions.
Business questions will be of the type mentioned at the end of the blog post.
So it might be: "What are the best options to reduce acquisition costs while not killing our traffic?"
You, the Ninja's action, might be to analyze all the acquisition streams, their costs, do some what-if and find CPC balancing is the best option (or more likely because you did not start at a rabbit hole you might find something different / better). You will do this via segmentation, drilling down below the KPI level (remember not everything is a KPI) and look at many metrics etc and answer many questions before you answer your business question of how to reduce cost.
Two different roles. The first outlines the big desire. The second leads to specific analysis on various fronts before giving the right answer.
Some of the latter work can be automated (perhaps by the system you are building). But I must admit that I have never believed that "automated systems" provide insights. They provide data. Often timely well processed data. But they are missing much of the context and tribal knowledge required to covert that data into insights. That is where you come in, an actual human.
I would lower the expectations in your company that your internal system will deliver insights. I would stress that it will automate things and give you access to better data faster and it will allow you to do n-level segmentation etc etc which will deliver faster insights. But it's you.
That is why we love analytics. You are important. 10/90 rule. :)
Josh: It takes a lot of practice, and the willingness to get beaten up and getting up and trying again. Sad but true.
This is very hard but as Joe said we have to figure out how to fight not just to get a pot of gold at the end but convert it into diamonds. That to me is a great motivating factor to try and see if on top of their data puke request I can add something that I think they really want or to go back and ask for some clarifying questions hoping to get to the root cause.
-Avinash.
Avinash, thanks for the great insight, i like many other have been susceptible for too long to the internal customer data reports. This is something i plan to start implementing immediately.
I would also like to say that if a person asks for a report and they can not come up with business question behind a report, i would tell them to come up with one and I would get them a report. There is not enough time in the day to get people reports for the sake of reports; if they dont have a need, they dont really need the data imho. (unless its a hippo)
Great post!
Wicked Avinash!
I've spent the whole morning and tried to identify Business Questions, just did not think it was called Business Questions until I read this post!
Since I work in the public sector is often our Business Questions[..] require you to go outside your current systems and sources to look for data and guidance in order to measure success.
This often creates exciting challenges for the person who came and asked me for the data in the first place.
So by using this method, you are so much more than a Reporting Squirrel! Now im off to have a meeting and I will with a twinkle in my eye talk about Business Questions ;)
Thanks again for an eye-opening blog!
From Sweden!
Johan: Government websites can pose a unique challenge, especially because most of the the time the government just wants to do and not think. :)
Here is a blog post I had written for an enterprising person working for the Belgian government about how I would approach measurement for them:
~ Web Analytics Success Measurement For Government Websites
Perhaps you will find it to be of some value.
Good luck!
Avinash.
I have recently been interviewing for two junior web analysts to join my team. A key question asked in the interviews was…
“If someone in authority (HiPPO) asked you for some ‘numbers’ what would you do”?
The two candidates that were offered the job answered with something along the lines of…“I would ask them why they wanted to know what they wanted to know”.
Despite having no analytics experience or any knowledge of analytics tools I have high hopes that these two individuals will go far.
We had spent the previous 12 months unsuccessfully trying to fill a vacancy for a senior analyst. Of the expert tool users and reporting specialists that were interviewed none showed an appreciation or understanding of what being a web analyst is really about.
Ok– I know you're right, you know you're right.
But when bound by the laws of organization physics, I come up against an insurmountable force:
Money.
When I ask the "why" of the data, I invariably learn one of two things.
* My organization doesn't yet own access to the tool I would need to get you a meaningful answer
* I can get you a meaningful answer with 10 ours of work.
Or, I can spend 10 minutes giving you a spreadsheet of historical page views. And since we have no budget and are 50% understaffed for even basic puking duties, that's what I do.
I hate it, you hate it. When people believe they can limp by with bile in lieu of meaning, they restrict funding to one bare-bones analytics tool and 2 seasick analysts. How do you show them that they need more? By asking the questions, and using the superior methods, that show value, so that they'll provide the funding, so that you can afford the headcount and the tools to ask the questions and use the superior methods…. oops.
It's circular. How do you teach a company to prioritize meaningful data when you're so strapped for resources that it makes you want to, well… puke?
One caveat to refusing report requests is to log all the requests you receive, no matter how small. It's a great way to communicate to stakeholders the demand for metrics as well as a way to substantiate the need for additional resources.
It's also a good way to look back and to consolidate repetitive requests into dashboards that all can benefit from.
As always, great post!
Maria: That is a great insight! Your last observation is particularly key.
So many people who are now "senior" in the industry grew up with the log file and technical implementation skills (which were key then) that they really struggle to bring a marketing / business perspective to analysis.
They are all really good at the technical and the complicated technical (valuable skills), but web analytics itself long out grew the value of implementation and efficient "data puking". It is about "show me the money!" now, and rightly so.
Kimberly: I empathize with you, and this is not just at large companies. Let me also say that you have guessed right, I have personally lived through this "hell".
I should have said at the start of my blog post that my recommendation works when there is at least the faintest of desires in the company to want to be data driven and / or some support for the Analysts to do their job well. If that does not exist then no question asking will get us any where.
Sad but true.
In scenarios such as this my approach (based on graduating from the school of hard knocks) is:
1. I first look inward at my own work as an analyst to see if I am doing this right. Are you insights powerful enough (whatever little I can provide in the compressed environment you are describing)? Am I just reporting page views or identifying some micro and macro conversions, quantifying economic value, focusing on visitor loyalty increases (and causes) etc?
So how can I improve what I am doing.
2. Have I tried the strategies outlined in how to create a data driven boss?
~ Six Rules For Creating A Data Driven Boss!
3. If #1 and #2 have failed and I have determined that I am working at a company described in this blog post:
~ Online Marketing Still A Faith Based Initiative. Why? What's The Fix?
It is time to move to another company that will appreciate my work. Life is too short to waste it in circumstances where the absolute futility of my job makes me want to jump off the Golden Gate bridge.
It won't be easy to find a job in a recession but I would certainly put my resume out there and start looking. Meanwhile I'll also take up quilting as a hobby to find meaning in my life! :)
All the best!
-Avinash.
Great post Avinash :)
Thanks alot for such a great post, I have shared the link of this post with some of my clients and hope they will understand the point.
Regards,
Praveen
Great blog post!
I have the biggest problem with report requests. Does it matter if you get 300 clicks from an ad if each of them bounces or never gets to the final conversion? Nope, but that is the kind of request I get. To make things worse, analytics is only half my job. I create monthly reports for at least 10 different clients aside from my other broad Internet marketing strategic development and implementation duties. It is really tough to avoid data pucking with the time crunch and with such little time on each client!
This is a great motivator for me to insist on asking (and thus receiving) those business questions and getting down to actual issues instead of numbers. I know my account execs and my clients don't get the numbers, but when I don't have anything coming in (questions, direction, feedback) it is truly hard to give good insight.
Hopefully I can start to demand (nicely) a little more critical thinking and involvement on the AE side in developing questions and goals to drive analytics.
I believe this through and through: unless we are using analytics to make decisions and reach goals, we are simply wasting our time looking at pretty charts and columns of numbers each month and I am wasting my time putting them together.
Thanks for the push in the right direction and away from all this time wasting.
Leslie
A lot of great feedback to this post. I wonder if we can simply get away with asking back:
Why do you want to know?
-or-
What are you hoping to learn?
Yeah, we might get the "just send me the report" response, but we should take some responsibility to open the dialog.
Oddly, I was just about to tweet that I need to change my educational diet. I want to read more about the industries I serve, and peel away from analytics reading a bit.
I'm hoping I can learn some of the good questions ahead of time, so i don't go down this road every month.
Thoughts on that?
Of course, I just noticed that Maria's response above was basically mine in a nutshell. Thanks Maria, and sorry to ask the same questions :)
Leslie: It is hard to get them to answer your questions, at least when you start the relationship, especially in a client consultant relationship where the Client believes you know nothing and they are a gift to you. So starting with some data puking to report requests is all we can do.
But you are right to stress that no insights will come of it so from week two (see I am the patient type :)) we start asking questions and trying to get to some root causes.
I am thrilled that you are on the side of non-time-wasting. Life is just too short for that! :)
Good luck.
Kevin: The two questions you have outlined are prudent and will do the job. They are open ended, which is nice.
I think if you back them up with some knowledge of the client and their industry you'll be able to have a much better conversation with the client. Because in response to your question they might say something odd or even but based on your knowledge you might be able to drill down or drill sideways to nail down exactly what they are looking to get done.
So your plan to change your "educational diet" is completely on the money!
Avinash.
So I have a question not directly related to this post.
I'm a beginning website manager and I would like to enhance my knowledge and understanding of web analysis. I've read many posts on this blog and I would like to go one step further and read one of your books. Which one should I begin with?
Good point there! Acutally one of the most important questions about web analytics for me at the moment.
In my experience its not that hard to ask the right questions as long as you have a clear target like sales (like an online shop that sells T-Shirts for example…btw I love the new page-analytics feature for that).
But it can become pretty though when you have to work with softer targets like creating awareness. Especially for websites that are selling "promotional-goods" it is a hard thing to do.
For example the music industry. Most of the new media / digital departments are having a hard time to proof that they are worth the money. TV, radio and print departments are still much more appreciated in most labels. On top of that iTunes & Co dominate the online retail so the online departments often have no direct control about the traffic and the impact of the awareness they are trying to create.
My point is: There is a difference between clear targets like sales and softer targets like awareness. What question do you ask someone who's aim is to create awareness (despite newsletter subscriptions and stuff like that) but is not directly selling? Maybe I should put like that: Is there even a way?
I would appreciate every hint ;)
Philipp
Hey Avinash,
What a great support from your blog to convince about the business questions and not only data pulling for report.The HiPPos will definitely check out the request they always used to send us only for data.
I am hoping for positive feedback from these guys.
Let us see. I looking forward to solve much more business questions not the reporting :).
Thanks,
Bhagawat.
Also I forgot to add one more thing that I liked the comments you got for this post.Thanks everyone for commenting and sharing their thoughts here.
Philipp: You are right that success measurement is a bit easier for ecommerce type sites and a bit harder for non-ecommerce sites. But. It is totally possible, just needs a pinch of imagination.
For example if you step outside the clickstream world you are able to tap into competitive intelligence and other sources to measure success in improving awareness. Some specific tips on this are in my post about measuring success for branding campaigns (where all the targets are "soft"):
~ Brand Measurement: Analytics & Metrics for Branding Campaigns
The music industry case is a bit more unique. I am not so sure that it is a measurement problem, or at least not yet. The music industry has decided to put its head into the sand and wish away the internet, their strategies reflect that. So it is a business problem they need to solve first (measurement will come later).
It is important to realize that we have limits when it comes to data. We can only do so much, after a certain point if the business side of the house is in a mess or the Senior Leadership is unimaginative about using the web then it is not the data's fault one way or the other. :)
Please see this post:
~ Online Marketing Still A Faith Based Initiative. Why? What's The Fix?
Avinash.
Hi Avinash,
These are great suggestions on driving meaningful analysis. Thanks!
Turning into a reporting squirrel is one of the biggest boobie traps in the world of being a web analyst, imo. No matter how hard you convince yourself that "things will be different this time," we'll end up putting those tables and columns together if we let any one else, especially the hippo, drive the web "analysis."
Of course, we are working to make our hippos and company/clients :D , but if we aren't resolute with our initial objectives, other voices and opinions will begin to dominate the course of the web analysis, inevitably leading to pages and pages of spreadsheets.
It sounds silly, but it feels like this is a beast I'm constantly fighting off with clients, especially for those that insist on having their hands in the data and want to make their authority known.
Loved this post.
The purpose of analytics is, after all, to further business goals… not ot just take a thermometer reading occasionally and never investigate causes and remedies…..
However, it is much more difficult to answer the business questions… and to be sure that you have really found a good true answer.
What advice do you have for aspiring analytics experts to guide them on the path to being able to contribute meaningfully through analytics?
Are there specific books that address the topic? Other sources of info?
Dear Avinash:
I was working in the field of web development for more than 12 years. And in the field of Web Analytics for the last two years. In all this time, I have heard a real good bussiness answer only once. Maybe I have asked the right question also only once.
Some day a refered unknown client call me and said: I need to improve my web. I said: "Ok. What is the purpose of your site?"
The client answered very quickly: "Get more patients, and as a result make more money". (The client was a doctor, neurosurgeon, that have a web with a great number of informative articles on his specialty).
The response, not the question, was open ended, forced me to look beyond the data offered by my beloved tool of analysis and, of course, brings to the table an issue that could not be resolved with rows and columns.
Perfect. And very,very exciting.
With this answer in my mind, I visited the site and in few minutes I find insights. Only one example: the site needed improvements in their metatags, for get organic trafic. But thinking in patients, not in other surgeons…
But I know that the real hard work not was done by me. The real hard work was done by the client before calling. He spent some time thinking about your site, and obtained a business response to a business question before no one asked for that.
After this event, in my diary work, I try to give customers time to think. It makes my career a lot more fun.
Regards
Felipe.
Avinash, I am a small business owner who need to know a litle of everything.
I love your ideas, I love reading your blog and I have all your books but I have to skim through all your articles even your books because they are extremely long and about %60 of it useless, time consuming and repetitive.
I am sorry that I'm not adding anything to the conversation above but I'd really love to see something short-easy-comprehensive rather then 100 pages mentor speaking to your ninjas.
@Matt – I'm sure your feedback to Avinash is valuable and he'll take your ideas into consideration, but I have to disagree in your evaluation of Avinash's work. I understand that, as a small business owner, you don't need the level of detail as someone like myself who specializes in this area, but that simply means all his ideas aren't things you can do yourself (most ideas are likely applicable to you and not "useless" as you claim, but you have to prioritize what you can do given your resources).
Maybe I'm alone in this train of thought, but I don't think you can expect to come to a place that's meant for in depth thought and discussion on a topic and then complain it's too long for you to read and that it's repetitive. If you have read Avinash's work and tried to employ some of these ideas, you'll find that the world of web analytics is anything but short and sweet. It's one of those "the more you know, the less you know" kind of deals. And don't get discouraged if you hear an idea repeated. Most of what Avinash talks about is much easier said than done. He makes a lot of this sound easy because he's such a great communicator, but doing these for yourself can take lots of time and experience. So before looking for the next great idea, think about ways to get better and what you already know how to do.
Personally, as a web analyst, the in-depth length and focus of Avinash's posts are what make them such a valuable resource. The web is full of short and quippy tips for people, and while those can be useful too, it is refreshing to me to be able to come and read book-quality explanations about topics that are incredibly relevant to my job and the way that I can improve the company that I work for.
@Matt – I don't expect my response to change or even address your needs, but I wanted to speak up on behalf of those, like myself, who find the very things you complain about to be the very reasons we like this site so much.
Matt: Wonderful feedback, thank you.
I want to write shorter posts, but there are two challenges: 1. I don't have enough time to focus on shortness. My bad. 2. I believe short posts sacrifice on nuance and small details, and they are what ensure success or failure.
Overall my hope is to try to make things simple. Some of them are easy to do, others very hard. If I can get people to understand the nuances and textures, because those key elements are simply stated, then they'll be able to execute better.
I'll keep trying to find an optimum path, while trying to write for the most relevant audience.
Chris, Kalani: I appreciate you both adding your perspectives as well!
Avinash.
Hi Avanish, I very much enjoyed your article… thanks for writing. You raise some very good points regarding the true questions that should be asked.
That being said, I wonder how one who is typically tasked with writing reports would be able to answer some of the questions you pose given the tools at their disposal (database, reporting tools): "How can I improve revenue by 15 percent in the next three months from our website? What are the most productive inbound traffic streams and which sources are we missing?"
The answers to those questions would seem to require external data not typically available to the average report writer. I think the "business honcho" should certainly be asking those questions, but I can't see how the typical analyst would answer those given the nature of data they would have access to.
Thoughts?
Andrew: You are right, a entry level analyst will not have access to some of the data required to answer this question, they might not even have the power to ask that question. They are after all just a report creator.
The thought to explore is: Do they always want to be a report creator, or do they want to progress in their career?
If the answer to the question is yes, then the limitations with data access or the power of the job is less relevant.
My recommendation is to create every single report the low-level analyst is asked to create. But add 10% more to it. If people want you to just create a report of landing page bounce rates, throw in Page Value too (because PV helps you understand how much revenue each page drives!). If you are asked to just report conversion rates, apply a couple of smart Acquisition or Behavior segments (more here: http://goo.gl/ghBz05) to the report and make it better.
So on and so forth. One day you are making the report smarter, and the next day (ok over a few months) you will get asked to answer more important questions and you are on your way to not being a low-level report writer!
-Avinash.
Hi Avinash,
This is 2017 and your article still applicable and will be forever with the concept your wrote.
One question here though it might sound silly. Is there a template of questions to ask business/marketers when they come with any report request.
I'm an Marketing Analyst and at times becomes very challenging asking them(marketers/business/Bosses) so many questions back and fro to know their exact requirement.
Thanks and btw I'm a keen follower of your blog.
Cheers
Yasam
Yasam: I'm afraid I don't have a template of questions.
I've defined the process, and that is what we usually implement and follow.
Avinash.
Thanks Avinash
Your process definitely works, and I'll try to make a template out of it based on our businesses.
Cheers
Yasam