Identify The Known Unknowns: Leverage Analytics Custom Alerts

Most of the time spent by Marketers & Analysts tends to be spend looking for "known knowns".

Things we know and expect to see in the data, we look to see if they are there. "Oh look Google is still our Number 1 referrer and we are selling lots of product x as we always do. Yea!"

Some of our time is spent reacting to the "known unknowns". Looking for things we know might be happening but don't know when they happen. "I would like to know when conversion rate dips below q%, let me go see if that happened last week."

None of it is spent looking for the "unknown unknowns"…. mostly because it is a hard problem to solve. But one that is important for Omniture and WebTrends and Coremetrics and other tools to solve. "I did not even know 20% of our customers were from Australia and that 9 days ago they all stopped coming to our site."

[For one approach to solving the unknown unknowns problem, and source of this framework, please see the second video in this blog post: Analytics Becomes Intelligent. Hello Insights!]

I believe that actions taken based on web analytics data dramatically increase when we shift from our obsession with the known knows to the known unknowns.

From: "Oh my God I did not know that metric had crashed for that segment!! If only I had known that I would have taken action sooner."

To: "Thank goodness I had an alert in my inbox about that big drop yesterday, I'm off to fix landing pages for that segment. No I can't talk to you about Desperate Housewives, I have to go take action!"

And you know what? That is easier to accomplish than you might think.

All you have to do is use the built in Custom Alerts feature in your web analytics tool (and every single tool worth its salt now has one, so you have no excuse not to use it!).

How does it work?

You want to know when something of value happened. But you don't want to hunt and peck at data. You want to be poked with a stick that it happened. You need. . . .

Being told when to look at important things you can take action on, sounds magical and revolutionary? It is. :)

In this blog post I want to share some alerts with you with the hope that it'll spark your creativity.

I also want to hear from those of you who have already use this feature. What is your favorite alert in Omniture? What is the one alert that you created in WebTrends that saved your job? What is the first alert you create for a client, and why?

But before we go jump into the alerts pool naked and all excited…

A Prerequisite:

There is one important reason custom alerts are not used more, or when used they provide little value: A lack of focus on the important.

Many of us toiling away in the field on the front line are just tasked with producing "numbers", or fulfilling certain contractual reports production expectation.

So the alerts we end up creating might be on random things, guesses, what we feel might be important or, again, random things. If you triggers alerts based on that you shouldn't be surprised no action gets taken.

Worse to impress our bosses we might spam everyone with alerts and it takes only a few days for people to configure their email filters to send all your alerts directly into spam.

Please do not underestimate how horrible this problem is.

So what's the fix?

You want the known unknowns right? Ask people around you what they want to know that is important to the business, but currently unknown.

You are asking what the business objectives are, you are asking for the goals, you are asking about targets.

In short you need to leverage the Web Analytics Measurement Framework. . .

See how important alerts to identify the known unknows just pop out at you right away?

If you don't put in the effort, as a in-house employee or as a outside Consultant, to go through the process of working out the Web Analytics Measurement Framework you will fail at this.

Spend time with your HiPPO's and Clients. Spend time with the Marketers. Spend time with people who have the power to take action. Ask all these people what's important but they don't know.

That'll give your effort the focus that will guarantee action.

You skip the above process and all you are doing is self foreplay that will yield nothing (except frustration).

A Helpful Tip For Increased Success:

In championing a rethink of how we all approach our segmentation strategy in our web analytics tools I had recommended a Web Analytics Segmentation Selector Framework.

It advocated identifying actionable insights by focusing on three key activities:

1. Acquisition 2. Behavior 3. Outcomes!

Do the same thing with your custom alerts.

Rather than creating all kinds of alerts, they are easy to create, go through the exercise recommended in the segmentation post and focus your energy on the 1. the top priorities and 2. things decision makers might action.

In web analytics it is never ok to not focus on the most important. It is especially criminal behavior if that waste of time and life is cause by you firing off "alerts".

Remember the tale about the boy who cried wolf? Don't be that.

Creating Custom Alerts:

You have your objectives, goals and targets squared away. You are not going to boil the ocean, you are going to focus on identifying the known unknowns in 3 key buckets, for things people care about.

Now, finally (!), it's time to get down to business!

It is not very difficult to create custom alerts. If you use Google Analytics in the left navigation click on Intelligence, then click on the link that says Create new alert. If you are using Site Catalyst or Yahoo! Web Analytics etc please check your user manual.

Let me walk you through a simple one.

You've convinced the HiPPO's that Twitter is where it is. Their response: "Meh!" But you have permission to tweet a storm away, but not during work hours. So you set out to do this as a hobby, but you know you are right, and while you don't want to spend looking at every twitter visit, you want to be alerted when twitter revenue shoots up!

Step one is to choose your primary alert settings. . . .

Give your alert a name. In this case High Twitter Revenue (since you are already adding campaign tracking parameters) to your tweet urls.

With Google Analytics you can apply this to one of your websites or all of 'em or just to a selected few. Quite convenient.

Choose the period for which the data will be analyzed. In this case you want to know the moment glory is achieved. You can also choose Week or Month.

Finally choose (with the check box) if you want to be emailed or for the alert to just be noted in analytics.

So far easy right?

Step two is choosing the sweet settings. . . .

You choose the dimension you are interested in. There are a bunch to choose from. New vs. returning visitors, countries, campaigns, a particular page someone came from or a page someone landed on your site etc. Depending on the tool you use you might have fewer or more options.

I choose Source and the Value I use is twitter.com.

Note the Condition in the middle. Quite important. You can choose Matches exactly or does not contain or ends with or whatever. This one box can be your shining moment or the start of your embarrassment, choose carefully.

Now for the last step. . . .

Choose the metric you want to focus on.

If this is your first alert, or the first few, try as hard as you can to focus on activity #3, Outcomes. That is what people care about the most. Try to resist, for now, the temptation to alert based on visits or time on site or % of new visits. They are nice and all but really…. no. :)

I choose the metric I like as an outcome on my blog (remember a non-ecommerce website!): Per Visit Goal Value.

Now the KEY PART!

For my value I choose 2. There is a lot of thinking behind that.

Not only do I want to prove Twitter brings in revenue, that would be easy. I want to prove that my efforts with Twitter are so magnificent that they will knock your pants off.

So I don't just have a alert set up, it is set up to cross a high bar. My average Per Visit Goal Value is $1.14. My alert is set to be triggered at $2.

You don't win people over by just meeting some averages, you win them by being big and brave. Keep that in mind when you create alerts.

Ok lecture over and as it turns out I am done with my first alert!

Click Save Alert, do a little jiggy, wait for glory.

When it comes, when you've cleared the high bar, it will look like this:

If you did not opt for your email to be sent in then it will look something like this in your web analytics reports:

Now you know when an unknown that you might not specifically be looking for has occurred and you can, as the email says above, partake in "happy analyzing"!

[Note: If you use Google Analytics make sure you use Annotations to add a quick note with your victories directly on the graph. These Annotations can be shared with others and now when they login they'll also say: "Ohhh that Jennifer is so smart, getting us so many wins, we need to promote her!" Video: Analytics Annotations.]

Ideas For Cool Custom Alerts:

The important word in "custom alerts" is the word custom. As in what you will end up creating will be custom to your business, based on what's important to you.

But I want to close this post with some ideas for alerts I have created recently. My hope is simply to spark your creativity as you use this cool feature.

#1: "Head" Keyword by Bounce Rate.

The "head" of your search terms consists of a few keywords that bring in very large amounts of traffic. A very prudent alert is one that keeps an eye on any ups or downs of these ten or so keywords.

I have set the bounce rate around 10% higher than what it actually is because every little increase in this bounce rate is bad for me, and I want to know that.

If you are running very specific search campaigns whose goal is to attract lots of new visits, then set up a alert for that.

If you, God forbid, are trying to get more page views for people who come from Bing, then set up an alert for that. [Note: The god forbid is for the metric not for Bing!]

Focus: Acquisition. Success: Initial goal met or not.

#2: Campaign by "Things of Real Value".

These are my favorite kinds of alerts.

Far too often we are obsessed with conversion rates in an eCommerce context. Why not focus on things that actually matter, things that might indicate real success or failure?

Like Average Order Value. Or Quantity (of items)?

Here's an alert I create, all the time, to set a higher bar of accountability for my campaigns (especially when I have a lot of people / resources dedicated to them):

Tell me when some email campaigns I am running cause an unusual spike in the number of items ordered. I want to know what I am doing right there.

In this case I am focusing on one specific campaign, you could focus on all your email campaigns to allow you to identify the diamond in the rough quickly.

#3: New Visitor by Revenue (Increase).

Making money from our existing customers is important, but getting better at convincing new customers to do business with us is important as well (especially in the context of the fact that we shamefully ignore all our existing customers and focus all the time on getting new ones!).

I like an alert like this one:

Tell me when I have an amazing increase in my daily revenue (not conversion!) from New Visitors when compared to same day in the previous week.

I have set a high enough bar for revenue, a 20% increase, before I am distracted by an email. Note also I have been careful to compare like week days, I don't really want to compare Sundays to Saturdays (for obvious reasons).

As soon as I get the alert I go look at an advanced segment I have already created for New Visitors to dive deeper into the sources (campaigns, direct, search) that might have seen this revenue spurt, the pages or products on my site that are doing well. All to learn what I should do more of.

Of if you apply the condition "% decreases by more than" then things you should stop doing!

#4: Source by Time on Site (Customer Behavior).

I am a movie studio. I have trailers for my movie. I have a blogging strategy. I would like to know when parts of that strategy are causing buzz and word of mouth and viral and …. pick your fav phrase. :)

Here is one small alert:

Thanks to your clever use of event tracking you are able to capture time spent watching the movie trailer optimally. The above alert will show you if there are any sites with the word blog in their name that sent visitors that watched your entire movie trailer (a rare occurrence! :)).

NOTE: Now I know that referral path contains blog will not capture all the blogs (like this one!). Remember this is just to spark your creativity.

#5: Country by Huge Visits.

I don't syndicate the content of my blog. But I did give Sidney permission a little while back to translate some of them into Chinese (like this one). He does a wonderful job.

Almost all of the success of my posts at China Web Analytics will be measured by Sidney, his increased readership or comments or rss subscribers or (sadly) number of times it is copied (pirated?) and posted without his permission on many many other blogs.

But there is a small amount of success for this effort that I can measure.

Do I get any traffic from these posts?

I don't know when it happens (a known unknown!) but I have set up an alert to let me know if there is a big improvement in Visits in context of my current 1,200 averagevisits from China…

When this alert is fired off, perhaps in sync with Sidney's publication of my posts, I'll know syndication was a good idea (on this small measure of success).

You can do the same if you have goals / priorities that are geographically focused.

Flip the coin…. and let's say you are the awesome South American giant Mercado Livre and you depend on the US for a good chunk of business.. you can set up custom alerts to know when traffic from the US or Florida or Miami takes a nose dive.

Consider that alert as insurance that if something broke in your online marketing strategy that you will find it quickly.

In Conclusion:

Custom alerts enhance your ability to find surprises in your data, things you might not be expecting.

If you start by using the Web Analytics Measurement Framework it will help bring a focus on what's important to your execution. If you use the Segmentation Selection Framework you'll find that it brings a discipline to your approach.

I hope the above five examples inspire you to go give the feature a whirl, regardless of the web analytics tool you use because all of 'em have it.

Your Turn!

I have barely scratched the surface of what is possible. How do you use custom alerts? Has an alert you had set up saved your bacon? Does your tool provide a particularly clever option? Do you have a best practice you want to recommend?

Share your ideas for custom alerts (for any type of website, using any tool)!

Thanks.

PS:
Couple other related posts you might find interesting:

• Move Beyond The Top Ten. Find Love!
• Leverage Custom Reports For Better Insights!
• Consultants: Present Impactful Analysis, Insightful Reports
• Excellent Analytics Tip#4: Make Your Analysis/Reports "Connectable"